Sunday, October 02, 2022

Linux-PREEMPT_RT Scheduler latency measurement with Oscilloscope


For those exploring Realtime capabilities of Linux using Preempt-RT patch, Here is a bootable sdcard-image for Beaglebone-Black(or Green) with fully preemptible kernel. This image includes all required libraries/utils/binaries and kernel-debug-capabilities required for evaluating Linux-Preempt-RT behavior. As a bonus, this image includes a gpio-test-program that mirrors an input signal to a gpio-output pin for measuring the scheduler latency using an Oscilloscope(Thanks to Maxime Chevallier of Bootlin for providing this demo-utility during PREEMPT_RT Training).

Setup Diagram: 


Required Items:

  1. Beaglebone-Black or Green
  2. +5V adapter
  3. micro-sdcard(1Gb or higher)
  4. Programmable-signal-generator-board
  5. 2-Channel-Oscilloscope
  6. Jumper-wires
Instructions:
  1. Prepare the setup as shown above(As a starter, set signal generator frequency to 100Hz with 50% duty)
  2. Download sdcard-beaglebone-preemptrt-demo.img.xz (image size is just under 25MB)
  3. Write sdcard-beaglebone-preemptrt-demo.img.xz to micro-sdcard using Balena-Etcher-Application.
  4. Remove the micro-sdcard from your PC and insert in Beaglebone and power ON the +5V
  5. After Linux boot, you will be able to see two square waves with a small phase difference between them(yellow waveform is the input to beaglebone and blue waveform is output from the beaglebone).
  6. Phase difference between yellow and blue signal indicates the latency introduced by scheduler while scheduling preemptrt-gpiotest program
  7.  On startup, /etc/init.d/S99PreemptrtGpioTester invokes "chrt -r 99 /usr/sbin/preemptrt-gpiotest gpiochip0 28 gpiochip0 17 h p" - which means starting preemptrt-gpiotest utility with priority 99. 

What is the purpose of this image?
For a quick evaluation of Linux-PREEMPT_RT behavior(or as a training material) one case use this readymade image to avoid long build process from sources using buildroot.
Using this image, you can study and tune your application to have deterministic timing behavior of the Linux scheduler at different load conditions. preemptrt-gpiotest program included in this image, mirrors a gpio-input to an output and the phase difference between these two signals gives an idea on how fast scheduler is able to give the required context to your application and the jitter(when measured with persist mode of the scope) on the output signal indicates the best and worst case scenario of the scheduler latency. Following picture shows best and worst case latency of the scheduler when subjected to different load on the hw(e.g network-traffic/interrupts/etc). For further details on this topic, check here to access Bootlin's PREEMPT_RT training material.




How can I build this bootable Linux-Image for beaglebone from sources?
Detailed build instructions are available here on my git-repository.

How do I modify and run my customized preemptrt-gpio utility on this beaglebone's target Linux?
Cross-compile instructions are available here on my git-repository.







3 comments:

Anonymous said...

Great article! Is the response time just 3 kHz? What speed runs the cpu on?

Maxime Chevallier said...

Hello Albert,

That's a very neat article ! Thanks for the credit :)

You made my day :)

Maxime

Anonymous said...

This post is a gem, bookmarking it for future reference. Clicking at lightning speed thanks to this exhilarating click test.