Towards Unit Testing RealTime Schedulers in LITMUSRT

1
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• Mac Mollison
• Björn Brandenburg
• James H. Anderson

2
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• What is LITMUSRT?
• What is this talk about?
• What is a typical scheduling policy?
• Why do we need a test tool?
• How do we test? (Answer Unit Testing)
• What are the specific tests?

3
LITMUSRT
LInux Testbed for MUltiprocesor Scheduling in
Real-Time Systems
Patch to Linux 2.6.24 Kernel
Before the patch
Linux schedulers
CFS
SCHED_RR
SCHED_FIFO
4
LITMUSRT
LInux Testbed for MUltiprocesor Scheduling in
Real-Time Systems
Patch to Linux 2.6.24 Kernel
After the patch
Linux schedulers
LITMUSRT schedulers
CFS
G-EDF

SCHED_RR
C-EDF
SCHED_FIFO
P-EDF
PD2
5
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• What is LITMUSRT?
• What is this talk about?
• What is a typical scheduling policy?
• Why do we need a test tool?
• How do we test? (Answer Unit Testing)
• What are the specific tests?

6
Overview

• What LITMUSRT is
• Why we want to test LITMUSRT schedulers
• Implementing real-time schedulers is nontrivial
  bugs can be subtle
• How to test LITMUSRT schedulers
• Unit Testing - testing small pieces of code
  programmatically with a twist

7
My Work

• Developed specification for a tool to test
  schedulers
• Implemented prototype of the tool for the G-EDF
  scheduling policy

8
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• What is LITMUSRT?
• What is this talk about?
• What is a typical scheduling policy?
• Why do we need a test tool?
• How do we test? (Answer Unit Testing)
• What are the specific tests?

9
Sporadic Task Model
WCET
Period
Release
Deadline
T1 (2,4)
Job
Completion
Time (ms)
Execution on CPU
10
Global Scheduling Policies
Ti
Ti
Ti
Ti
Ti
Ti
Ti
Ti
Ti
CPU 3
CPU 4
CPU 2
CPU 1
M CPUs
11
Global Earliest Deadline First(G-EDF)
T1 (2,3)
At all times, the M jobs with earliest deadlines
should be executing.
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
12
Global Earliest Deadline First(G-EDF)
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
13
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• What is LITMUSRT?
• What is this talk about?
• What is a typical scheduling policy?
• Why do we need a test tool?
• How do we test? (Answer Unit Testing)
• What are the specific tests?

14
Why Develop a Test Tool?

• Code is very complex
• We cannot easily tell if the code is correct
• We need help debugging

15
(No Transcript)
16
Why Develop a Test Tool?

• Code is very complex
• We cannot easily tell if the code is correct
• We need help debugging
• Resulting schedules are very complex
• We cannot easily tell if correct schedules are
  produced

17
Global Earliest Deadline First(G-EDF) ???
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
18
Global Earliest Deadline First(G-EDF)
Incorrect!
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
19
Global Earliest Deadline First(G-EDF)
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
20
Global Earliest Deadline First(G-EDF)
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
21
Why Develop a Test Tool?

• Code is very complex
• We cannot easily tell if the code is correct
• We need help debugging
• Resulting schedules are very complex
• We cannot easily tell if correct schedules are
  produced
• We need to minimize overhead
• Detailed regression testing is necessary

22
Overhead
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
23
Overhead
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
24
(No Transcript)
25
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• What is LITMUSRT?
• What is this talk about?
• What is a typical scheduling policy?
• Why do we need a test tool?
• How do we test? (Answer Unit Testing)
• What are the specific tests?

26
The Challenge
Without overhead, and if we did not need very
detailed feedback, we could check the
invariant At all times, the M jobs with earliest
deadlines should be executing.
27
The Challenge
Instead, we have to use a series of specific
tests to detect anomalies in scheduling and to
measure overhead.
28
Unit Testing
A series of specific tests with detailed
feedback that can be produced after each code
revision suggests Unit Testing
29
Unit Testing
A series of specific tests with detailed
feedback that can be produced after each code
revision suggests Unit Testing
Unit Testing programmatically
testing small
modules of code after each revision
30
Unit Testing
A series of specific tests with detailed
feedback that can be produced after each code
revision suggests Unit Testing
Unit Testing programmatically
testing small
modules of code after each revision
We test recorded schedule traces instead
31
Feather-Trace
Event records
Release
Switch To
Switch Away
Completion
32
Towards Unit Testing Real-Time Schedulers in
LITMUSRT

• What is LITMUSRT?
• What is this talk about?
• What is a typical scheduling policy?
• Why do we need a test tool?
• How do we test? (Answer Unit Testing)
• What are the specific tests?

33
Unit Tests
Unit Tests
Deadline Test
Sporadic Task Model Test
Completion Test
G-EDF Decision Test
G-EDF Latency Test
34
Unit Tests
Unit Tests
Deadline Test
Useful for any scheduling policy
Sporadic Task Model Test
Completion Test
G-EDF Decision Test
G-EDF Latency Test
35
Deadline Test

• Did all jobs complete by their deadlines?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
36
Deadline Test

• Did all jobs complete by their deadlines?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
37
Deadline Test

• Did all jobs complete by their deadlines?

What about soft real-time?
T1 (2,3)
Scheduler developers can specify maximum
acceptable tardiness.
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
38
Sporadic Task Model Test

• Were job releases separated by at least the
  period of the task?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
39
Sporadic Task Model Test
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
40
Sporadic Task Model Test
Scheduler developers can specify a tolerance
variable
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
41
Completion Test

• Did all released jobs actually complete?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
42
Completion Test

• Did all released jobs actually complete?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
43
Unit Tests
Unit Tests
Deadline Test
Sporadic Task Model Test
Completion Test
Check for G-EDF adherence in an overhead-agnostic
manner
G-EDF Decision Test
G-EDF Latency Test
44
G-EDF Decision Test

• Are jobs switched to execution
• in EDF order?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
45
G-EDF Decision Test

• Are jobs switched to execution
• in EDF order?

Test algorithm models execution state to check
for correct decisions.
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
46
G-EDF Decision Test

• Are jobs switched to execution
• in EDF order?

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
47
G-EDF Decision Test

• Are jobs switched to execution
• in EDF order?

Incorrect?
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
48
G-EDF Decision Test
This cannot be distinguished from overhead.

• Are jobs switched to execution
• in EDF order?

Incorrect?
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
49
Unit Tests
Unit Tests
Deadline Test
Sporadic Task Model Test
Completion Test
G-EDF Decision Test
Helps ensure acceptable amounts of overhead
G-EDF Latency Test
50
G-EDF Latency Test

• Measures latency

T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
51
G-EDF Latency Test
Test algorithm can use sequence of events on each
processor to determine type of latency

• Measures latency

Type 1
Type 2
Type 3
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
52
G-EDF Latency Test
Type 1
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
53
G-EDF Latency Test
Type 2
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
54
G-EDF Latency Test
Type 3
T1 (2,3)
T2 (3,5)
T3 (4,8)
Time (ms)
Execution on CPU1
Execution on CPU2
Overhead
55
Summary

• What LITMUSRT is
• Why we want to test LITMUSRT schedulers
• Implementing real-time schedulers is nontrivial
  bugs can be subtle
• How to test LITMUSRT schedulers
• Unit Testing - testing small pieces of code
  programmatically with a twist

56
Questions?
Towards Unit Testing Real-Time Schedulers in
LITMUSRT