Where%20Testing%20Fails%20

1
Where Testing Fails
2
Problem Areas

• Stack Overflow
• Race Conditions
• Deadlock
• Timing
• Reentrancy

3
Stack Overflow

• Stack
• local variables
• parameter passed, return address
• register saved
• scratch pad for a compiler
• Problem
• the size of a stack is nondeterministic
• the memory is limited

4
Stack Overflow

• Static analysis of stack depth
• use a call tree
• if an ISR use its own stack, add them at the
  leaves
• stack depth of the ISR
• space to save the context of the running thread
• stack used by RTOS
• Problem
• source code is necessary
• run time library is difficult to analyze
• Java, C entails hidden stacks

5
Stack Analysis
6
Stack Analysis

• What if you dont have the source code?
• Measuring Stack size
• fill a large area with -1 for stacks
• Recovery
• protect important code in protected pages
• stack resizing
• Stack Usage Information
• code, library, compiler change
• Life critical systems need formal proof OR
  sufficient stack space

7
Race Conditions

• major source of bugs!!!
• extern float shared_sensor void
  Update_Sensor(float offset, float
  scale)    float temp get_raw()    temp
  scale    temp offset    shared_sensortemp
  
• old value may be read by another thread
• if bus is not 32-bit, partial value may be read

8
Race Condition

• fixing the problem is easy
• mutex
• use a message instead of shared data
• finding them is extremely difficult
• find shared data
• find if there are conflict accesses to it
• what about a pointer?
• ISR is inside the kernel

9
Deadlock

• mutex/messages may lead to deadlock
• conditions
• mutex
• nonpreemption
• hold and wait
• circle
• Solutions
• break any one of the conditions

10
Deadlock

• there are many heuristics
• when you wait on a blocked thread, release what
  you have
• acquire all locks before it begins
• acquire locks in a preset order
• detecting and recovery is expensive
• static detection is not practical
• every wait should be recorded to find a circle
• abort a thread (which one?)

11
Timing Problem

• Priority Inversion
• Processor Utilization
• worst case analysis is more important (tbl 1)
• Schedulability bound
• is it important?
• WCE time
• cache effect
• interrupts may be double counted
• I/O, paging, system call, malloc, library calls,
  blocking, .
• for hard real time, remove these factors if
  possible

12
Interrupts and events

• Execution of each ISR needs to be analyzed
• Response time for an event
• interrupt latency (hw latency context s/w)
• WCE time of the ISR
• what if the CPU was sleeping?
• what if the interrupt was masked out?
• More general system incorporates the event queue
• events are stored in a queue for later processing

13
Others

• Watchdog
• a timer to reboot when the system get into a
  crazy state
• issue frequent false positives
• Reentrancy
• do not modify anything local
• do not cause device state changes
• mostly caused by ISR