CSE 565 Software Testing in RealTime Systems

1
CSE 565Software Testing in Real-Time Systems

• A Guest Lecture on
• Model Checking Introduction
• Hai Huang

2
(No Transcript)
3
Agenda

• Why we need model checking
• Purpose of model checking
• The process of model checking
• Technical issues
• Model checking techniques
• Model checkers

4
Why We Need Model Checking

• What is model checking
• Model checking is an automatic technique for
  verifying finite state concurrent systems
  Clarke et al., 2002
• Features of model checking
• Automatic
• Exhaustive verification
• Mathematic proof

5
Model Checking vs. Testing

• Testing
• Not fully automatic
• Need human intelligence to develop test cases
• Can be partially automated test automation
• Design / Execution / Data collection / Data
  analysis
• Test automation focus on test execution
• Not exhaustive
• Not a mathematic proof

6
Example Deadlocks

• How to use testing to detect deadlocks
• Design a test case to detect deadlocks
• Input / Expected output
• What could be the possible expected output for
  deadlocks?
• How to observe such expected output?
• The execution terminates?
• The execution does not terminate within 1 hour?
• The execution does not terminate within 10,000
  hours?

7
Finite State Automata

• Model checking verifies finite state automata
• Finite state automata
• States and transitions
• The number of states is finite
• The number of transitions is finite

Unlock
Locked
Unlocked
Lock
8
Deadlocks (Cont.)

• How to use model checking to detect deadlocks
• Deadlock states
• A state with no outgoing transitions
• Detecting deadlock states
• Reachability checking whether a deadlock state
  is reachable in the state transition diagram
• Reachable deadlock can happen
• Unreachable deadlock cannot happen

9
Dining Philosopher

• Simplified version two philosophers
• Every philosopher needs two forks for dining
• Only two forks available on the dining table
• Deadlock happens when each philosopher has one
  fork

10
Dining Philosopher FSA
11
Summary

• Model Checking
• Automatic
• Exhaustive explore all reachable states
• Mathematic proof

• Testing
• Manual test case design
• Not exhaustive
• No mathematic proof

12
Summary (Cont.)

• Model checking
• Require a formal model
• UML? FA?
• Require theoretical training
• Verify the model

• Testing
• Require the executable
• Source code
• Less theoretical training required
• Validate the real system

13
Purpose of Model Checking

• Safety requirements
• Malicious events never occur
• Deadlock
• Non-progressiveness
• Customer defined malicious events or states
• Fairness requirements
• An activity is executed eventually
• An activity is executed infinitely often
• Automatic machine checking
• Complex (concurrent) system is hard for manual
  checking

14
Process of Model Checking

• Modeling
• Develop a formal model of the system such that
  rigid computation can be conducted over the model
• Property specification
• Specify the to-be-checked problem
• Deadlock, non-progressiveness, and others?
• Checking
• Feed in the model and the properties into the
  model checker and run the checking

15
Modeling

• Finite state automata
• Control flow automata
• Modeling software programs
• Control flow diagram
• Some existing model checkers can automatically
  extract a finite state automata from the control
  flow automata

16
Control Flow Automata

• Two types of edges
• Assignment assign the result of an expression to
  a variable
• Models the assignment statements in a program
• Assumption the edge is taken only if the
  associated condition is satisfied
• Models if-then-else, switch-case, for-loop,
  while-loop, do-loop, etc.

17
Sample CFA

• C code
• include ltassert.hgtint main()    int x 0,
  y 0    y y 1    while ( x gt 0 )   
          x x y        y y - 1   

Assign x ? 0
Assign y ? 0
Assign y ? y 1
Assume x ? 0
Assume x lt 0
Assign x ? x y
Assume True
Assign y ? y - 1
18
Property Specification

• LTL and CTL
• LTL Linear temporal logic
• CTL Computation tree logic
• Both are formal specifications of temporal
  (ordering) properties
• Assertions
• Models the ASSERT statements in the software
  programs

19
LTL

• LTL is build up from propositions and connectives
• Boolean logic ?, ?, ?
• Temporal logic N (Next), G (Always), F
  (Eventually), U (Until)
• LTL can express the temporal relations among
  activities and events along an execution path

20
LTL Formulas

• LTL Formulas
• A single proposition p
• Next state N p
• Always G p
• Finally F p
• Until p U q

p
p
p
p
p
p
p
p
p
p
q
21
Example LTL Formula

• Following the Dining Philosopher problem
• A deadlock state is either A pick left, B pick
  left or A pick right, B pick right
• No deadlock is expressed as
• G ? ( (A pick left ? B pick left) ?
• (A pick right ? B pick right) )

22
CTL

• CTL formulas
• CTL express temporal properties over a tree,
  instead of path
• Execution may have branches
• Path quantifiers
• For all paths A
• Exist one path E

23
Example CTL Formulas

• A ( G p )

• E ( G p )

p
p
p
p
p
p
p
p
p
p
p
24
Assertions

• Assertions model the ASSERT statements in the
  programs
• At a location of the control flow, some condition
  must hold
• Intuitive to software engineers
• Most LTL and CTL properties can be expressed by
  one or multiple assertions

25
Example Assertions

• C code
• include ltassert.hgtint main()    int x 0,
  y 0    y y 1    while ( x gt 0 )   
          x x y        y y - 1   
• ASSERT ( x gt 0 )

• A ( G ( x ? 0 ) )
• int main()    int x 0, y 0    y y
  1
• ASSERT ( x gt 0 )     while ( x gt 0
  )            x x y
• ASSERT ( x gt 0 )         y y -
  1 ASSERT ( x gt 0 )
•    
• ASSERT ( x gt 0 )

26
Technical Issues

• State explosion
• M component, each has N states. The system will
  have NM states, exploding exponentially
• Infinite state space
• Number of transitions increases in the same scale
• Running time
• Storage
• The focus of model checking techniques is to
  reduce the complexity of the system

27
Model Checking Methods

• Symbolic execution
• Reachability checking
• During execution, mark each reachable state, and
  check whether malicious states are reached
• Fixpoint
• A fixpoint is the maximal set of reachable states
• Function is determined by the to-be-checked
  properties
• Check whether fixpoint subsumes malicious states
• Automata theory
• Languages of the model and the properties
• Check whether the intersection of two automata
  produces empty language
• Büchi automata for LTL and SPIN

28
Fixpoint

• Fixpoing
• x f(x)
• Computing fixpoint for reachability checking
• X is a set of reachable states
• F(X) is the set of states reachable from X
• A fix point F?(X) is the set of all reachable
  states

I
I
f(I)
f(f(I))
T f(T)


29
Model Checking Techniques (Cont.)

• OBDD
• Compact diagrams representation of states
• Partial order reduction
• Exploiting independent events, the order of which
  does not affect model checking
• Compositional reasoning
• Properties of subsystems imply that of the entire
  system
• Abstraction
• Abstract data (variable / value pairs) by
  predicates (Boolean variable / truth value pairs)
• Symmetry
• Equivalence relations among identical copies of
  subsystems
• Induction
• Invariant process representing parameterized
  copies of identical subsystems

30
BDD Binary Decision Diagram
31
Partial Order Reduction
pq or qp
s
r
32
Predicates Abstraction

• State explosion problem
• The state space grows exponentially when
  composing a system by components
• Infinite state space
• Model a non-negative integer variable salary
• Abstract many states with one predicate
• salary ? 10 abstract 11 states 0, 1, , 10
• salary ? 20 abstract infinite states 20, 21,

33
Example

• Suppose we select two predicates to abstract the
  state space
• a gt 0 and b 0
• Three-Valued Model
• 0 means the predicate is false
• 1 means the predicate is true
• means the predicate is unknown
• With the two predicates, we can prove all ASSERT
  statements

• int a, b
• b 0
• if ( a gt 0 )
• ASSERT( a gt b )
• else
• ASSERT( a lt b )

a lt 0
a ? Z, b ? Z
a ? Z, b 0
a gt 0, b 0
a ? 0, b 0
34
Summary
35
Model Checkers

• SPIN
• BLAST
• Java PathFinder
• Blade

36
SPIN

• SPIN targets software systems, instead of
  hardware systems. The constructs of PROMELA
  clearly defines this.
• Starting with version 4, SPIN provides direct
  support for embedding C code into the PROMELA
  model specifications. This leverages the usage
  toward implementation level software systems.
• SPIN supports fully all properties expressible by
  LTL.
• SPIN supports dynamic creation and deletion of
  processes.
• SPIN supports message passing and shared memory
  as communication protocols.
• SPIN support a simulator mode, allowing user
  guided simulation.
• Available at http//spinroot.com/spin/whatispin.h
  tml

37
BLAST

• Developed by University of California at Berkeley
• Accept C program as input
• Assertion checking
• Available at http//www-cad.eecs.berkeley.edu/bla
  st/

38
Java PathFinder

• Developed by NASA
• Accept Java source code as input
• Translate Java code into PROMELA and use SPIN as
  backend
• Dynamic creation of objects with data and methods
• Class inheritance
• Thread and synchronization primitives for
  modeling monitors (synchronized statements, and
  the wait and notify methods)
• Exceptions
• Thread interrupts
• Available at http//ase.arc.nasa.gov/havelund/jpf.
  html

39
Blade

• Developed by Arizona State University
• Accept CFA as input
• Assertion checking
• Better performance than BLAST
• Running time
• Storage usage
• Available at http//asusrl.eas.asu.edu/blade/

40
Next ClassService Collaboration and Verification