SLAM :Software Model Checking From Theory To Practice

1
SLAM Software Model Checking From Theory To
Practice

• Sriram K. Rajamani
• Software Productivity Tools
• Microsoft Research

2
People behind SLAM

• MSR
• Tom Ball and Sriram Rajamani
• Summer interns
• Sagar Chaki, Todd Millstein, Rupak Majumdar
  (2000)
• Satyaki Das, Wes Weimer, Robby (2001)
• Jakob Lichtenberg, Mayur Naik (2002)
• Shuvendu Lahiri, Jakob Lichtenberg, Georg
  Weissenbacher (2003)
• Visitors
• Giorgio Delzanno, Andreas Podelski, Stefan
  Schwoon
• Windows Partners
• Byron Cook, Vladimir Levin
• Abdullah Ustuner, John Henry, Con McGarvey, Bohus
  Ondrusek
• Nar Ganapathy

3
Agenda

• Specifying and checking software
• SLAM overview
• Lessons

4
Software Validation

• Large scale reliable software is hard to build
  and test.
• Different groups of programmers write different
  components.
• Integration testing is a nightmare.

5
Property Checking

• Programmer provides redundant partial
  specifications
• Code is automatically checked for consistency
• Different from proving whole program correctness
• Specifications are not complete

6
Interface Usage Rules

• Rules in documentation
• Incomplete, unenforced, wordy
• Order of operations data access
• Resource management
• Disobeying rules causes bad behavior
• System crash or deadlock
• Unexpected exceptions
• Failed runtime checks

7
Does a given usage rule hold?

• Checking this is computationally impossible!
• Equivalent to solving Turings halting problem
  (undecidable)
• Even restricted computable versions of the
  problem (finite state programs) are prohibitively
  expensive

8
Why bother?

• Just because a problem is undecidable, it doesnt
  go away!

9
Scientific curiosity

• Undecidability and complexity theory are the most
  significant contributions of theoretical computer
  science.
• Software property checking, a very practical and
  pressing problem is undecidable.

10
Automatic property checking Study of tradeoffs

• Soundness vs completeness
• Missing errors vs reporting false alarms
• Annotation burden on the programmer
• Complexity of the analysis
• Local vs Global
• Precision vs Efficiency
• Space vs Time

11
Broad classification

• Underapproximations
• Testing
• After passing testing, a program may still
  violate a given property
• Overapproximations
• Type checking
• Even if a program satisfies a property, the type
  checker for the property could still reject it

12
Current trend

• Confluence of techniques from different fields
• Model checking
• Automatic theorem proving
• Program analysis
• Significant emphasis on practicality
• Several new projects in academia and industry

13
Model Checking

• Algorithmic exploration of state space of the
  system
• Several advances in the past decade
• symbolic model checking
• symmetry reductions
• partial order reductions
• compositional model checking
• bounded model checking using SAT solvers
• Most hardware companies use a model checker in
  the validation cycle

14

• enum N, T, C state1..2
• int turn
• init
• state1 N state2 N
• turn 0
• trans
• statei N turn 0 -gt statei
  T turn i
• statei N turn !0 -gt statei
  T
• statei T turn i -gt statei
  C
• statei C state2-i N -gt statei
  N
• statei C state2-i ! N -gt statei
  N turn 2-i

15
N1,N2 turn0
N noncritical, T trying, C critical
16
Model Checking

• Strengths
• Fully automatic (when it works)
• Computes inductive invariants
• I such that F(I) ? I
• Provides error traces
• Weaknesses
• Scale
• Operates only on models
• How do you get from the program to the model?

17
Theorem proving

• Early theorem provers were proof checkers
• They were built to support asssertional reasoning
  in the Hoare-Dijkstra style
• Cumbersome and hard to use
• Automatic theorem provers used desicision
  procedures for restricted theories
• Theory of equality with uninterpreted functions
• Theory of lists
• Theory of linear arithmetic
• Combination of the above !
• e.g. Nelson-Oppen provers are widely used
• ESC, ESC-Java
• Proof Carrying Code

18
Theory of Equality.

• Symbols , ¹, f, g,
• Axiomatically defined

E E
E2 E1
E1 E2
E1 E2 E2 E3
E1 E3
E1 E2
f(E1) f(E2)

• Example of a satisfiability problem
• g(g(g(x)) x ? g(g(g(g(g(x))))) x
  ? g(x) ¹ x
• Satisfiability problem decidable in O(n log n)

19

• a array 1..len of int
• int max -MAXINT
• i 1
• ? 1 ? j ? i. aj ? max
• while (i ? len)
• if( ai gt max)
• max ai
• i i1
• endwhile
• ? 1 ? j ? len. aj ? max

( ? 1 ? j ? i. aj ? max) ? ( i gt len) ? (?
1 ? j ? len. aj ? max
20
Automatic theorem proving

• Strengths
• Handles unbounded domains naturally
• Good implementations for
• equality with uninterpreted functions
• linear inequalities
• combination of theories
• Weaknesses
• Hard to compute fixpoints
• Requires inductive invariants
• Pre and post conditions
• Loop invariants

21
Program analysis

• Originated in optimizing compilers
• constant propagation
• live variable analysis
• dead code elimination
• loop index optimization
• Type systems use similar analysis
• Are the type annotations consistent?

22
Program analysis

• Strengths
• Works on code
• Pointer aware
• Integrated into compilers
• Precision efficiency tradeoffs well studied
• flow (in)sensitive
• context (in)sensitive
• Weaknesses
• Abstraction is hardwired and done by the designer
  of the analysis
• Not targeted at property checking (traditionally)

23
Model Checking, Theorem Proving and Program
Analysis

• Very related to each other
• Different histories
• different emphasis
• different tradeoffs
• Complementary, in some ways
• Combination can be extremely powerful

24

• What is the key design challenge in a model
  checker for software?
• It is the model!

25
Model Checking Hardware

• Primitive values are booleans
• States are boolean vectors of fixed size
• Models are finite state machines !!

26
Characteristics of Software

• Primitive values are more complicated
• Pointers
• Objects
• Control flow (transition relation) is more
  complicated
• Functions
• Function pointers
• Exceptions
• States are more complicated
• Unbounded graphs over values
• Variables are scoped
• Locals
• Shared scopes
• Much richer modularity constructs
• Functions
• Classes

27
Traditional approach
model checker
FSM
Finite state machines
Source code
Sequential C program
28
Automatic abstraction
SLAM
model checker
Data flow analysis implemented using BDDs
Finite state machines
Push down model
FSM
Boolean program
abstraction
C data structures, pointers, procedure calls,
parameter passing, scoping,control flow
Source code
Sequential C program
29
Computing power doubles every 18
months -Gordon Moore

• An optimizing compiler doubles performance every
  18 years
• -Todd Proebsting

30

• When I use a model checker, it runs and runs for
  ever and never comes back when I use a static
  analysis tool, it comes back immediately and says
  I dont know
• - Patrick Cousot

31
Agenda

• Specifying and checking software
• SLAM overview
• Lessons

32
Rules
Static Driver Verifier
Development
Testing
Source Code
33
SLAM Software Model Checking

• SLAM innovations
• boolean programs a new model for software
• model creation (c2bp)
• model checking (bebop)
• model refinement (newton)
• SLAM toolkit
• built on MSR program analysis infrastructure
• c2bp and newton are written in OCAML
• bebop is written in C

34
SLIC

• Finite state language for stating rules
• monitors behavior of C code
• temporal safety properties
• familiar C syntax
• Suitable for expressing control-dominated
  properties
• e.g. proper sequence of events
• can encode data values inside state

35
State Machine for Locking
Rel
Acq
Unlocked
Locked
Rel
Acq
Error
36
CallDriver
start NP
IRP completion state machine
SKIP2
SKIP1
return child status
Skip
IPC
CallDriver
synch
MPR3
NP
CallDriver
prop completion
PPC
not pending returned
MPR completion
Complete request
CallDriver
MPR1
MPR2
DC
return not Pend
no prop completion
synch
CallDriver
N/A
N/A
IRP accessible
CallDriver
start P
SKIP2
Mark Pending
SKIP1
Skip
IPC
CallDriver
synch
MPR3
NP
CallDriver
return Pending
prop completion
PPC
not pending returned
MPR completion
Complete request
CallDriver
MPR1
MPR2
DC
no prop completion
CallDriver
N/A
37
The SLAM Process
boolean program
c2bp
prog. P
prog. P
slic
bebop
SLIC rule
predicates
path
newton
38
Example
Does this code obey the locking rule?
do KeAcquireSpinLock() nPacketsOld
nPackets if(request) request
request-gtNext KeReleaseSpinLock() nPackets
while (nPackets ! nPacketsOld) KeRelease
SpinLock()
39
Example
Model checking boolean program (bebop)
do KeAcquireSpinLock() if() KeRe
leaseSpinLock() while () KeReleaseSpin
Lock()
U
L
L
L
U
L
U
L
U
U
E
40
Example
Is error path feasible in C program? (newton)
do KeAcquireSpinLock() nPacketsOld
nPackets if(request) request
request-gtNext KeReleaseSpinLock() nPackets
while (nPackets ! nPacketsOld) KeRelease
SpinLock()
U
L
L
L
U
L
U
L
U
U
E
41
Example
Add new predicate to boolean program (c2bp)
b (nPacketsOld nPackets)
do KeAcquireSpinLock() nPacketsOld
nPackets b true if(request) request
request-gtNext KeReleaseSpinLock() nPackets
b b ? false while (nPackets !
nPacketsOld) !b KeReleaseSpinLock()
U
L
L
L
U
L
U
L
U
U
E
42
Example
Model checking refined boolean program (bebop)
b (nPacketsOld nPackets)
do KeAcquireSpinLock() b true
if() KeReleaseSpinLock() b b ?
false while ( !b ) KeReleaseSpinLock
()
U
L
b
L
b
L
b
U
b
!b
L
U
b
L
U
b
U
E
43
Example
Model checking refined boolean program (bebop)
b (nPacketsOld nPackets)
do KeAcquireSpinLock() b true
if() KeReleaseSpinLock() b b ?
false while ( !b ) KeReleaseSpinLock
()
U
L
b
L
b
L
b
U
b
!b
L
U
b
L
b
U
44
Observations about SLAM

• Automatic discovery of invariants
• driven by property and a finite set of (false)
  execution paths
• predicates are not invariants, but observations
• abstraction model checking computes inductive
  invariants (boolean combinations of observations)
• A hybrid dynamic/static analysis
• newton executes path through C code symbolically
• c2bpbebop explore all paths through abstraction
• A new form of program slicing
• program code and data not relevant to property
  are dropped
• non-determinism allows slices to have more
  behaviors

45
Some bugs found with SDV

• Ran on DDK 3677
• Overnight run
• 4 processors

46
What kinds of bugs can SDV find?

• Example driver Parallel port driver
• Lines of code 35k
• Example rule DoubleCompletion
• Summary Checks that driver dispatch routines do
  not call IoCompleteRequest() twice on the I/O
  request packet passed to it by the OS or another
  driver

47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
Call 1
59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
Call 2
67
What kinds of bugs can SDV find?

• Example driver Floppy disk controller
• Lines of code 10k
• Example rule NullDevobjForwarded
• Summary Checks that driver dispatch routines do
  not call IoCallDriver or PoCallDriver on a null
  device object pointer

68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
many steps later .....................
78
(No Transcript)
79
(No Transcript)
80
FdcStartDevice is supposed to initialize a device
object pointer in the device extension
81
(No Transcript)
82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
(No Transcript)
87
(No Transcript)
88
(No Transcript)
89
many steps later .....................
90
(No Transcript)
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
Uninitialized pointer passed to FcFdcEnabler
95
(No Transcript)
96
(No Transcript)
97
(No Transcript)
98
(No Transcript)
99
(No Transcript)
100
(No Transcript)
101
(No Transcript)
102
DeviceObjectNULL
103
(No Transcript)
104
Agenda

• Specifying and checking software
• SLAM overview
• Lessons

105
SLAM

• Specifications are like programs
• It is hard to get them right the first time
• They evolve, just like programs
• Tools need to tie specifications to programs
• You can hire people to write them!

106
SLAM

• Boolean program model has proved itself
• Successful for domain of device drivers
• control-dominated safety properties
• few boolean variables needed to do proof or find
  real counterexamples
• Counterexample-driven refinement
• terminates in practice
• incompleteness of theorem prover not an issue

107
What is hard?

• Abstracting
• from a language with pointers (C)
• to one without pointers (boolean programs)
• All side effects need to be modeled by copying
  (as in dataflow)
• Open environment problem

108
What stayed fixed?

• Boolean program model
• Basic tool flow
• Repercussions
• newton has to copy between scopes
• c2bp has to model side-effects by value-result
• finite depth precision on the heap is all boolean
  programs can handle

109
What changed?

• Interface between newton and c2bp
• We now use predicates for doing more things
• refine alias precision via aliasing predicates
• newton helps resolve pointer aliasing imprecision
  in c2bp

110
Scaling SLAM

• Largest driver we have processed has 60K lines
  of code
• Largest abstractions we have analyzed have
  several hundred boolean variables
• Routinely get results after 20-30 iterations
• Out of 672 runs in one set, 607 terminate within
  20 minutes

111
Scale and SLAM components

• Out of 67 runs that time out, tools that take
  longest time
• bebop 50, c2bp 10, newton 5, constrain 2
• C2bp
• fast predicate abstraction (fastF) and
  incremental predicate abstraction (constrain)
• re-use across iterations
• Newton
• biggest problems are due to scope-copying
• Bebop
• biggest issue is no re-use across iterations

112
SLAM Status

• 2000-2001
• foundations, algorithms, prototyping
• papers in CAV, PLDI, POPL, SPIN, TACAS
• March 2002
• Bill Gates review
• May 2002
• Windows committed to hire two people with model
  checking background to support Static Driver
  Verifier (SLAMdriver rules)
• July 2002
• running SLAM on 100 drivers, 20 properties
• September 3, 2002
• made initial release of SDV to Windows (friends
  and family)
• April 1, 2003
• made wide release of SDV to Windows (any internal
  driver developer)

113
What worked well?

• Specific domain problem
• Safety properties
• Shoulders synergies
• Separation of concerns
• Summer interns visitors
• Strategic partnership with Windows

114
Predictions

• The holy grail of full program verification has
  been abandoned. It will probably remain abandoned
• Less ambitious tools like powerful type checkers
  will emerge and become more widely used
• These tools will exploit ideas from various
  analysis disciplines
• Tools will alleviate the chicken-and-egg
  problem of writing specifications

115
Further Reading

• See papers, slides from
• http//research.microsoft.com/slam
• http//research.microsoft.com/sriram

116
Glossary
Model checking Checking properties by systematic exploration of the state-space of a model. Properties are usually specified as state machines, or using temporal logics
Safety properties Properties whose violation can be witnessed by a finite run of the system. The most common safety properties are invariants
Reachability Specialization of model checking to invariant checking. Properties are specified as invariants. Most common use of model checking. Safety properties can be reduced to reachability.
Boolean programs C-like programs with only boolean variables. Invariant checking and reachability is decidable for boolean programs.
Predicate A Boolean expression over the state-space of the program eg. (x lt 5)
Predicate abstraction A technique to construct a boolean model from a system using a given set of predicates. Each predicate is represented by a boolean variable in the model.
Weakest precondition The weakest precondition of a set of states S with respect to a statement T is the largest set of states from which executing T, when terminating, always results in a state in S.