Event Name

1
(No Transcript)
2
ESP Program Verification Of Millions of Lines of
Code

• Manuvir Das
• Researcher
• PPRC Reliability Team
• Microsoft Research

3
Motivation
4
Approach

• Redundency is good
• Redundancy exposes inconsistency
• Inconsistency points to errors
• Compare
• what programmer should do
• what her code actually does

5
Lightweight specifications

• Rules
• Describe correct behavior
• Readable/writable by programmers
• Specify limited properties
• not total correctness/verification
• Compare rules against code

6
Types are rules

• Programmers use types to
• document interface syntax
• represent program abstractions
• Types are written, read and checked
• routine part of development process
• Why are types successful?
• types are lightweight specifications
• type checking is fast routine
• errors are found early, at compile-time

7
Can we extend this approach?

• Specify and check other properties
• languages to express rules
• tools to check that code obeys rules
• Goal is partial correctness
• detect and report important classes of errors
• no guarantee of program correctness
• Systematic tools of various flavors
• compile-time verifiers and bug-finders
• run-time monitors and fault injectors
• document generators

8
Rule-basedprogramming
Rules
Development
Testing
Static Verification Tool
Read for understanding
Drive testing tools
Precise Rules
New API rules
Program Analysis Engine
Source Code
9
ESP
Rules
ESP
OPAL Rules
Path-sensitive Dataflow Analysis
C/C Code
10
Requirements

• Scalability
• Complete coverage
• Millions of lines of code
• All features of C/C
• Usability
• Low number of false positives
• Simple rule description language
• Informative error reports

11
The bottom line

• Can ESP verify a million lines of code?
• Were not sure . yet
• Weve done 150 KLOC in 70s and 50MB
• So, were cautiously optimistic

12
Are we running into a wall?

• Verification demands precision
• Need to minimize false error reports
• Must analyze each execution path
• Big programs demand scalability
• Exponentially/infinitely many paths
• Cannot analyze each execution path
• Must use approximate analysis

13
Research problem

• Can we invent a verification method that
• is always conservative,
• is always scalable,
• is almost always precise, and
• matches our intuition?
• Yes, for a certain class of rules
• Finite state, temporal safety properties

14
Finite state safety properties

• Property is described by an FSA
• As the program executes, a monitor
• tracks the current state of the FSA
• updates the current state
• signals an error when the FSA transitions into
  special error states
• Goal of verification
• Is there some execution path that would cause the
  monitor to signal an error?

15
Example stdio usage in gcc
void main () if (dump) fil
fopen(dumpFile,w) if (p) x 0 else
x 1 if (dump) fclose(fil)
void main () if (dump) Open if (p)
x 0 else x 1 if (dump)
Close
16
Path-sensitive property analysis

• Symbolically evaluate the program
• Track FSA state and execution state
• At branch points
• Execution state implies branch direction?
• Yes process appropriate branch
• No split state and process both branches

17
Example
Closed
CloseddumpT
OpeneddumpT
OpeneddumpT,pT
OpeneddumpT,pF
OpeneddumpT,pT,x0
OpeneddumpT,pF,x1
OpeneddumpT,pT,x0
OpeneddumpT,pF,x1
CloseddumpT,pT,x0
CloseddumpT,pF,x1
18
Dataflow property analysis

• Track only FSA state
• Ignore non-state-changing code
• At control flow join points
• Accumulate FSA states

19
Example
Closed
Closed,Opened
Closed,Opened
Error,Closed,Opened
20
Why is this code correct?
void main () if (dump) Open if (p)
x 0 else x 1 if (dump)
Close
21
When is a branch relevant?

• Precise answer
• When the value of the branch condition determines
  the property FSA state
• Heuristic answer
• When the property FSA is driven to different
  states along the arms of the branch statement

22
Property simulation

• Modification of path-sensitive analysis
• At control flow join points
• States agree on property FSA state?
• Yes merge states
• No process states separately

23
Example
Closed
OpeneddumpT
CloseddumpF
OpeneddumpT
CloseddumpF
CloseddumpT
CloseddumpF
CloseddumpT
Closed
24
Loop example
entry
new old
Closed
Closednewold1
Open
Openednewold

T
T
Close
F
new
new ! old
Closednewold1
Openednewold
F
Close
exit
Closednewold
25
Making property simulation work

• Real programs are complex
• Multiple FSAs
• Aliasing
• Real code bases are very large
• Well beyond a million lines
• ESP
• Property Simulation Multiple FSAs
• Aliasing Component-wise Analysis

26
Problem Multiple FSAs
void main () if (dump1) fil1
fopen(dumpFile1,w) if (dump2) fil2
fopen(dumpFile2,w) if (dump1)
fclose(fil1) if (dump2) fclose(fil2)
27
Property simulation, bit by bit

• Problem property state can be exponential
• Solution track one FSA at a time

void main () if (dump1) Open if
(dump2) ID if (dump1) Close if
(dump2) ID
void main () if (dump1) ID if
(dump2) Open if (dump1) ID if
(dump2) Close
28
Property simulation, bit by bit

• One FSA at a time
• Avoids exponential property state
• Fewer branches are relevant
• Lifetimes are often short
• Smaller memory footprint
• Embarassingly parallel
• - Cannot correlate FSAs

29
Problem Aliasing
void main () if (dump1) fil1
fopen(dumpFile1,w) if (dump2) fil2
fopen(dumpFile2,w) fil3 fil1 if
(dump1) fclose( fil3 ) if (dump2)
fclose( fil2 )
30
ESP Model Values Have State

• During execution, the program
• creates stateful values
• changes the state of stateful values
• The programmer defines
• how values are created (syntactic patterns)
• how values change state (syntactic patterns)
• Syntactic expressions are aliases for values

31
OPAL Rule Descriptions

• Object Property Automata Language

State Closed State Opened State Error Initial
Event Open _object_ ASTFUNCTIONCALL
ASTSYMBOL fopen _anyargs_
Event Close ASTFUNCTIONCALL
ASTSYMBOL fclose _object_ Transition _
- Opened on Open Transition Opened - Closed on
Close Transition Closed - Error on Close File
already closed
32
Parameterized transitions
void main () if (dump1) fil1
fopen(dumpFile1,w) if (dump2) fil2
fopen(dumpFile2,w) fil3 fil1 if
(dump1) fclose( fil3 ) if (dump2)
fclose( fil2 )
33
Parameterized transitions
void main () if (dump1) t1
fopen(dumpFile1,w) Open(t1) fil1 t1
if (dump2) t2 fopen(dumpFile2,w)
Open(t2) fil2 t2 fil3 fil1 if
(dump1) fclose( fil3 ) Close(fil3)
if (dump2) fclose( fil2 ) Close(fil2)

34
Expressions are value aliases
void main () if (dump1) t1
fopen(dumpFile1,w) Open(t1) fil1 t1
if (dump2) t2 fopen(dumpFile2,w)
Open(t2) fil2 t2 fil3 fil1 if
(dump1) fclose( fil3 ) Close(fil3)
if (dump2) fclose( fil2 ) Close(fil2)

35
Value-alias analysis

• Is expression e an alias for value v?
• ESP uses GOLF to answer this query
• Generalized One Level Flow
• Context-sensitive
• Largely flow-insensitive
• Millions of lines of code, in seconds

36
Putting it all together

• Property simulation
• Identify and track relevant execution state
• Syntactic patterns value-alias analysis
• Identify and isolate individual FSAs
• One FSA at a time
• Bit vector analysis for safety properties

37
Case study stdio usage in gcc

• cc1 from gcc version 2.5.3 (Spec95)
• Does cc1 always print to opened files?
• cc1 is a complex program
• 140K non-blank, non-comment lines of C
• 2149 functions, 66 files, 1086 globals
• Call graph includes one 450 function SCC

38
Skeleton of cc1 source
FILE f1, , f15 int p1, , p15 void
compileFile() if (p1) f1 fopen()
if (p15) f15 fopen() restOfComp()
if (p1) fclose(f1) if (p15)
fclose(f15)
void restOfComp() if (p1) printRtl(f1)
if (p15) printRtl(f15)
restOfComp() void printRtl(FILE f)
fprintf(f)
39
OPAL rules for stdio usage
State Uninit State Closed State Opened State
Error Initial Event Decl ASTDECLARATION
_object_ ASTSYMBOL _any_ Initial
Event Open _object_ ASTFUNCTIONCALL
ASTSYMBOL fopen _anyargs_ Event
Print ASTFUNCTIONCALL ASTSYMBOL
fprintf _object_,_anyargs_ Event Close
ASTFUNCTIONCALL ASTSYMBOL
fclose _object_ Transition _ - Uninit on
Decl Transition _ - Opened on Open Transition
Uninit - Error on Print File not
opened Transition Opened - Opened on
Print Transition Closed - Error on Print
Printing to closed file Transition Opened -
Closed on Close Transition Closed - Error on
Close File already closed
40
Experimental results

• Precision
• Verification succeeds for every file handle
• No transitions to Error no false errors
• Scalability
• Ave. per handle 72.9 seconds, 49.7 MB
• Single 1GHz PIII laptop with 512 MB RAM
• We have proved that
• Each of the 646 calls to fprintf in the source
  code prints to a valid, open file

41
Ongoing research

• Path-sensitive value-alias analysis
• Value-alias sets
• Expressions that hold tracked value
• Track value-alias sets during simulation
• Add value-alias sets to property state
• When things get complicated, use GOLF
• Component-wise analysis
• Identify and analyze components
• Link using less precise analysis