Title: Efficient Software Model Checking of Soundness of Type Systems
1Efficient Software Model Checking of Soundness of
Type Systems
- Michael Roberson, Melanie Harries,
- Paul T. Darga, Chandrasekhar Boyapati
- University of Michigan
2Type Soundness
- All Possible Programs
- defined by the syntax
- Good Programs
- do not cause certain kinds of runtime errors
- e.g. memory errors, race conditions, etc.
- defined by the semantics
3Type Soundness
- Type System
- designed to statically prevent errors
- must accept only good programs
- Well Typed Programs
- defined by the type system
4Type Soundness
- Type Soundness
- well typed implies good
- entire class of errors eliminated at compile time
5Type Soundness
- Much research in new type systems
- Proving type soundness is important
- Becoming its own research area
- POPLmark workshop
- Proving type soundness is tedious
- Manual proofs
- Machine checked proofs
- Both require intensive human effort
- Our technique is completely automatic
- Our technique provides counterexamples
6Outline
- Motivation
- Example
- Approach
- Experimental Results
- Related Work
- Conclusions
7Example
- Syntax
- e true false 0 succ e pred e iszero
e if e1 then e2 else e3
Semantics
e ? e
succ e ? succ e
e ? e
pred e ? pred e
pred 0 ? 0
pred succ nv ? nv
e ? e
iszero e ? iszero e
iszero 0 ? true
iszero succ nv ? false
e1 ? e1
if e1 then e2 else e3 ? if e1 then e2 else e3
if true then e2 else e3 ? e2
if false then e2 else e3 ? e3
Final States v true false nv nv
0 succ nv
Good program iszero 0
Bad program iszero true
8Example
- Type system
- T Nat Bool
- Typing rules
true Bool
false Bool
0 Nat
e Nat
succ e Nat
e Nat
pred e Nat
e Nat
iszero e Bool
e1 Bool e2 T e3 T
if e1 then e2 else e3 T
Type soundness Every well typed program is a
good program.
9Soundness Proof
- Parts to a Soundness Proof
- Progress Theorem
- If wellTyped(t) then t is a final state or t
transitions to t - Preservation Theorem
- If wellTyped(t) and t transitions to t then
wellTyped(t) - Together, these two theorems imply Type Soundness
10Software Model Checking
- Software Model Checking
- For each well typed program state
- Check Progress
- Check Preservation
- If a check fails, show a counterexample
11Challenges
- Search space is infinite!
- Impose finite bounds on the search space
- Small Scope Hypothesis justifies this approach
- Jackson96Offutt00Marinov03
- Search space is huge!
- Find similarities and eliminate redundancy
12State Space Reduction
- Many software model checkers
- Verisoft, JPF, CMC, SLAM, Blast, Magic,
- Many state space reduction techniques
- Partial order reduction
- Predicate abstraction
- Not effective for checking type soundness
13Our Key Idea
if iszero 0 then true else false
PRUNED
14Outline
- Motivation
- Example
- Approach
- Defining the language
- Searching the state space
- Representing the state space
- Reducing the state space
- Experimental Results
- Related Work
- Conclusions
15Language Definition
class ExpressionLanguage implements Language
static class Expression int kind //
TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF
_at_Tree Expression e1, e2, e3
_at_Declarative boolean wellTyped() return
syntaxOk() (kind TRUE gt true
) (kind
FALSE gt true )
(kind ZERO gt true
) (kind SUCC gt
e1.wellTyped() e1.type() INT)
(kind PRED gt e1.wellTyped() e1.type()
INT) (kind ISZERO gt
e1.wellTyped() e1.type() INT)
(kind IF gt e1.wellTyped() e1.type()
BOOL
e2.wellTyped() e3.wellTyped()
e2.type() e3.type())
Expression smallStep() throws StuckException
if (isValue()) return this if
(!e1.isValue()) e1 e1.smallStep() return
this if (kind PRED
e1.kind ZERO ) return e1 if (kind
PRED e1.kind SUCC ) return e1.e1
if (kind ISZERO e1.kind ZERO ) return
True() if (kind ISZERO e1.kind
SUCC ) return False() if (kind IF
e1.kind TRUE ) return e2 if (kind
IF e1.kind FALSE) return e3
throw new StuckException() // Helper
Functions Expression root
_at_Declarative public boolean wellTyped() return
root.wellTyped() public void smallStep()
throws StuckException root root.smallStep()
public boolean isFinalState() return
root.isValue()
16Language Definition
class ExpressionLanguage implements Language
static class Expression int kind //
TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF
_at_Tree Expression e1, e2, e3
_at_Declarative boolean wellTyped() return
syntaxOk() (kind TRUE gt true
) (kind
FALSE gt true )
(kind ZERO gt true
) (kind SUCC gt
e1.wellTyped() e1.type() INT)
(kind PRED gt e1.wellTyped() e1.type()
INT) (kind ISZERO gt
e1.wellTyped() e1.type() INT)
(kind IF gt e1.wellTyped() e1.type()
BOOL e2.wellTyped()
e3.wellTyped() e2.type()
e3.type()) Expression smallStep()
throws StuckException if (isValue())
return this if (!e1.isValue()) e1
e1.smallStep() return this if
(kind PRED e1.kind ZERO ) return e1
if (kind PRED e1.kind SUCC )
return e1.e1 if (kind ISZERO e1.kind
ZERO ) return True() if (kind ISZERO
e1.kind SUCC ) return False() if
(kind IF e1.kind TRUE ) return e2
if (kind IF e1.kind FALSE)
return e3 throw new StuckException()
// Helper Functions Expression
root _at_Declarative public boolean wellTyped()
return root.wellTyped() public void
smallStep() throws StuckException root
root.smallStep() public boolean
isFinalState() return root.isValue()
static class Expression int kind // TRUE,
FALSE, ZERO, SUCC, PRED, ISZERO, IF _at_Tree
Expression e1, e2, e3
17Language Definition
class ExpressionLanguage implements Language
static class Expression int kind //
TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF
_at_Tree Expression e1, e2, e3
_at_Declarative boolean wellTyped() return
syntaxOk() (kind TRUE gt true
) (kind
FALSE gt true )
(kind ZERO gt true
) (kind SUCC gt
e1.wellTyped() e1.type() INT)
(kind PRED gt e1.wellTyped() e1.type()
INT) (kind ISZERO gt
e1.wellTyped() e1.type() INT)
(kind IF gt e1.wellTyped() e1.type()
BOOL e2.wellTyped()
e3.wellTyped() e2.type()
e3.type()) Expression smallStep()
throws StuckException if (isValue())
return this if (!e1.isValue()) e1
e1.smallStep() return this if
(kind PRED e1.kind ZERO ) return e1
if (kind PRED e1.kind SUCC )
return e1.e1 if (kind ISZERO e1.kind
ZERO ) return True() if (kind ISZERO
e1.kind SUCC ) return False() if
(kind IF e1.kind TRUE ) return e2
if (kind IF e1.kind FALSE)
return e3 throw new StuckException()
// Helper Functions Expression
root _at_Declarative public boolean wellTyped()
return root.wellTyped() public void
smallStep() throws StuckException root
root.smallStep() public boolean
isFinalState() return root.isValue()
Expression smallStep() throws StuckException
if (isValue()) return this if (!e1.isValue())
e1 e1.smallStep() return this if
(kind PRED e1.kind ZERO ) return e1
if (kind PRED e1.kind SUCC ) return
e1.e1 if (kind ISZERO e1.kind ZERO )
return True() if (kind ISZERO e1.kind
SUCC ) return False() if (kind IF
e1.kind TRUE ) return e2 if (kind IF
e1.kind FALSE) return e3 throw new
StuckException()
18Language Definition
class ExpressionLanguage implements Language
static class Expression int kind //
TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF
_at_Tree Expression e1, e2, e3
_at_Declarative boolean wellTyped() return
syntaxOk() (kind TRUE gt true
) (kind
FALSE gt true )
(kind ZERO gt true
) (kind SUCC gt
e1.wellTyped() e1.type() INT)
(kind PRED gt e1.wellTyped() e1.type()
INT) (kind ISZERO gt
e1.wellTyped() e1.type() INT)
(kind IF gt e1.wellTyped() e1.type()
BOOL e2.wellTyped()
e3.wellTyped() e2.type()
e3.type()) Expression smallStep()
throws StuckException if (isValue())
return this if (!e1.isValue()) e1
e1.smallStep() return this if
(kind PRED e1.kind ZERO ) return e1
if (kind PRED e1.kind SUCC )
return e1.e1 if (kind ISZERO e1.kind
ZERO ) return True() if (kind ISZERO
e1.kind SUCC ) return False() if
(kind IF e1.kind TRUE ) return e2
if (kind IF e1.kind FALSE)
return e3 throw new StuckException()
// Helper Functions Expression
root _at_Declarative public boolean wellTyped()
return root.wellTyped() public void
smallStep() throws StuckException root
root.smallStep() public boolean
isFinalState() return root.isValue()
_at_Declarative boolean wellTyped() return
syntaxOk() (kind TRUE gt true
) (kind FALSE
gt true )
(kind ZERO gt true
) (kind SUCC gt e1.wellTyped()
e1.type() INT) (kind PRED gt
e1.wellTyped() e1.type() INT) (kind
ISZERO gt e1.wellTyped() e1.type() INT)
(kind IF gt e1.wellTyped()
e1.type() BOOL
e2.wellTyped() e3.wellTyped()
e2.type() e3.type())
19Approach
- Defining the language
- Searching the state space
- Representing the state space
- Reducing the state space
20Searching
- Select an unchecked well typed state
if
iszero
if
false
0
false
false
true
21Searching
- Check progress and preservation
- Progress Run one small step
if
iszero
if
false
void smallStep() throws StuckException
0
false
false
true
if
true
if
false
false
false
true
22Searching
- Check progress and preservation
- Preservation Check if the result is well typed
if
iszero
if
false
0
false
false
true
if
_at_Declarative boolean wellTyped()
true
if
false
false
false
true
23Searching
- Identify and prune similar states
PRUNED
24Search Algorithm
Let W be the well typed state space While W is
not empty Choose a state s in W Run one step of
evaluation on s If Progress or Preservation
fail Print out s as a bug trace Let P be the
set of states similar to s W W P
25Approach
- Defining the language
- Searching the state space
- Representing the state space
- Reducing the state space
26Representing the State Space
- Represent a set as a boolean formula
- Encode each field as bits (b0, b1, )
- Constrain the bits using boolean operations
n1
n1.kind IF n1.e1.kind TRUE
(b0 ? b1 ? b2) ? b3 ? (b6 ? b7 ? b8)
n2
At AST height 5, a formula representing
2,504,790,381 well typed states is only 24,117
terms long.
27Representing the State Space
- Initialize to the set of well typed states
- Construct a formula describing the type system
_at_Declarative boolean wellTyped() return
syntaxOk() ( kind TRUE gt true
) ( kind FALSE
gt true ) (
kind ZERO gt true
) ( kind SUCC gt e1.wellTyped()
e1.type() INT) ( kind PRED gt
e1.wellTyped() e1.type() INT) ( kind
ISZERO gt e1.wellTyped() e1.type() INT)
( kind IF gt e1.wellTyped()
e1.type() BOOL e2.wellTyped()
e3.wellTyped() e2.type() e3.type()
)
boolean formula
28Representing the State Space
- Initialize to the set of well typed states
- Construct a formula describing the type system
- Declarative methods
- May not use object creations, assignments, or
loops - May use gt, forall, exists operators
- Declarative methods translate to small formulas
_at_Declarative boolean wellTyped() return
syntaxOk() ( kind TRUE gt true
) ( kind FALSE
gt true ) (
kind ZERO gt true
) ( kind SUCC gt e1.wellTyped()
e1.type() INT) ( kind PRED gt
e1.wellTyped() e1.type() INT) ( kind
ISZERO gt e1.wellTyped() e1.type() INT)
( kind IF gt e1.wellTyped()
e1.type() BOOL e2.wellTyped()
e3.wellTyped() e2.type() e3.type()
)
29Search Algorithm
Let W be the well typed state space While W is
not empty Choose a state s in W Run one step of
evaluation on s If Progress or Preservation
fail Print out s as a bug trace Let P be the
set of states similar to s W W P
Use a SAT solver
Add P to the SAT solver
30Approach
- Defining the language
- Searching the state space
- Representing the state space
- Reducing the state space
- Dynamic analysis
- Static analysis
- Other optimizations
31Dynamic Analysis
- Discover and prune states that are similar
- Touched analysis
- Track which fields are accessed
- Symbolic execution
- Track which fields are accessed and how they are
used
root.kind IF root.e1.kind ISZERO
root.e1.e1.kind ZERO
(root.kind IF root.kind ISZERO
root.kind SUCC root.kind PRED)
root.e1.kind ISZERO root.e1.e1.kind ZERO
32Static Analysis
33Static Analysis
class WhyStaticAnalysis private boolean a,
b public void smallStep() a
!a _at_Declarative public boolean wellTyped()
return a b
34Static Analysis
class WhyStaticAnalysis private boolean a,
b public void smallStep() a
!a _at_Declarative public boolean wellTyped()
return a b
Choose a well typed state
35Static Analysis
class WhyStaticAnalysis private boolean a,
b public void smallStep() a
!a _at_Declarative public boolean wellTyped()
return a b
Choose a well typed state
Check the transition
36Static Analysis
class WhyStaticAnalysis private boolean a,
b public void smallStep() a
!a _at_Declarative public boolean wellTyped()
return a b
Choose a well typed state
Check the transition
Prune similar transitions
well typed
PRUNED
37Static Analysis
- Dynamic analysis alone is not always correct!
- Need to perform additional checks
- Could check each pruned transition separately
- Negates the benefits of pruning
- We use static analysis to ensure correctness
- All pruned transitions checked together
- Any error within finite bounds is caught
38Static Analysis
class WhyStaticAnalysis private boolean a,
b public void smallStep() a
!a _at_Declarative public boolean wellTyped()
return a b
wellTyped (a b)
Prestate of a pruned transition
wellTypedPre (a b)atrue true
wellTypedPost (a b)afalse b
wellTypedPre ? wellTypedPost
b
Poststate of the pruned transition
Not valid when b false!
39Static Analysis
- Partially evaluate wellTyped formula
- Use SAT solver to check the above implication
- If the implication holds then pruning is sound
- If not, we have a counterexample
- Often, wellTypedPre and wellTypedPost are
identical - Checking the implication is trivial then
wellTypedPre
wellTypedPost
?
40Other Optimizations
- Isomorphism pruning
- Prune more when checking heaps
- Handling term cloning
- Perform symbolic copy
- Handling term substitution
- Perform incremental substitution
41Outline
- Motivation
- Example
- Approach
- Experimental Results
- Related Work
- Conclusions
42Experimental Results
- We verified type soundness of five languages
- Expression Language
- Imperative Language
- Featherweight Java
- Mini Java
- Mini Java with Ownership Types
43Experimental Results
- Imperative Language
- Simple language from Winskel93, Chapter 2
- Features variables, assignment, conditionals,
loops - Features arithmetic, relational, logical operators
int a a 5 while (a lt 7) a a1
44Experimental Results
- Featherweight Java Igarashi99
- Subset of Java
- Features classes, inheritance, methods, fields
class A extends Object A() super()
class B extends Object B() super()
class Pair extends Object Object fst
Object snd Pair(Object fst, Object snd)
super() this.fstfst this.sndsnd Pair
setfst(Object newfst) return new
Pair(newfst, this.snd)
45Experimental Results
- Mini Java
- Extends Featherweight Java
- Allows mutation of objects
- Features an explicit heap
- Features integers and booleans
class Node extends Object int value
Node(int value) super() this.value
value Node mutate() return
this.value this.value 1
46Experimental Results
- Mini Java with Ownership Types
- Clarke98Boyapati03Aldrich02
- Types are parameterized with owners
- Enforces object encapsulation
class NodeltA,Bgt extends ObjectltAgt ObjectltBgt
data NodeltA,Bgt next Node(ObjectltBgt data,
NodeltA,Bgt next) super() this.data data
this.next next class StackltA,Bgt extends
ObjectltAgt Nodeltthis,Bgt root
Stack(Nodeltthis,Bgt root) super() this.root
root StackltA,Bgt push(ObjectltBgt data)
return this.root new Nodeltthis,Bgt(data,
this.root)
47Experimental Results
Benchmark Max Expression Size well typed States (approx.) States Checked Time (s)
Expression Language 1 2 3 4 13 40 121 364 1093 3280 21 22 22 24 210 231 293 2280 2841 22523 1 3 3 5 11 17 23 29 35 41 0.068 0.093 0.105 0.122 0.246 0.551 1.376 3.633 10.833 38.543
IMP 1 2 3 4 5 6 7 15 31 63 127 255 511 23 25 28 29 212 214 217 236 275 2157 2327 2679 21402 1 7 11 19 34 34 34 61 96 147 230 377 652 0.102 0.185 0.256 0.408 0.710 0.739 0.816 2.158 5.107 10.066 21.013 52.208 331.138
Checks 2157 well typed IMP programs in about 10
seconds
48Experimental Results
Benchmark Max Expression Size States Checked Time (s)
Featherweight Java 1 2 3 4 5 21 85 341 3 7 9 9 13 70 298 1210 1.148 1.594 1.650 1.899 2.151 6.905 43.756 475.022
Mini Java 1 2 3 4 5 21 85 341 5 21 40 53 59 275 1133 4565 2.721 3.117 3.897 5.750 6.191 37.354 342.435 5981.114
Mini Java with Ownership Types 1 2 3 4 5 21 25 29 33 13 73 110 135 157 733 877 1021 1165 50.818 77.135 103.230 231.328 247.954 2760.734 3963.836 5271.509 6255.260
49Experimental Results
- We inserted 25 errors into Mini Java with
Ownership Types
Max Expression Size Percentage of Errors Caught
1 2 3 4 5 6 7 8 0 8 40 68 76 80 84 100
50Experimental Results
- We inserted 25 errors into Mini Java with
Ownership Types
Max Expression Size Percentage of Errors Caught Time to Check the Entire Search Space (s)
1 2 3 4 5 6 7 8 21 25 29 33 0 8 40 68 76 80 84 100 100 100 100 100 50.818 77.135 103.230 231.328 247.954 339.632 403.249 836.568 2760.734 3963.836 5271.509 6255.260
51Related Work
- State space reduction techniques
- Abstraction refinement SLAM Blast Magic
- Partial order reduction Godefroid97 Flanagan05
- Heap canonicalization Musuvathi05 Iosif02
- Symmetry reduction Ip93
52Related Work
- Software model checkers
- Verisoft Godefroid97
- Java PathFinder Visser00
- CMC Musuvathi02
- Bandera Corbett00
- Bogor Dwyer05
- SLAM Ball01
- Blast Henzinger02
- Magic Chaki03
- Jalloy Vaziri03
- Miniatur Dolby07
53Conclusions
- Fully automatic technique to check type soundness
- Counterexamples provided if soundness fails
- Useful to check small to medium sized languages
- Research languages
- Core subsets of complex languages