Title: Lazy Abstraction
1Lazy Abstraction
Lecture 2 Modular Analyses
Ranjit Jhala UC San Diego
With Tom Henzinger, Rupak Majumdar, Ken
McMillan, Gregoire Sutre
2Program Verification byLazy Abstraction
Lecture 1
- Ranjit Jhala
- UC San Diego
With Tom Henzinger, Rupak Majumdar, Ken
McMillan, Gregoire Sutre
3Last lecture
- Lazy Abstraction for Sequential Programs
- Predicates
- Abstract infinite program states
- Counterexample-guided Refinement
- Find predicates tailored to prog, property
- Abstraction Expensive
- Reachability Tree
- Refinement Find predicates, use locations
Proof of unsat of TF Interpolation
4This Lecture Modular Analyses
- Procedures
- - Summaries
- Concurrency
- - Thread-Context Reasoning
5An example
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
6Inline Calls in Reach Tree
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
Initial
1
4
2
1,4
1,2
2,4
3,4
2,2
3,2
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
4,4
4,4
4,2
4,2
5
5
3
3
7Inline Calls in Reach Tree
- Problem
- Repeated analysis for inc
- Exploding call contexts
Initial
1
4
2
int x //global f1() 1 x 0 2 if()
f2() 3 else f2() 4 if (xlt0)
ERROR return
1,4
1,2
f2() 1 if() f3() 2 else f3() return
f3() 1 if() f4() 2 else f4() return
2,4
3,4
2,2
3,2
f4() 1 if() f5() 2 else f5() return
fn() 1 x return
4,4
4,4
4,2
4,2
5
5
3
3
2n nodes in Reach Tree
8Inline Calls in Reach Tree
- Problem
- Repeated analysis for inc
- Exploding call contexts
- Cyclic call graph (Recursion)
- Infinite Tree!
Initial
1
4
2
1,4
1,2
2,4
3,4
2,2
3,2
4,4
4,4
4,2
4,2
5
5
3
3
9Solution Procedure Summaries
- Summaries Input/Output behavior
- Plug summaries in at each callsite
- instead of inlining entire procedure
- Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95
- Summary set of (F ? F)
- F Precondition formula describing input state
- F Postcondition formula describing output
state
10Solution Procedure Summaries
- Summaries Input/Output behavior
- Plug summaries in at each callsite
- instead of inlining entire procedure
- Sharir-Pnueli 81, Reps-Horwitz-Sagiv 95,
Ball-Rajamani 01 - Summary set of (F ? F)
- F Precondition formula describing input state
- F Postcondition formula describing output
state
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
- ( sign0 ? rv gt a)
- (sign 0 ? rv lt a)
Q. How to compute, use summaries ?
11Lazy Abstraction Procedure Summaries
Yes
Safe
Abstract
C Program
Refine
No
Property
Trace
Q. How to compute, use summaries ?
12Abstraction with Summaries
main
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
flag!0
2
flag0
ax signflag
sign0
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
13Abstraction with Summaries
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
1
sign0
sign!0
2
flag0
2
sign0
ax signflag
rva1
sign0
4
rvgta
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
Summary ( sign0 ? rvgta),
14Summary Successor
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
1
sign0
ax signflag
2
flag0
2
assume rvgta
3
ygtx
yrv
4
rvgta
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
Summary ( sign0 ? rvgta),
15Abstraction with Summaries
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
1
sign0
flag0
sign0
2
4
flag0
flag0
3
2
3
ygtx
sign0
4
rvgta
yltx
ax signflag
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
Summary ( sign0 ? rvgta),
16Abstraction with Summaries
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
1
1
sign0
sign0
2
4
flag0
flag0
3
3
2
2
3
ygtx
sign0
4
4
rvgta
rvlta
ax signflag
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
Summary ( sign0 ? rvgta),
(sign0 ? rvlta)
17Summary Successor
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
1
1
sign0
sign0
2
4
flag0
flag0
3
3
2
2
3
3
ygtx
yltz
4
4
rvgta
rvlta
ax signflag
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
assume rvlta
yrv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
Summary ( sign0 ? rvgta),
(sign0 ? rvlta)
18Abstraction with Summaries
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
1
1
1
sign0
sign0
2
4
flag0
flag0
3
3
2
2
3
3
ygtx
yltz
4
4
rvgta
rvlta
ygtz
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
Predicates flag0 , ygtx , yltz sign0
, rvgta , rvlta
Summary ( sign0 ? rvgta),
(sign0 ? rvlta)
19Another Call
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR 6
y1 inc(z1,1) 7 if (y1ltz1) ERROR return
1
1
1
sign0
sign0
2
4
flag0
flag0
3
3
2
2
3
3
ygtx
yltz
4
4
rvgta
rvlta
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
6
6
az1 sign1
sign0
Summary ( sign0 ? rvgta),
(sign0 ? rvlta)
Predicates flag0 ,ygtx,yltz, y1gtz1
sign0 , rvgta , rvlta
20Another Call
main
inc
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR 6
y1 inc(z1,1) 7 if (y1ltz1) ERROR return
1
1
1
sign0
sign0
2
4
flag0
flag0
3
3
2
2
3
3
ygtx
yltz
4
4
rvgta
rvlta
Note Predicates are well-scoped
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
6
6
SAFE
az1 sign1
7
y1gtz1
assume rvgta
y1rv
Summary ( sign0 ? rvgta),
(sign0 ? rvlta)
Predicates flag0 ,ygtx,yltz, y1gtz1
sign0 , rvgta , rvlta
21Lazy Abstraction Procedure Summaries
Yes
Safe
Abstract
C Program
Refine
No
Property
Trace
Q. How to find scoped predicates ?
22Traces with Procedure Calls
Trace Formula
Trace
pc1 x1 3 pc2 assume (x1gt0) pc3 x3
f1(x1) pc4 y2 y1 pc5 y3 f2(y2) pc6 z2
z11 pc7 z3 2z2 pc8 return z3 pc9
return y3 pc10 x4 x31 pc11 x5
f3(x4) pc12 assume(w1lt5) pc13 return w1 pc14
assume x4gt5 pc15 assume (x1x32)
pc1 x1 3 pc2 assume (x1gt0) pc3 x3
f1(x1) pc4 y2 y1 pc5 y3 f2(y2) pc6 z2
z11 pc7 z3 2z2 pc8 return z3 pc9 return
y3 pc10 x4 x31 pc11 x5 f3(x4) pc12
assume(w1lt5) pc13 return w1 pc14 assume
x4gt5 pc15 assume(x1x32)
Find predicate needed at point i
i
i
23Interprocedural Analysis
Trace Formula
Trace
NO
Find predicate needed at point i
YES
i
i
NO
Require at each point i Scoped predicates YES
Variables visible at i NO Callers local
variables
24Problems with Cutting
Trace Formula
Trace
?-
i
i
?
- Caller variables common to ?- and ?
- Unsuitable interpolant not well-scoped
25Scoped Cuts
Trace Formula
Trace
Call begins
i
i
26Scoped Cuts
Trace Formula
Trace
Call begins
?-
?
i
i
Predicate at pci Interpolant from cut i
27Common Variables
Trace Formula
Trace
Common Variables
Formals
?-
?
Formals
Current locals
i
i
Well-scoped
Predicate at pci Interpolant from i-cut
28Example Trace
main() ? 1 if (flag) 2 y
inc(x,flag) 3 if (yltx) ERROR else 4
y inc(z,flag) 5 if (ygtz) ERROR
? return
m1 assume(flag!0) m2 yinc(x,flag) i1
assume(sign!0) i2 rv a1 i4 return rv
m3 assume(yltx) ERROR
inc(int a, int sign) 1 if (sign) 2 rv
a1 else 3 rv a-1 4 return
rv
29Trace Formula
m1 assume(flag!0) m2 yinc(x,flag) i1
assume(sign!0) i2 rv a1 i4 return rv
m3 assume(yltx)
m1 assume(flag0!0) m2 a0x0, sign0flag0
i1 assume(sign0!0) i2 rv0a01 i4 y0rv0
m3 assume(y0 lt x0)
SSA Trace
Trace
30Trace Formula
m1 assume(flag!0) m2 yinc(x,flag) i1
assume(sign!0) i2 rv a1 i4 return rv
m3 assume(yltx)
Æ flag0 ? 0 Æ a0x0 Æ sign0 flag0 Æ
sign0 ? 0 Æ rv0a01 Æ y0rv0 Æ y0
lt x0
Call begins
i
i
Trace
Trace Formula
31Trace Formula
m1 assume(flag!0) m2 yinc(x,flag) i1
assume(sign!0) i2 rv a1 i4 return rv
m3 assume(yltx)
Æ flag0 ? 0 Æ a0x0 Æ sign0 flag0 Æ
sign0 ? 0 Æ rv0a01 Æ y0rv0 Æ y0
lt x0
?-
?
i
i
Trace
Trace Formula
32Trace Formula
m1 assume(flag!0) m2 yinc(x,flag) i1
assume(sign!0) i2 rv a1 i4 return rv
m3 assume(yltx)
Æ flag0 ? 0 Æ a0x0 Æ sign0 flag0 Æ
sign0 ? 0 Æ rv0a01 Æ y0rv0 Æ y0
lt x0
a0
Interpolate
?-
?
rv0
a0
i
i
rv0
rv0gt a0
Trace
Trace Formula
33Lazy Abstraction Procedure Summaries
Yes
Safe
Abstract
C Program
Refine
No
Property
Trace
Q. How to find scoped predicates ? Solution
Scoped Cuts Interpolation
34Review Procedures
- Modular Analysis via Summaries
- Summary (In,Out) vertices formulas
- Requires scoped predicates
- Scoped cuts Interpolation
35This Lecture Modular Analyses
- Procedures
- - Summaries
- Concurrency
- - Thread-Context Reasoning
36This Lecture Modular Analyses
- Procedures
- - Summaries
- Concurrency
- - Thread-Context Reasoning
37Multithreaded Programs
Thread
Thread
x
Shared Memory
- OS, WebServers, Databases, Embedded Systems
- Curse of Interleaving
- Non-deterministic scheduling
- Exponentially many behaviors hard to detect,
reproduce errors - Testing exercises a fraction of possible behaviors
38Data Races
? x x1 ?
? x x-5 ?
lock(l) unlock(l)
lock(l) unlock(l)
x
- A data race on x is a state where
- Two threads can access x
- One of the accesses is a write
- Unpredictable, undesirable
- Synchronization Must hold lock when accessing x
39Previous Work Locks
? x x1 ?
lock(l) unlock(l)
- Infer some lock(s) that protect x
- Check lock(s) held when accessing x
- Report error if lock(s) not held
Scalable Restricted to locking
40Other Synchronization Idioms
atomic old state if(state0)
state1 ? if(old0) x
state0
x
Producer-Consumer
State-based
Interrupt-toggling
41Previous Work Model Checking
atomic old state if(state0)
state1 ? if(old0) x
state0
x
Producer-Consumer
State-based
Interrupt-toggling
Godefroid 97 HolzmannHavelund-Visser Dwyer-
HatcliffAvrunin-Clarke Musuvathi-Dill-Engler
02 Yahav 01
Any Synch. Idiom Fixed threads Manual Abstraction
State Explosion
Model Checking (State Exploration)
42Race Checking by State Exploration
Initial
Shared Memory
Is there a path from Initial to Race ?
Data Race
43Problem State Explosion
Initial
Shared Memory
Is there a path from Initial to Race ?
Data Race
44Problem State Explosion
Initial
1. Data Infinitely many valuations for program
variables
- 2. Control
- k threads, m locations mk
- - k4,m100, states 1 billion
- Unbounded threads ?
Data Race
45LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Context Inference
46LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Context Inference
47Problem State Explosion
Initial
1. Data Infinitely many valuations for program
variables
- 2. Control
- k threads, m locations mk
- - k4,m100, states 1 billion
- Unbounded threads ?
Data Race
48Solution Abstract Irrelevant Detail
Observe - Few relevant variables, relationships -
Track predicates (relationships) instead of
values 1. Predicate Abstraction
1. Data Infinitely many valuations for program
variables
2. Control k threads, m locations mk -
k4,m100, states 1 billion Unbounded threads ?
Observe - Analyze system as Thread Context -
Context Summary of all other threads
(w.r.t. property) 2. Thread-Context Analysis
49Example
Check for races on x Initially s is 0 1st
thread into atomic - sets old to 0 (value of
s) - sets s to 1 - passes test before
access Later threads - set old to 1
(value set by 1st thread) - fail test before
access (until the 1st thread is done)
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
50LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Thread-Context Reasoning
- Verifying Multithreaded Programs
51Predicate Abstraction
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
1
s0
Predicates on Variables s0, old0
2
s0
3
s0, old0
4
s0, old0
s0, old0
5
s0 old0
6
x
s0 old0
1
Q What about other threads ?
Reachability Graph
52LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Contexts
- Thread-Context Reasoning
- Context Inference
53Threads, Contexts
Assume threads run same code Context
Summary of all other threads - Precise enough
to check property System Thread Context
Context
Shared Memory
Q What about other threads ?
Q What does a Context look like ?
54Thread, Contexts
Thread
Summary
Context
Context Summary of all other threads 1.
Summarize a single thread 2. Multiple threads by
counting
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
Q What does a Context look like ?
55Many Threads by Counting
Context
Initial loc 1 , other 0 Operations 1. Pick
edge w/ source counter gt 0, 2. Source counter
-1 Target counter 1 Havoc variables on
edge, Assume predicate on
target Unbounded threads k-Counter
Abstraction Value gt k abstracted to 1 for k1,
values 0,1,1
1
1
1
1
1
1
0
2
0
State
0
s0
True
s? 0Æ s0
s? 0
Q What does a Context look like ?
56LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Contexts
- Thread-Context Reasoning
- Context Inference
57Thread-Context Reasoning
- Q How to check race-freedom ?
- Given an Abstraction
- 1. DataPredicates 2. Control Summary, k
k
Summarize
Reach Graph
µ
No Race
Given Summary
Computed Summary
Use Context Build finite Reach Graph
Check Race unreachable
1
Verify Context Sound Check Summary
Overapproximates single Threads behavior
2
58- Q How to check race-freedom ?
- Given an Abstraction
- 1. DataPredicates 2. Control Summary, k
Thread-Context Reasoning
2
1
Summarize
Reach Graph
µ
No Race
Given Summary
Computed Summary
Assume-Guarantee (Use) (Verify)
No Race
59LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Thread-Context Reasoning
- Context Inference
60LA for Multithreaded Programs
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Thread-Context Reasoning
- Context Inference
61Inference Build Summary
Abstraction Preds P0 Ctr k0
1
Trace
Reach Graph
Race
1
2
Reach Graph
Summarize
µ
No Race
62Inference Trace Analysis
Abstraction Preds P0 Ctr k0
Abstraction Preds P1 Ctr k1
Trace
Feasible
Infeasible
- Refine using Trace
- Either
- Add new predicates
- Increase k
Report Trace Interleaved sequence of threads ops
63Inference Build Summary
Abstraction Preds P0 Ctr k0
Abstraction Preds P1 Ctr k1
1
2
Summarize
Reach Graph
µ
1
2
Summarize
Reach Graph
µ
1
2
Summarize
Reach Graph
µ
64Context Inferred
1
2
Summarize
Reach Graph
µ
Assume-Guarantee
No Race
65Context Inference
BUILD SUMMARY
Update Summary
1
2
YES
NO
Init. Abstraction Preds P0 Ctr k0
Safe?
Init. Summary Summary
Reach Graph
Summarize
µ ?
YES
NO (trace)
NO
Refine using Trace
Feasible?
Output SAFE No Data Races
YES
TRACE ANALYSIS
Output Data Race
66Plan
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Thread-Context Reasoning
- Context Inference
- Example
67Ex Races on x
Build Summary
1
T
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
2
1
2
Reach Graph
3
Summarize
µ
4
x
5
6
x
Abstraction Preds k1
7
Control-Flow Graph
68Ex Races on x
Build Summary
1
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
1
2
Reach Graph
3
x
4
5
6
Abstraction Preds k1
6
x
x
Race
69Ex Races on x
Trace Analysis
Trace
Thread 1
Thread 0
1
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
assume (True)
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
old s
1
2
assume (s0)
Reach Graph
s 1
3
// do_work()
assume (old0)
4
//write x enabled
5
assume (True)
old s
6
assume (s0)
Abstraction Preds k1
s 1
6
// do_work()
x
x
assume (old0)
//write x enabled
70Ex Races on x
Trace Analysis
Trace
Thread 1
Thread 0
Time
assume (True)
old s
assume (s0)
s 1
s is set to 1
// do_work()
assume (old0)
//write x enabled
assume (True)
old s
Infeasible branch
assume (s0)
s 1
Infeasible Trace
// do_work()
New Predicate s 0
assume (old0)
//write x enabled
71Ex Races on x
Build Summary
1
s0
1 while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
2
s0
2
s0
1
s
s0 old0
3
µ
s?0
Summarize
Reach Graph
x
s0 old0
4
s?0
s
s1
s0 old0
5
Local Pred. old0 - Prunes infeasible paths -
Invisible to other threads - Quantified away
s0 old0
6
Abstraction Preds s0 k1
x
s0 old0
7
, old0
s0
72Ex Races on x
Build Summary
s0 Ç s?0
Context changes s
1
T
1while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
1
2
2
T
s0 Ç s?0
s,x
3
Reach Graph
Summarize
µ
s0
s?0
s0 old0
4
5
s
s?0 old?0
s?0
s1
s?0 old0
5
Local Pred. old0 - Cuts locally infeasible
paths - Invisible to other threads - Quantified
away
s0
s?0 old0
6
x
Abstraction Preds s0 k1
, old0
73Ex Races on x
SAFE No Races on x
Build Summary
1while(1) atomic 2 old s 3
if(s0) 4 s 1 //
do_work() ? 5 if(old0) 6 x 7
s0
1
2
Reach Graph
Sumz
µ
Abstraction Preds s0 k1
, old0
74Plan
- Data Races
- Previous Work
- State Explosion
- Abstractions
- Data Predicate Abstraction
- Control Thread-Context Reasoning
- Context Inference
- Example
- 4. Experiments
75Data Races in NesC Programs
- PL for Networked Embedded Systems Gay et al. 03
- Interrupts fire events, which fire other events
- or post tasks which run asynchronously
- Race-freedom important
- Non-trivial synchronization idioms
- Flow-based analysis
- Compiled to C
76Case Study sense.nc
Interrupt 1 fires ? old state if (state
0) state 1 ? assume (old 0)
about to write x
Interrupt 1 handler disables interrupt
2 BLAST finds information proves no races
atomic old state if(state0)
state1 ? if(old0) x ?
Interrupt 2 fires ? state 0
Interrupt 1 fires ? old state assume (state
0) state 1 ? If (old 0) about
to write x
77Review Multithreaded
- Multithreaded Verification is hard
- Data, Control Explosion
- Abstract Irrelevant details
- Combine Abstractions
- Data Predicates
- Control Contexts (Summary Counter)
- Iterative Context Inference
78Next lecture
- Partial-program analysis
- From verification to interfaces