Title: Evaluating the Effectiveness of Slicing for Model Reduction of Concurrent ObjectOriented Programs
1Evaluating the Effectiveness of Slicing for
Model Reduction of Concurrent Object-Oriented
Programs
Indus
SAnToS Laboratory, Kansas State University,
USA eSquared Laboratory, University of Nebraska,
USA
http//indus.projects.cis.ksu.edu
Matthew B. Dwyer John Hatcliff
Robby Todd Wallentine
Matthew Hoosier Venkatesh Ranganath
Support
US Army Research Office US Defense Advanced
Research Projects Agency (DARPA) Lockheed Martin
Rockwell-Collins ATC
IBM US Air Force Office of Scientific Research US
National Science Foundation
2Model Checking Realistic Systems
Goal
model check implementations of realistic Java
systems against a particular property or
collection of properties
- concurrency, heap intensive data, extensive use
of layering, design patterns, etc. - theres just a lot of code!
- libraries, infrastructure code, etc.
3Model Reduction Strategies
- Partial order reductions
- Thread symmetry reductions
- Heap canonicalization
- Abstraction mechanisms
- etc.,
There have been extensive experimental studies to
evaluate the effectiveness of these techniques.
4Slicing for Model Reduction
Folklore
program slicing is useful for model-reduction
5Properties and Program Features
public synchronized void remove(int
index) assert 0 lt index
index lt capacity assert
size lt capacity if
(!this.isEmpty()) for (int i
index i lt (size - 1)
i) listi listi
1 listsize - 1
null size--
Property
6Properties and Program Features
public synchronized void remove(int
index) assert 0 lt index
index lt capacity assert
size lt capacity if
(!this.isEmpty()) for (int i
index i lt (size - 1)
i) listi listi
1 listsize - 1
null size--
Property
- temporal property(_at_remove -gt ltgt size lt 1)
7Properties and Program Features
public synchronized void remove(int
index) assert 0 lt index
index lt capacity assert
size lt capacity if
(!this.isEmpty()) for (int i
index i lt (size - 1)
i) listi listi
1 listsize - 1
null size--
Property
8Property-directed Slicing
Source program
- slicing criterion generated automatically from
observables mentioned in the property
- backwards slicing automatically removes all
components that definitely do not influence the
observables
9Expected Benefits of Slicing
Statements Removed
- fewer transitions
- fewer interleavings
- fewer states
10Whats the Evidence?
- Millet Teitelbaum Slicing Promela (SPIN
98) - Clarke et. al. Program Slicing of Hardware
Description Languages (CHARME 99) - from 1038 states to 1022 states
- Corbett et. al. Bandera Extracting Finite
Models from Java Source Code (ICSE 00) - Jia Graf -- Verification Experiments on the
MASCARA Protocol (SPIN 01) - 1 example, 4 properties, 1-2 orders of magnitude
reduction
11Conventional Wisdom is Inconclusive
Gerard Holzmann Promela slicing algorithm in
SPIN
I haven't found good use for the Promela
slicing algorithm.
I use it, and recommend it, mainly as a way of
finding small redundancies in models.
For realistic properties, the slicing does not
remove all that much -- so I don't consider it a
hugely important technique in this context.
Of course, this could all be very different if
you apply it to Java source programs.
12Bandera Next Generation
Java Model Checking Framework
13Outline
14Bounded Buffer Example
class BoundedBuffer protected int numSlots
0 / size of buffer / protected
Object buffer null / buffer array /
protected int putIn 0 / next empty
slot / protected int takeOut 0 /
next item / protected int count 0
/ number items / public BoundedBuffer(int
numSlots) public synchronized void
deposit(Object value) public
synchronized Object fetch ()
15Data Dependence
public class BoundedBuffer protected int
numSlots 0 protected Object buffer
null protected int putIn 0 protected int
takeOut 0 protected int count 0 public
BoundedBuffer(int numSlots) if (numSlots lt
0) throw new IllegalArgumentException()
this.numSlots numSlots buffer
new ObjectnumSlots ....
Data Dependence
Definition of variable V at statement S1 reaches
a use of V at statement S2
16Data Dependence
public class BoundedBuffer protected int
numSlots 0 protected Object buffer
null protected int putIn 0 protected int
takeOut 0 protected int count 0 public
BoundedBuffer(int numSlots) if (numSlots lt
0) throw new IllegalArgumentException()
this.numSlots numSlots buffer
new ObjectnumSlots ....
Data Dependence
Definition of variable V in statement S1 reaches
a use of V at statement S2
17Data Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence w/ Interprocedural Control Flow
18Data Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence w/ Interprocedural Control Flow
19Data Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence w/ Interprocedural Control Flow
20Interference Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence across threads
21Control Dependence
public class BoundedBuffer protected int
numSlots 0 protected Object buffer
null protected int putIn 0 protected int
takeOut 0 protected int count 0 public
BoundedBuffer(int numSlots) if (numSlots lt
0) throw new IllegalArgumentException()
this.numSlots numSlots buffer
new ObjectnumSlots ....
Control Dependence
A conditional statement controls whether or not
the current statement is executed
22Divergence Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
capturing delaying influences within a single
thread
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Divergence Dependence
Statement S1 influences S2 if S1 may infinitely
delay S2 (e.g., by going into an infinite loop)
23Ready Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
capturing delaying influences across threads
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Ready Dependence
The failure of a lock release to complete may
cause a lock acquire to be delayed (blocked)
indefinitely
24Ready Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Ready Dependence
The failure of a notify to complete may cause a
wait to be delayed indefinitely
25Indus / Kaveri Architecture
26Eclipse-based Front-end
Visualization of Slice
27Eclipse-based Front-end
Inspection of underlying Jimple
28Eclipse-based Front-end
Navigation of dependences
29Outline
30Classes of Examples
- Algorithm Sketches (used elsewhere in the
literature) - Bounded Buffer
- Pipeline
- Sleeping Barbers
- Readers/Writers
- RAX (distillation of NASA Remote Agent Bug)
- Small Applications (significantly more heap
intensive) - Disk Scheduler
- Alarm Clock
- Larger Applications
- Replicated Workers (client/server data
distribution framework) - Ray Tracer (Java Grande)
- Siena (internet-scale publish/subscribe
middleware)
31Forming an Executable
32Size in Bytecodes
33Outline
34Libraries -- Assessment
- Significant portion of program bytecodes come
from libraries - Some library code may be unreachable during
execution but still contribute to state vector - Some library code may be reachable but irrelevant
to property
- Much unreachable code can be pruned away using a
precise call-graph analysis (based on precise
points-to analysis) - To avoid straw man arguments, we will take
CallGraph-reachability-pruned code-bases as the
baseline of slicing/model-checking experiments
Comparing slicing to an already smart framework!
35Static Metrics for Slicing
What is the effect of Call-Graph-reachability and
slicing on the static program model structure?
Measurements (see paper)
- bytecodes
- classes
- methods
- fields
- new (allocations)
- threads
- exceptions
- synchronization statements
- wait
- notify
App Lib
Reachable App Lib
Sliced App Lib
App
36Static Metrics Byte-codes
Removes 700 byte codes
Reduces size by half
Reduces size by two-thirds
37Static Metrics Fields
Similar reductions for fields (4x)
38Static Metrics - New
39Static Metrics - Sync
40Outline
41Experimental Setup
- Property
- (d) Deadlock or
- (a) Assertion checking
- Partial Order Reductions
- (P) POR on
- () POR off
- Reachability/Slicing
- (R) Reachability
- (S) Slicing
Postive Outliers removed (more than two standard
deviations from mean)
Opteron 250 processor 12 GB of RAM Sun JDK 1.5 on
Linux
42Dynamic Metrics - BBuffer
Transitions
States
Between factor of 4-5 reduction
36405
90882
7484
18578
Byte-codes
Memory
Time
0.11M
3.9M
13.9s
873
61.2s
194
43Dynamic Metrics BBuffer w/POR
Transitions
States
State reductions w/ POR are similar
138
3980
28
630
Byte-codes
Memory
Time
0.12M
0.12M
10.7s
873
14.4s
194
44Buffer and indices are sliced away because
synchronization only depends on count.
45Buffer and indices are sliced away because
synchronization only depends on count.
46Navigating Dependences
Display and navigate Jimple and dependences for a
particular statement
47Dynamic Metrics - ReadersWriters
Transitions
States
Significant reductions
25197913
26780595
113727
415098
Byte-codes
Memory
Time
4.15M
5.86M
89.8s
48084.6s
945
268
48Dynamic Metrics ReadersWriters w/POR
Transitions
States
Also good reductions with POR included
603
134
44261
2103
Byte-codes
Memory
Time
0.12M
4.13M
11.3s
71.2s
945
268
49Assessment
- Very little application code sliced away (a
counter, and system.out.printlns) - Dramatic amount of library code is sliced away
- 25 classes, 53 methods, 40 fields, and 683
statements - java.lang.System gets completed gutted
java.util.Properties java.io.PrintStream java.io.I
nputStream java.io.OutputStream
50Dynamic Metrics RepW
Transitions
States
In this example, the reductions are not as
dramatic..
2818232
8102858
2736984
7867615
Byte-codes
Memory
Time
4.74M
4.77M
5134.2s
7799.9s
1792
1460
51Dynamic Metrics RepW (w/POR)
Transitions
States
In this example, the reductions are not as
dramatic..
3091
96798
2676
90735
Byte-codes
Memory
Time
3.52M
1.65M
121.1s
163.2s
1792
1460
52Dynamic Metrics - DiskSched
Transitions
States
7687219
2044564
5487745
14302033
Byte-codes
Memory
Time
5.88M
5.71M
4594.5s
1314
35281.8s
643
53Dynamic Metrics DiskSched w/ POR
Transitions
States
7690
858963
7688
816991
Byte-codes
Memory
Time
1.81M
4.16M
317.6s
1314
1859.7s
643
54Slicing Siena Middleware
public void setData(final byte data,
final int offset,
final int length) buffer data
setLength(length)
55Slicing Siena Middleware
Summary Via several intermediate steps, new
byte1 is flowing across multiple methods and
classes only into a variable that is never used
in the slice. Very difficult to detect manually
especially on large code bases!
public void setData(final byte data,
final int offset,
final int length) buffer data
setLength(length)
56Research Questions
How do the benefits of slicing vary with program
size and complexity?
- Considering static size
- Large examples dR/dS 1.4
- Skeletons small examples dR/dS 2.7
- Considering state-space sparseness
- Non-sparse examples dR/dS 4.0
- Sparse examples dR/dS 1.9
- Conclusions
- Did not observe the inverse scalability factor
on static program size mentioned by Clarke et.
al. for VHDL, but we believe this is because our
sample space is small - Did observe that impact of slicing is greater
when more of the state-space is searched
sparse statespace ratio of matched states to
stored states is less than 0.1
57Research Questions
What is the incremental benefit of slicing over
Call-Graph-based reachability pruning?
- Mean reduction factor (dR/dS) across all examples
was 2.5 - Slicing is computationally cheap even on our
largest example, slicing completed in only 40
seconds - Never makes things worse
- Almost always improves some cost factor
- e.g., for RW turned a 13 hour run into a 1.5
minute run - Conclusions
- Slicing often turns infeasible checks into
feasible ones - Cost of slicing is so low that resulting savings
almost always dominates
58Research Questions
What is the incremental benefit of slicing over
state-of-the-art POR ?
- Mean reduction factor (dRP/dSP)
- across all runs was 2.1
- across larger (non-sparse) searches was 2.9
- Reduction factor (dR/dRP) for POR alone
- across all runs was 48.3
- across larger (non-sparse) searchers was 105.5
- Conclusions
- Both slicing and POR provide greater benefit when
a larger portion of statespace is searched - Slicing reduction is largely orthogonal to POR
reduction
59Research Questions
Does slicing yield greater reductions when
checking sets of assertions compared to checking
deadlock?
- For examples on which both deadlock and
assertions where checked, - Mean deadlock slicing reduction factor (dRP/dSP)
was 2.8 - Mean assertion slicing reduction factor (aRP/aSP)
was 2.7 - Conclusions
- Surprising, since we expected that a more
localized specification would require less of the
program to be included in the slice - Synchronization couples most of the examples that
we considered - Larger applications may have clusters of
synchronizing threads that are separable
60Conclusions
- A robust program slicer that can be applied to
real Java is a sophisticated analysis and
transformation framework with many applications. - Applying Indus slicing for Java model checking
- is easy (completely automatic no user
intervention) - is inexpensive (compared to model checking costs)
- is effective and orthogonal to other reduction
strategies (typical reduction factors range from
2x 5x) - Slicing techniques like Indus seem particularly
relevant for real systems that make significant
use of libraries - Indus can be downloaded and immediately applied
in the context of Java verification efforts - Weve used Indus with JPF, and engineers at
Fujitsu are also using Indus with JPF. - Data/Predicate abstraction (orthogonal) will
still be needed in many cases to reduce resources
required for model-checking
61Context
Technology Transition
- Funding from a team at Lockheed Martin to apply
this model checking infrastructure to reasoning
about satellite mission control systems
Technical Foundations
- Foundations of Control Dependence (ESOP 05)
- Pruning Interference Dependence using Escape
Analysis (CC 04) - Preserving LTL with Slicing (HOSC 01)
- Notions of Dependence for Java (SAS 99)
62Future Work
- Weve identified a number of improvements that
can be implemented in the slicer (requiring some
additional static analyses) that we believe will
yield additional model-checking reductions - Current notions of dependence used are very
conservative (safe) we are exploring the extent
to which these can be relaxed to obtain greater
reductions for bug-finding
63For More Information
SAnToS Laboratory, Kansas State
University http//www.cis.ksu.edu/santos
ESqUAReD Laboratory, University of Nebraska,
Lincoln http//www.cse.unl.edu/esquared
Indus Project
http//indus.projects.cis.ksu.edu
Bogor Project
http//bogor.projects.cis.ksu.edu
Bandera Project
http//bandera.projects.cis.ksu.edu
64Experimental Setup
- Property
- (d) Deadlock or
- (a) Assertion checking
- Partial Order Reductions
- (P) POR on
- () POR off
- Reachability/Slicing
- (R) CFG Reachability
- (S) Slicing
65public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
66public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
67public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
68Research Questions
- How do the benefits of slicing vary with program
size and complexity? - What is the incremental benefit of slicing over
CFG-based reachability pruning? - What is the incremental benefit of slicing over
state-of-the-art POR and heap/thread symmetry
reductions? - Does slicing yield greater reductions when
checking assertions compared to checking deadlock?
69Bandera Next Generation
70(No Transcript)
71Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
This sequence To be completed or omitted
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
72Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
73Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
74Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
75Assessment
To be completed