Evaluating the Effectiveness of Slicing for Model Reduction of Concurrent ObjectOriented Programs

1
Evaluating the Effectiveness of Slicing for
Model Reduction of Concurrent Object-Oriented
Programs
Indus
SAnToS Laboratory, Kansas State University,
USA eSquared Laboratory, University of Nebraska,
USA
http//indus.projects.cis.ksu.edu
Matthew B. Dwyer John Hatcliff
Robby Todd Wallentine
Matthew Hoosier Venkatesh Ranganath
Support
US Army Research Office US Defense Advanced
Research Projects Agency (DARPA) Lockheed Martin
Rockwell-Collins ATC
IBM US Air Force Office of Scientific Research US
National Science Foundation
2
Model Checking Realistic Systems
Goal
model check implementations of realistic Java
systems against a particular property or
collection of properties

• concurrency, heap intensive data, extensive use
  of layering, design patterns, etc.
• theres just a lot of code!
• libraries, infrastructure code, etc.

3
Model Reduction Strategies

• Partial order reductions
• Thread symmetry reductions
• Heap canonicalization
• Abstraction mechanisms
• etc.,

There have been extensive experimental studies to
evaluate the effectiveness of these techniques.
4
Slicing for Model Reduction
Folklore
program slicing is useful for model-reduction
5
Properties and Program Features
public synchronized void remove(int
index) assert 0 lt index
index lt capacity assert
size lt capacity if
(!this.isEmpty()) for (int i
index i lt (size - 1)
i) listi listi
1 listsize - 1
null size--
Property

• sets of assertion

6
Properties and Program Features
public synchronized void remove(int
index) assert 0 lt index
index lt capacity assert
size lt capacity if
(!this.isEmpty()) for (int i
index i lt (size - 1)
i) listi listi
1 listsize - 1
null size--
Property

• temporal property(_at_remove -gt ltgt size lt 1)

7
Properties and Program Features
public synchronized void remove(int
index) assert 0 lt index
index lt capacity assert
size lt capacity if
(!this.isEmpty()) for (int i
index i lt (size - 1)
i) listi listi
1 listsize - 1
null size--
Property

• absence of deadlock

8
Property-directed Slicing
Source program

• slicing criterion generated automatically from
  observables mentioned in the property

• backwards slicing automatically removes all
  components that definitely do not influence the
  observables

9
Expected Benefits of Slicing
Statements Removed

• fewer transitions
• fewer interleavings
• fewer states

10
Whats the Evidence?

• Millet Teitelbaum Slicing Promela (SPIN
  98)
• Clarke et. al. Program Slicing of Hardware
  Description Languages (CHARME 99)
• from 1038 states to 1022 states
• Corbett et. al. Bandera Extracting Finite
  Models from Java Source Code (ICSE 00)
• Jia Graf -- Verification Experiments on the
  MASCARA Protocol (SPIN 01)
• 1 example, 4 properties, 1-2 orders of magnitude
  reduction

11
Conventional Wisdom is Inconclusive
Gerard Holzmann Promela slicing algorithm in
SPIN
I haven't found good use for the Promela
slicing algorithm.
I use it, and recommend it, mainly as a way of
finding small redundancies in models.
For realistic properties, the slicing does not
remove all that much -- so I don't consider it a
hugely important technique in this context.
Of course, this could all be very different if
you apply it to Java source programs.
12
Bandera Next Generation
Java Model Checking Framework
13
Outline
14
Bounded Buffer Example
class BoundedBuffer protected int numSlots
0 / size of buffer / protected
Object buffer null / buffer array /
protected int putIn 0 / next empty
slot / protected int takeOut 0 /
next item / protected int count 0
/ number items / public BoundedBuffer(int
numSlots) public synchronized void
deposit(Object value) public
synchronized Object fetch ()
15
Data Dependence
public class BoundedBuffer protected int
numSlots 0 protected Object buffer
null protected int putIn 0 protected int
takeOut 0 protected int count 0 public
BoundedBuffer(int numSlots) if (numSlots lt
0) throw new IllegalArgumentException()
this.numSlots numSlots buffer
new ObjectnumSlots ....
Data Dependence
Definition of variable V at statement S1 reaches
a use of V at statement S2
16
Data Dependence
public class BoundedBuffer protected int
numSlots 0 protected Object buffer
null protected int putIn 0 protected int
takeOut 0 protected int count 0 public
BoundedBuffer(int numSlots) if (numSlots lt
0) throw new IllegalArgumentException()
this.numSlots numSlots buffer
new ObjectnumSlots ....
Data Dependence
Definition of variable V in statement S1 reaches
a use of V at statement S2
17
Data Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence w/ Interprocedural Control Flow
18
Data Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence w/ Interprocedural Control Flow
19
Data Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence w/ Interprocedural Control Flow
20
Interference Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Data Dependence across threads
21
Control Dependence
public class BoundedBuffer protected int
numSlots 0 protected Object buffer
null protected int putIn 0 protected int
takeOut 0 protected int count 0 public
BoundedBuffer(int numSlots) if (numSlots lt
0) throw new IllegalArgumentException()
this.numSlots numSlots buffer
new ObjectnumSlots ....
Control Dependence
A conditional statement controls whether or not
the current statement is executed
22
Divergence Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
capturing delaying influences within a single
thread
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Divergence Dependence
Statement S1 influences S2 if S1 may infinitely
delay S2 (e.g., by going into an infinite loop)
23
Ready Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
capturing delaying influences across threads
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Ready Dependence
The failure of a lock release to complete may
cause a lock acquire to be delayed (blocked)
indefinitely
24
Ready Dependence
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
Ready Dependence
The failure of a notify to complete may cause a
wait to be delayed indefinitely
25
Indus / Kaveri Architecture
26
Eclipse-based Front-end
Visualization of Slice
27
Eclipse-based Front-end
Inspection of underlying Jimple
28
Eclipse-based Front-end
Navigation of dependences
29
Outline
30
Classes of Examples

• Algorithm Sketches (used elsewhere in the
  literature)
• Bounded Buffer
• Pipeline
• Sleeping Barbers
• Readers/Writers
• RAX (distillation of NASA Remote Agent Bug)
• Small Applications (significantly more heap
  intensive)
• Disk Scheduler
• Alarm Clock
• Larger Applications
• Replicated Workers (client/server data
  distribution framework)
• Ray Tracer (Java Grande)
• Siena (internet-scale publish/subscribe
  middleware)

31
Forming an Executable
32
Size in Bytecodes
33
Outline
34
Libraries -- Assessment

• Significant portion of program bytecodes come
  from libraries
• Some library code may be unreachable during
  execution but still contribute to state vector
• Some library code may be reachable but irrelevant
  to property

• Much unreachable code can be pruned away using a
  precise call-graph analysis (based on precise
  points-to analysis)
• To avoid straw man arguments, we will take
  CallGraph-reachability-pruned code-bases as the
  baseline of slicing/model-checking experiments

Comparing slicing to an already smart framework!
35
Static Metrics for Slicing
What is the effect of Call-Graph-reachability and
slicing on the static program model structure?
Measurements (see paper)

• bytecodes
• classes
• methods
• fields
• new (allocations)
• threads
• exceptions
• synchronization statements
• wait
• notify

App Lib
Reachable App Lib
Sliced App Lib
App
36
Static Metrics Byte-codes
Removes 700 byte codes
Reduces size by half
Reduces size by two-thirds
37
Static Metrics Fields
Similar reductions for fields (4x)
38
Static Metrics - New
39
Static Metrics - Sync
40
Outline
41
Experimental Setup

• Property
• (d) Deadlock or
• (a) Assertion checking
• Partial Order Reductions
• (P) POR on
• () POR off
• Reachability/Slicing
• (R) Reachability
• (S) Slicing

Postive Outliers removed (more than two standard
deviations from mean)
Opteron 250 processor 12 GB of RAM Sun JDK 1.5 on
Linux
42
Dynamic Metrics - BBuffer
Transitions
States
Between factor of 4-5 reduction
36405
90882
7484
18578
Byte-codes
Memory
Time
0.11M
3.9M
13.9s
873
61.2s
194
43
Dynamic Metrics BBuffer w/POR
Transitions
States
State reductions w/ POR are similar
138
3980
28
630
Byte-codes
Memory
Time
0.12M
0.12M
10.7s
873
14.4s
194
44
Buffer and indices are sliced away because
synchronization only depends on count.
45
Buffer and indices are sliced away because
synchronization only depends on count.
46
Navigating Dependences
Display and navigate Jimple and dependences for a
particular statement
47
Dynamic Metrics - ReadersWriters
Transitions
States
Significant reductions
25197913
26780595
113727
415098
Byte-codes
Memory
Time
4.15M
5.86M
89.8s
48084.6s
945
268
48
Dynamic Metrics ReadersWriters w/POR
Transitions
States
Also good reductions with POR included
603
134
44261
2103
Byte-codes
Memory
Time
0.12M
4.13M
11.3s
71.2s
945
268
49
Assessment

• Very little application code sliced away (a
  counter, and system.out.printlns)
• Dramatic amount of library code is sliced away
• 25 classes, 53 methods, 40 fields, and 683
  statements
• java.lang.System gets completed gutted

java.util.Properties java.io.PrintStream java.io.I
nputStream java.io.OutputStream
50
Dynamic Metrics RepW
Transitions
States
In this example, the reductions are not as
dramatic..
2818232
8102858
2736984
7867615
Byte-codes
Memory
Time
4.74M
4.77M
5134.2s
7799.9s
1792
1460
51
Dynamic Metrics RepW (w/POR)
Transitions
States
In this example, the reductions are not as
dramatic..
3091
96798
2676
90735
Byte-codes
Memory
Time
3.52M
1.65M
121.1s
163.2s
1792
1460
52
Dynamic Metrics - DiskSched
Transitions
States
7687219
2044564
5487745
14302033
Byte-codes
Memory
Time
5.88M
5.71M
4594.5s
1314
35281.8s
643
53
Dynamic Metrics DiskSched w/ POR
Transitions
States
7690
858963
7688
816991
Byte-codes
Memory
Time
1.81M
4.16M
317.6s
1314
1859.7s
643
54
Slicing Siena Middleware
public void setData(final byte data,
final int offset,
final int length) buffer data
setLength(length)
55
Slicing Siena Middleware
Summary Via several intermediate steps, new
byte1 is flowing across multiple methods and
classes only into a variable that is never used
in the slice. Very difficult to detect manually
especially on large code bases!
public void setData(final byte data,
final int offset,
final int length) buffer data
setLength(length)
56
Research Questions
How do the benefits of slicing vary with program
size and complexity?

• Considering static size
• Large examples dR/dS 1.4
• Skeletons small examples dR/dS 2.7
• Considering state-space sparseness
• Non-sparse examples dR/dS 4.0
• Sparse examples dR/dS 1.9
• Conclusions
• Did not observe the inverse scalability factor
  on static program size mentioned by Clarke et.
  al. for VHDL, but we believe this is because our
  sample space is small
• Did observe that impact of slicing is greater
  when more of the state-space is searched

sparse statespace ratio of matched states to
stored states is less than 0.1
57
Research Questions
What is the incremental benefit of slicing over
Call-Graph-based reachability pruning?

• Mean reduction factor (dR/dS) across all examples
  was 2.5
• Slicing is computationally cheap even on our
  largest example, slicing completed in only 40
  seconds
• Never makes things worse
• Almost always improves some cost factor
• e.g., for RW turned a 13 hour run into a 1.5
  minute run
• Conclusions
• Slicing often turns infeasible checks into
  feasible ones
• Cost of slicing is so low that resulting savings
  almost always dominates

58
Research Questions
What is the incremental benefit of slicing over
state-of-the-art POR ?

• Mean reduction factor (dRP/dSP)
• across all runs was 2.1
• across larger (non-sparse) searches was 2.9
• Reduction factor (dR/dRP) for POR alone
• across all runs was 48.3
• across larger (non-sparse) searchers was 105.5
• Conclusions
• Both slicing and POR provide greater benefit when
  a larger portion of statespace is searched
• Slicing reduction is largely orthogonal to POR
  reduction

59
Research Questions
Does slicing yield greater reductions when
checking sets of assertions compared to checking
deadlock?

• For examples on which both deadlock and
  assertions where checked,
• Mean deadlock slicing reduction factor (dRP/dSP)
  was 2.8
• Mean assertion slicing reduction factor (aRP/aSP)
  was 2.7
• Conclusions
• Surprising, since we expected that a more
  localized specification would require less of the
  program to be included in the slice
• Synchronization couples most of the examples that
  we considered
• Larger applications may have clusters of
  synchronizing threads that are separable

60
Conclusions

• A robust program slicer that can be applied to
  real Java is a sophisticated analysis and
  transformation framework with many applications.
• Applying Indus slicing for Java model checking
• is easy (completely automatic no user
  intervention)
• is inexpensive (compared to model checking costs)
• is effective and orthogonal to other reduction
  strategies (typical reduction factors range from
  2x 5x)
• Slicing techniques like Indus seem particularly
  relevant for real systems that make significant
  use of libraries
• Indus can be downloaded and immediately applied
  in the context of Java verification efforts
• Weve used Indus with JPF, and engineers at
  Fujitsu are also using Indus with JPF.
• Data/Predicate abstraction (orthogonal) will
  still be needed in many cases to reduce resources
  required for model-checking

61
Context
Technology Transition

• Funding from a team at Lockheed Martin to apply
  this model checking infrastructure to reasoning
  about satellite mission control systems

Technical Foundations

• Foundations of Control Dependence (ESOP 05)
• Pruning Interference Dependence using Escape
  Analysis (CC 04)
• Preserving LTL with Slicing (HOSC 01)
• Notions of Dependence for Java (SAS 99)

62
Future Work

• Weve identified a number of improvements that
  can be implemented in the slicer (requiring some
  additional static analyses) that we believe will
  yield additional model-checking reductions
• Current notions of dependence used are very
  conservative (safe) we are exploring the extent
  to which these can be relaxed to obtain greater
  reductions for bug-finding

63
For More Information
SAnToS Laboratory, Kansas State
University http//www.cis.ksu.edu/santos
ESqUAReD Laboratory, University of Nebraska,
Lincoln http//www.cse.unl.edu/esquared
Indus Project
http//indus.projects.cis.ksu.edu
Bogor Project
http//bogor.projects.cis.ksu.edu
Bandera Project
http//bandera.projects.cis.ksu.edu
64
Experimental Setup

• Property
• (d) Deadlock or
• (a) Assertion checking
• Partial Order Reductions
• (P) POR on
• () POR off
• Reachability/Slicing
• (R) CFG Reachability
• (S) Slicing

65
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
66
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
67
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
68
Research Questions

• How do the benefits of slicing vary with program
  size and complexity?
• What is the incremental benefit of slicing over
  CFG-based reachability pruning?
• What is the incremental benefit of slicing over
  state-of-the-art POR and heap/thread symmetry
  reductions?
• Does slicing yield greater reductions when
  checking assertions compared to checking deadlock?

69
Bandera Next Generation
70
(No Transcript)
71
Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
This sequence To be completed or omitted
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
72
Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
73
Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
74
Deadlock-Preserving Slicing
public synchronized void deposit(Object value)
while (count numSlots) try
wait() catch (InterruptedException e)
bufferputIn value putIn (putIn 1)
numSlots count if (count 1)
notify()
public synchronized Object fetch() Object
value while (count 0) try
wait() catch (InterruptedException e)
value buffertakeOut takeOut
(takeOut 1) numSlots count-- if
(count (numSlots 1)) notify()
return value
75
Assessment
To be completed