Pointer and Escape Analysis for Multithreaded Programs

1
Pointer and Escape Analysis for (Multithreaded)
Programs

• Martin Rinard
• MIT Laboratory for
• Computer Science

2
Traditional Escape Analysis

• Procedures
• Dynamically allocated objects
• Does an object escape the allocating procedure?
• Is the lifetime of object contained in lifetime
  of procedure?
• If so, can stack allocate object
• Sequential programs

3
Modern Escape Analysis

• Region of Program
• Procedure
• Thread
• Group of Threads
• Multiple-Entry Component
• Dynamically allocated objects
• Is object captured within region?

4
Uses of Escape Information

• Negative Interaction Information
• Past Traditional Compiler Optimizations
• Stack Allocation
• Synchronization Elimination
• Variety of Dynamic Check Eliminations

5
Foundation for Interaction Analyses

• Systems built from groups of reconfigurable
  components
• Important to understand interactions
• Safety of composition
• Transform to enhance fast reconfiguration,
  fault-tolerance, predictability, performance
• Escape analysis focuses interaction analyses
• Eliminates host of potential interactions
• Cuts away large parts of program
• Enables use of more powerful interaction analyses

6
Importance of Interaction Analysis

• Need a global information about properties of
  potential component compositions
• Interaction patterns
• Inter-component dependences
• Theme see across component boundaries
• Payoff
• Information about behavior of potential composed
  systems
• Customized implementations
• Functionality-improving transformations
• Reduce vulnerability to failures
• Improve adaptation response times

7
Key Requirements

• Rapid reconfiguration, adaptability,
  customization
• Huge range of potential customized combinations
• Envisioned large-scale transformations
  impractical to perform manually
• Need to automate interaction analysis and
  subsequent transformations
• Distributed systems inherently concurrent
• Analyze multithreaded programs
• Characterize and exploit interactions between
  threads

8
Outline

• Combined Pointer and Escape Analysis
• Sequential Programs
• Multithreaded Programs
• Implementation in Flex Compiler
• Experimental Results

9
Points-to Escape Graph in Example
void computeMax() int max 0 Enumeration
enum database.elements() while
(enum.hasMoreElements()) Employee e
enum.nextElement() if (max lt e.salary()) max
e.salary() highestPaid e
dotted outside

vector
elementData
solid inside
enum
database
highestPaid
this
e
10
Definitions node types

• NI inside nodes
• represent objects created within the computation
  of the method
• one inside node for each object creation site
  represents all objects created at site
• NO outside nodes
• represent objects created outside of the
  computation of the method

11
Definitions outside node types

• NP parameter nodes
• represent objects passed as incoming parameters
• NL load nodes
• one load node for each load statement in method
• represents objects loaded from an escaped node
• NCL class nodes
• node from which static variables are accessed

12
Points-to Escape Graph in Example
void computeMax() int max 0 Enumeration
enum database.elements() while
(enum.hasMoreElements()) Employee e
enum.nextElement() if (max lt e.salary()) max
e.salary() highestPaid e
dotted outside

vector
elementData
solid inside
enum
database
highestPaid
this
e
13
Points-to Escape Graph in Example
void computeMax() int max 0 Enumeration
enum database.elements() while
(enum.hasMoreElements()) Employee e
enum.nextElement() if (max lt e.salary()) max
e.salary() highestPaid e
dotted outside

solid inside
vector
elementData
enum
database
highestPaid
red escaped
white captured
this
e
14
Escaped nodes

• Escaped nodes
• parameter nodes
• class nodes
• thread nodes
• nodes in return set
• nodes reachable from other escaped nodes
• captured is the opposite of escaped

15
Dataflow Analysis

• Computes a points-to escape graph for each
  program point
• Points-to escape graph is a triple ltI,O,egt
• I - set of inside edges
• O - set of outside edges
• e - escape function

16
Dataflow Analysis

• Initial state
• I formals point to parameter nodes,
• classes point to class nodes
• O Ø
• Transfer functions
• I (I KillI ) U GenI
• O O U GenO
• Confluence operator is U

17
Intraprocedural Analysis

• Must define transfer functions for
• copy statement l v
• load statement l1 l2.f
• store statement l1.f l2
• return statement return l
• object creation site l new cl
• method invocation l l0.op(l1lk)

18

• copy statement l v
• KillI edges(I, l)
• GenI l succ(I, v)
• I (I KillI ) ? GenI

Existing edges
l
v
19

• copy statement l v
• KillI edges(I, l)
• GenI l succ(I, v)
• I (I KillI ) ? GenI

Generated edges
l
v
20

• load statement l1 l2.f
• SE n2 ? succ(I, l2) . escaped(n2)
• SI ?succ(I, n2,.f) . n2 ? succ(I, l2)
• case 1 l2 does not point to an escaped node (SE
  Ø)
• KillI edges(I, l1)
• GenI l1 SI

Existing edges
l1
f
l2
21

• load statement l1 l2.f
• SE n2 ? succ(I, l2) . escaped(n2)
• SI ?succ(I, n2,.f) . n2 ? succ(I, l2)
• case 1 l2 does not point to an escaped node (SE
  Ø)
• KillI edges(I, l1)
• GenI l1 SI

Generated edges
l1
f
l2
22

• load statement l1 l2.f
• case 2 l2 does point to an escaped node (SE ? Ø)
• KillI edges(I, l1)
• GenI l1 (SI ? n)
• GenO (SE f) n

Existing edges
l1
l2
23

• load statement l1 l2.f
• case 2 l2 does point to an escaped node (SE ? Ø)
• KillI edges(I, l1)
• GenI l1 (SI ? n)
• GenO (SE f) n

Generated edges
l1
f
l2
24

• store statement l1.f l2
• GenI (succ(I, l1) f) succ(I, l2)
• I I ? GenI

Existing edges
l1
l2
25

• store statement l1.f l2
• GenI (succ(I, l1) f) succ(I, l2)
• I I ? GenI

Generated edges
l1
f
l2
26

• object creation site l new cl
• KillI edges(I, l)
• GenI ltl, ngt

Existing edges
l
27

• object creation site l new cl
• KillI edges(I, l)
• GenI ltl, ngt

Generated edges
l
28
Method call

• Transfer function for method call
• Take points-to escape graph before the call site
• Retrieve the points-to escape graph from analysis
  of callee
• Map callee graph into caller graph
• Result is the points-to escape graph after the
  call site

29
Interprocedural Mapping

• Set up a mapping between caller and callee
• outside nodes in the callee may refer to any
  number of inside nodes in the caller
• add all reachable inside edges from callees
  graph into callers graph
• outside edges from a node in the callee need to
  be added to the mapped caller node if it escapes

30
Interprocedural Mapping Example

• void printStatistics()
• BufferedReader r new BufferedReader(
• new InputStreamReader(System.in))
• EmployeeDatabase e new EmployeeDatabase(r)
• e.computeMax()
• System.out.println(max salary
  e.highestPaid)

31
Interprocedural Mapping Example
void printStatistics() BufferedReader r new
BufferedReader( new InputStreamReader(System.in)
) EmployeeDatabase e new EmployeeDatabase(r)
e.computeMax() System.out.println(max salary
e.highestPaid)
graph before call site

elementData
database
e
32
Interprocedural Mapping Example
void printStatistics() BufferedReader r new
BufferedReader( new InputStreamReader(System.in)
) EmployeeDatabase e new EmployeeDatabase(r)
e.computeMax() System.out.println(max salary
e.highestPaid)
graph before call site

elementData
database
e
callee graph

elementData
database
this
highestPaid
Enum object is not present because it was
captured in the callee.
33
Step 1 Map formals to actuals
void printStatistics() BufferedReader r new
BufferedReader( new InputStreamReader(System.in)
) EmployeeDatabase e new EmployeeDatabase(r)
e.computeMax() System.out.println(max salary
e.highestPaid)
graph before call site

elementData
database
e
callee graph

elementData
database
this
highestPaid
34
Step 2 Match edges to extend mapping
void printStatistics() BufferedReader r new
BufferedReader( new InputStreamReader(System.in)
) EmployeeDatabase e new EmployeeDatabase(r)
e.computeMax() System.out.println(max salary
e.highestPaid)
graph before call site

elementData
database
e
callee graph

elementData
database
this
highestPaid
35
Step 3 Map nodes and edges to construct new graph
graph after call site

elementData
database
e
highestPaid
graph before call site

elementData
database
e
callee graph

elementData
database
this
highestPaid
36
Key Feature

• Even if an object escapes one method
• Often possible to recapture object in caller
  methods
• Common in practice
• Iterators
• Objects that hold multiple return values

37
Life is not so Simple

• Dependences between phases
• Mapping best framed as constraint satisfaction
  problem
• Solved using constraint satisfaction

38
Algorithm Features

• Compositional
• Analyze each method once
• Obtain parameterized result
• Reuse result at different call sites
• Independent preanalysis of libraries
• Partial
• Can analyze method without analyzing methods it
  invokes
• Useful if not all code available in analyzable
  form

39
Incrementalized Analysis

• Compositional Partial Enables
  Incrementalization
• Choose object to attempt to capture
• Analyze method containing allocation site
• Track where object escapes
• Specific call sites
• Caller
• Incrementally analyze only those parts
• Usual Result
• Significant reduction in analysis time
• Almost all benefit of full analysis

40
Key Limitation (so far)

• No analysis of interactions between threads
• Objects that escape from allocating thread are
  NEVER recaptured
• Solution extend algorithm to analyze
  interactions between threads
• Challenge avoid analyzing all interleavings of
  statements from parallel threads

41
Interactions Between Threads
Heap
b
a
d
c
42
Interactions Between Threads
Heap
b
a
d
c
White Thread
Yellow Thread
43
Interactions Between Threads
Heap
b
a
d
c
White Thread
Yellow Thread
a.f b
44
Interactions Between Threads
Heap
b
a
d
c
t
White Thread
Yellow Thread
a.f b
t a.f
45
Interactions Between Threads
Heap
b
a
d
c
t
White Thread
Yellow Thread
a.f b
t a.f t.f c
46
Interactions Between Threads
Heap
b
a
d
c
p
White Thread
Yellow Thread
a.f b p b.f
t a.f t.f c
47
Interactions Between Threads
Heap
b
a
d
c
p
White Thread
Yellow Thread
a.f b p b.f p.f d
t a.f t.f c
48
Interactions Between Threads
Heap
b
a
d
c
White Thread
Yellow Thread
a.f b p b.f p.f d
t a.f t.f c
49
Important Properties

• Result Depends on Specific Interleaving
• Analyze all Interleavings?
• Iterate Across Threads to Fixed Point?

50
Important Properties

• Result Depends on Specific Interleaving
• Analyze all Interleavings?
• Iterate Across Threads to Fixed Point?
• Analyze Each Thread Once
• Parameterized Analysis Result
• Characterizes All Potential Interactions
• Combine Analysis Results From Different Threads
  to Compute Actual Interactions
• Compositional Analysis

51
Our Approach

• Build Points-To Escape Graph
• Result Characterizes Potential Interactions
• Inside Edges - Represent References Created By
  Currently Analyzed Thread
• Outside Edges - Represent References Created By
  Parallel Threads
• Inside Nodes - Represent Objects Created By
  Current Analyzed Thread
• Outside Nodes - Represent Objects Accessed Via
  Outside Edges

52
Analysis of Each Thread
Analysis Result for Yellow Thread
Analysis Result for White Thread
b
a
Yellow Thread
White Thread
t a.f t.f c
a.f b p b.f p.f d
53
Analysis of Each Thread
Analysis Result for Yellow Thread
Analysis Result for White Thread
b
a
1
Outside Edge
Yellow Thread
White Thread
t a.f t.f c
a.f b p b.f p.f d
54
Analysis of Each Thread
Analysis Result for Yellow Thread
Analysis Result for White Thread
b
a
d
1
Yellow Thread
White Thread
t a.f t.f c
a.f b p b.f p.f d
55
Analysis of Each Thread
Analysis Result for Yellow Thread
Analysis Result for White Thread
2
a
b
a
d
1
Yellow Thread
White Thread
t a.f t.f c
a.f b p b.f p.f d
56
Analysis of Each Thread
Analysis Result for Yellow Thread
Analysis Result for White Thread
2
a
c
b
a
d
1
Yellow Thread
White Thread
t a.f t.f c
a.f b p b.f p.f d
57
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Analysis Results From Threads
b
a
d
1
2
a
c
58
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Analysis Results From Threads
b
a
d
1
Mapping
2
a
c
59
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Analysis Results From Threads
b
a
d
1
Mapping
2
a
c
60
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Analysis Results From Threads
b
a
d
1
Mapping
2
a
c
61
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Combined Result
Analysis Results From Threads
b
a
d
1
Mapping
b
a
2
a
c
62
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Combined Result
Analysis Results From Threads
b
a
d
1
Mapping
b
a
c
2
a
c
63
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Combined Result
Analysis Results From Threads
b
a
d
1
Mapping
b
a
d
c
2
a
c
64
Combining Analysis Results

• Match Corresponding Edges
• Map Outside Nodes to Nodes that They Represent
  During the Analysis
• Use Mapping to Combine Graphs

Combined Result
Analysis Results From Threads
b
a
d
1
1
Mapping
b
a
d
c
2
a
c
2
65
Recapturing Nodes

• If a, b, c, and d may be recaptured after
  analyzing interactions between threads
• Have complete points-to information for
  recaptured nodes

Combined Result After Interthread Analysis
Combined Result After Recapturing Nodes
1
b
a
d
c
b
a
d
c
2
66
Common Usage Pattern

• Current Thread
• Creates and Initializes Objects (Synchronization)
• Passes Objects As Parameters to New Thread
• Never Accesses Objects Again

• New Thread
• Starts Running
• Accesses Objects (Synchronization)

67
Key Enhancement

• Instrument Analysis to Record
• Actions on Nodes
• synchronization operations
• reads and writes
• Ordering Between Actions and Thread Starts
• Use Ordering to Rule Out Potential Interactions
• Synchronizations from different threads
  temporally separated by thread creation events

68
Server Benchmark Characteristics
Inter Thread Analysis Time (secs)
Intra Thread Analysis Time (secs)
Pre Analysis Time (secs)
IR Size (instrs)
Number of Methods
4,639
131
28
74
73
echo
time
4,573
136
29
70
74
http
10,643
292
103
199
269
phone
9,547
267
75
191
256
69
Percentage of Eliminated Synchronization
Operations
100
80
Interprocedural
60
Only
Interthread
40
20
0
http
phone
time
echo
mtrt
70
Complications

• Using Connectivity Information to Define Concepts
  of Escaped and Captured Nodes
• Recapturing Nodes After Combining Results
• Treating Escaped and Captured Nodes Differently
  During Analysis and Combination
• Escaped Nodes Can Have Outside Edges
• Captured Nodes Have No Outside Edges
• Recursively Generated Threads
• Accurate Call Graph
• Modeling Caller/Callee Interaction

71
Application to Real-Time Java

• Real-Time Java has Scoped Memory (regions)
• Motivation
• Keep Javas safe memory model (no explicit
  deallocation)
• Obtain predictable allocation times (no garbage
  collection)
• Solution
• Preallocate a subregion of memory
• Task allocates its objects in predictable time
  from that region
• Region deallocated as a unit when done

72
Scoped Memory Model

• Scoped Memory is a separate object
• Can run a computation in a Scoped Memory (restore
  old memory when finished)
• Get a tree of nested computations, each with its
  Scoped Memory
• Interaction with Threading
• New thread can use same scoped memory as parent
  thread
• Or can use new scoped memory

73
Nested Scoped Memories
Scoped Memory
Object
74
Referencing Constraints
Scoped Memory
Object
Referencing Down Scopes Is NOT OK
Referencing Up Scopes Is OK
75
Preventing Downward References

• Reference Checks Done Dynamically
• At every write of a reference into an object
  field or array element
• Check that written object is allocated in a scope
  below that of referred object
• If not, throw an exception
• Drawbacks
• Dynamic checking overhead
• Errors when program runs

76
Escape Analysis

• Escape analysis provides information about how
  lifetimes of objects relate to computation
• Use escape analysis to automatically insert
  scoped memories
• Use escape analysis results to check programs
  with explicit scoped memories
• Check that NO object escapes computation that
  executes in Scoped Memory
• If no object escapes, then program will never
  violate referencing constraints
• Eliminate dynamic checks
• Eliminate potential source of errors
• Analyzing interactions between threads may be
  crucial (depending on usage patterns)

77
Implementation Status

• FLEX compiler infrastructure
• Full Java compiler
• Lots of utilities and packages
• Support for deep program analyses and
  transformations
• Implemented Scoped Memory checks
• Implemented combined points-to escape analysis
• Used analysis results to eliminate scoped memory
  checks
• Benchmarks
• Matrix Multiply (with Integers)
• Linked List Sum (with Integers)

78
Results

• Verified that programs do not violate
  ScopedMemory constraints
• Execution Times (seconds)

No Checks
Checks
Matrix Multiply
35.6
31.7
Linked List Sum
4.00
3.05
79
Conclusion

• Combined Points-to and Escape Analysis
• Compositional, Partial, Incrementalized
• Implemented Transformations
• Stack Allocation, Private Heap Allocation,
  Synchronization Elimination, Scoped Memory Check
  Elimination
• Foundation for Interaction Analysis
• Information about system behavior
• Customized implementations
• Functionality-improving transformations
• Optimized adaptation handling