Title: Memory Management for Real-Time Java
1Memory Management for Real-Time Java
- Wes Beebee and Martin Rinard
- Laboratory for Computer Science
- Massachusetts Institute of Technology
- Supported by DARPA Program Composition for
Embedded Systems (PCES) Program
2Goal Enable Use of Java for Real-Time and
Embedded Systems
3Vision
Standard Java Applications
Real-Time Computation In Java
Downloaded Java Applets
Unified Language/Environment Facilitates
Interaction
4Why Java?
- Type safe language, no memory corruption
- Reasonably modern approach
- Object oriented
- Garbage collected memory management
- Popular and supported
- Programmers available
- Tools available
- Libraries available
5Implications and Issues
- Heterogeneous components with different needs and
goals - Real-time computation
- User interface
- Data management
- Issues
- Memory management
- Scheduling
- Event management and delivery
- Processor allocation
6Why NOT Java
- Unpredictable memory usage
- Dynamic memory allocation
- Allocation hidden in extensive set of libraries
and native methods - Allocation hidden in exception model
- Unpredictable execution times
- Garbage collection
- No scheduling guarantees
- Thread scheduling
- Event delivery
- Complex libraries and native methods
7Why NOT Java
- Impoverished set of abstractions
- Threads, mutex locks, signal and wait
- No good way express relationship between
- Events in system
- Corresponding pieces of computation
- No good way to express timing expectations
- Real-Time Java Approach
- Extend library
- Native methods for new mechanisms
8Real-Time Java Standard
- Goal Augment Java to better support real-time
systems - Augment memory model to enable threads to avoid
garbage collection pauses - Augment thread scheduling model to add more
control over task scheduling - Augment synchronization model to include
lightweight event delivery mechanism
9Our View
- Real-time Java is a work in progress
- Many of extensions generate
- More complex programming model
- More possibilities for errors
- Our goal
- Isolate general principles/concepts we think will
last - Develop new program analyses and implementation
mechanisms - That help programmers use real-time extensions
safely and effectively
10Java Memory Models
- Java single garbage-collected heap
- Real-time Java multiple kinds of memories
- Garbage-collected heap memory
- Immortal memory (live for full computation)
- Scoped memories
(live for specific subcomputations) - Linear-time allocation (LTMemory)
- Variable-time allocation (VTMemory)
11Problems/Issues with Memory Model
- Scoped memory issues
- Scoped memory reference checks
- Scoped memory sizes
- Avoiding garbage collection interaction issues
- No-heap real-time thread access checks
- Priority inversions caused by indirect
interactions with garbage collector
12Scoped Memory Overview
Standard Java Computation
13Scoped Memory Overview
Objects in GC Heap
Standard Java Computation
14Scoped Memory Overview
Objects in GC Heap
Standard Java Computation
15Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
16Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Scoped Memory
New Thread Runs In Scoped Memory
17Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Threads New Objects Allocated in Scoped Memory
18Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Threads New Objects Allocated in Scoped Memory
19Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Threads New Objects Allocated in Scoped Memory
20Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Computation Terminates
21Scoped Memory Overview
Objects in GC Heap
New Computation Typically new thread Maybe even
real-time thread
Standard Java Computation
Objects
Scoped Memory
Objects in Scoped Memory Deallocated as a
Unit without GC
22Scoped Memory Motivation
- Dynamic memory allocation without GC
- Tie object lifetimes to computation lifetimes
- Eliminate need to dynamically trace out reachable
objects - Warning
- Example illustrates primary intended use
- Specification allows more behaviors
- Scoped memories shared by multiple threads
- Nested scoped memories
- Scoped memories entered multiple times
23Safety Issue for Scoped MemoriesDangling
References
- Lifetimes of objects in scoped memory determined
by lifetime of computation - Must ensure that no reference goes from
long-lived object to short-lived object
24Nested Scoped Memories
Scoped Memory
Object
25Referencing Constraints
Scoped Memory
Object
Referencing Down Scopes Is NOT OK
Referencing Up Scopes Is OK
26Preventing Downward References
- Dynamic Reference Checks
- At every write of a reference into an object
field or array element - Check that written object is allocated in a scope
with a lifetime at least as long as that of
referred object - If not, throw an exception
- Drawbacks
- Dynamic checking overhead
- New class of dynamic errors
27Static Analysis
- Goal
- Eliminate need for dynamic checks by
- Statically checking that program does not violate
referencing constraints - Basic approach escape analysis
28What Escape Analysis Provides
void compute(d,e)
- Control Flow Graph
- Nodes methods
- Edges invocation relationships
void multiplyAdd(a,b,c)
void multiply(m)
void add(u,v)
29What Escape Analysis Provides
void compute(d,e)
- Control Flow Graph
- Nodes methods
- Edges invocation relationships
void multiplyAdd(a,b,c)
void multiply(m)
void add(u,v)
Allocation Site
30What Escape Analysis Provides
void compute(d,e)
- Control Flow Graph
- Nodes methods
- Edges invocation relationships
void multiplyAdd(a,b,c)
void multiply(m)
void add(u,v)
Object Allocated Here Does Not Escape Computation
of multiplyAdd method
Allocation Site
31Our Escape Analysis
- Interprocedural
- Analyzes interactions between methods
- Recaptures objects in callers of allocating
methods - Compositional
- Analyzes each method once
- Single analysis result that can be specialized
for use in different calling contexts - Suitable for multithreaded programs
- Analyzes interactions between threads
- Recaptures objects that do not escape a given
multithreaded computation
32Using Escape Analysis to Verify Correct Use of
Scoped Memories
- For each computation that runs in scoped memory
- Check that allocated objects do not escape
33Implementation
- FLEX compiler infrastructure (www.flexc.lcs.mit.ed
u) - Full Java compiler
- Lots of utilities and packages
- Support for deep program analyses and
transformations - Implemented scoped memories and checks
- Implemented escape analysis
- Used results to eliminate checks
- In applications, eliminated all checks
34Experimental Results
120
100
80
Scope Checks
Time (sec)
60
Application
40
20
0
Array
Array
Tree
Tree
Water
Water
Barnes
Barnes
(Heap)
(Scope)
(Heap)
(Scope)
(Heap)
(Scope)
(Heap)
(Scope)
Benchmarks
35Scoped Memory Sizes
- Scoped memory creation and size
- MemoryArea ma new LTMemory(10000)
- create a new scoped memory with 10,000 bytes
- If try to allocate more than 10,000 bytes,
implementation throws an exception - Problems
- Java does not specify object sizes
- Size of given object may change during its
lifetime in computation - So how big to make scoped memory?
36Modularity Problems
Objects in GC Heap
Scoped Memory Size Determined Here
Standard Java Computation
Objects
Scoped Memory
Required Size Determined by Behavior of Code in
this Computation
37Modularity Problems
Objects in GC Heap
Scoped Memory Size Determined Here
Standard Java Computation
Objects
Scoped Memory
- If change program, size may need to change!
- Amount of allocated memory becomes part of
interface!
38More Issues
- Different executions may allocate different
amounts of data - Lots of hidden allocation in libraries
- Difficult to find out how much memory is really
allocated - If change implementation, may need to change
scoped memory size in clients
39Analysis Solution
- Analyze program to symbolically compute allocated
memory sizes - Input variables
- Object sizes
- Compiler knows object sizes, can conservatively
generate scoped memory sizes
40Interaction with Garbage Collector
- Standard Collector Assumptions
- Can interrupt computation at any point
- Can suspend for unbounded time
- Real-Time Java extension
- No-Heap Real-Time Threads
- Can Access
- Immortal memory
- Scoped memory
- Do not interact with GC heap AT ALL
- Can run asynchronously with GC
Immortal
Scoped
GC Heap
41No-Heap Real-Time Thread Checks
- Dynamically check that no-heap real-time threads
never access a location containing a reference
into garbage-collected heap - At every read, check to make sure result does not
point into garbage-collected heap - At every write, check to make sure not
overwriting reference into GC heap - If check fails, throw exception
- Drawbacks
- Dynamic checking overhead
- New class of dynamic errors
42Implementation
- FLEX compiler infrastructure (www.flexc.lcs.mit.ed
u) - Implemented no-heap real-time threads
- Implemented access checks
- Measured performance with and without checks
43Experimental Results
Scope Checks
Heap Checks
300
Application
250
200
150
Time (sec)
100
50
0
Array
Tree
Water
Benchmark
44Program Analysis for Eliminating Checks
- Control-flow analysis to identify code that may
execute in no-heap real-time thread - Global value-flow analysis
- Tags each value that points to GC heap
- Identifies all locations into which these values
may flow - Combine results
- Look at all no-heap real-time thread code
- Check statically for access violations
45Analysis Issues and Solutions
- Complicated, whole-program analysis
- Simple annotations can enable local analysis
- Annotate each reference with source
- Scoped memory
- Immortal memory
- Heap memory
- Annotation checker validates annotations
- Result
- Scalable analysis
- Eliminate checks, eliminate programming errors
46Indirect Priority Inversions
Interaction Between Resource Sharing and Garbage
Collection
Lock Acquire
No-heap Thread
Garbage Collector
Standard Java Thread
Lock Acquire
47Indirect Priority Inversions
Interaction Between Resource Sharing and Garbage
Collection
Lock Acquire
No-heap Thread
Blocks
Garbage Collector
Standard Java Thread
Lock Acquire
48Indirect Priority Inversions
No-heap thread must wait for standard Java
thread to release lock Standard thread must
wait for GC to finish (heap inconsistent until
it finishes) No-heap thread must wait for GC!
Lock Acquire
No-heap Thread
Garbage Collector
Standard Java Thread
Lock Acquire
49Using Non-Blocking Synchronization to Eliminate
Indirect Priority Inversions
Start Atomic Region
End Atomic Region
No-heap Thread
Does Not Block!
Garbage Collector
Abort, Retry
Standard Java Thread
Start Atomic Region
50Implementation Status
- Non-blocking synchronization implemented for
memory management primitives - Useful when threads share scoped memory
- Uses non-blocking synchronization instructions
from processor - Software implementation underway for general
atomic regions
51Goal
Enable safe real-time code to interact
successfully with code that accesses GC data
Issues and Complications
- Dangling references for scoped memories
- Resource needs of computations
- Isolating computations from garbage collector
- Ensure threads with real-time constraints dont
access garbage collected data - Eliminate indirect interactions
- Our view
- Dynamic checks inadequate
- Statically verify correct use, eliminate checks
52Broader View of Real-Time Java
- Java is best suited to express batch
computations on objects - Not so good for control in asynchronous,
parallel, distributed, time-aware systems - Inadequate for design/requirements
- Can be part of solution, but only a part
53Multiple Perspectives
- Any system has many desired properties
- Data structure invariants
- Flow of data between different components
- Timing requirements for computations
- Each property is inherently partial
- Many properties are best expressed as
- Declarative constraints
- NOT translatable into implementation
54Design Conformance
Object Referencing Relationships
Dataflow Interactions
Timing Constraints
Check that implementation conforms to design
properties
Implementation
55Future
- Scheduling and event delivery
- More precise referencing relationship analysis
- Completely characterize aliasing behavior
- More flexible memory management algorithms that
preserve predictability - More flexible regions
- Immediate deallocation
- Less restricted memory access constraints for
real-time threads - Design conformance for more control-oriented
properties
56Summary
- Real-time Java code coexists and interacts with
standard Java code - New complications (overhead failure modes)
- Scoped memory checks
- Scoped memory sizes
- No-heap real-time threads
- Indirect priority inversions
- Attacked with program analysis
- Future scheduling and timing