A New Transpant JAVA Thread Migration System Using Just-in-Time Recompilation

1
A New Transpant JAVA Thread Migration System
Using Just-in-Time Recompilation

• Wenzhang Zhu, Cho-Li Wang,
• Weijian Fang, and Francis C.M. Lau
• The University of Hong Kong

2
Java Thread

• Lightweight computation unit
• Supports concurrency
• Supported by Java at language level
• Shared memory paradigm
• Multithreading used in large Java programs e.g.
  Tomcat, JBoss

3
True Parallelism for Threads

• Multiple threads of a program spreading over
  multiple compute nodes in a cluster
• A global object space to support data sharing
• JESSICA2 8

threads
Heap
objects
CPU1
CPU2
CPU3
4
Dynamic Thread Migration

• To achieve load balancing and best performance
  for applications

underloaded
overloaded
balanced workload
5
Example of Thread Migration
Program 1 i1 2 j2 3 kij

• We need to carry the code and its execution state
  to remote node
• What is in the execution state? At least
• the program counter
• the intermediate values of variables

Node 1
i1 j2
Node 2
kij
6
Previous Approaches

• User level
• Bytecode level 67
• JVM level 45, with JIT 9
• Normal thread execution suffers
• time checking flags and flushing registers to
  memory
• space code size expanded
• A multithreaded Java application will have gt95
  normal execution, lt5 migration
• What we present will help optimize the 95

7
JIT Compilation

• Can we migrate native code?
• No, machines are not the same

Javabytecodeinexecution
Native code
Migrate?
JITC
8
New Approach

• Derive portable bytecode-oriented thread context
  (BTC) from native (raw) thread context (RTC)
• Restore BTC to RTC
• Re-run JIT compilation in both phases to
  transform the thread context

Source node
Target node
9
The Detailed Steps

• Very quickly

10
Just-in-Time Recompilation (source node)
Stack walk
compute()
main()
JVM-kernel function()
11
Just-in-Time Recompilation (source node)
Stack walk
Frame segmentation
Java frames to migrate
compute()
main()
JVM-kernel function()
12
Just-in-Time Recompilation (source node)
Stack walk
Frame segmentation
compute()
Bytecode PC positioning
iinc 1, 1 iload_1 bipush 30 if_icmplt 5
incl ebx cmpl 0x1e, ebx jl 0x82512432
Mapping
main()
13
Just-in-Time Recompilation (source node)
Stack walk
Frame segmentation
compute()
Bytecode PC positioning
iinc 1, 1 iload_1 bipush 30 if_icmplt 5
incl ebx cmpl 0x1e, ebx jl 0x82512432
Breakpoint selection
breakpoint
getstatic 2
return
14
Just-in-Time Recompilation (source node)
Stack walk
compute()
Frame segmentation
local 0 java.lang.thread local 1 int stack 0
float stack 1 int
Bytecode PC positioning
Breakpoint selection
Type derivation
main()
local 0 java.lang.thread stack 0 double stack
2 double
15
Just-in-Time Recompilation (source node)
Stack walk
re-generate native code with breakpoints
Frame segmentation
compute()
Bytecode PC positioning
inc ebx cmpl 1e, ebx jl 0x82512432 0x82512432
capture frames --- save local vars and their
type return
getstatic 2 Iload_1
Breakpoint selection
Type derivation
Translation
main()
...other native code... jmp start_migration
16
Just-in-Time Recompilation (source node)
Stack walk
replace saved return address with the
newly generated code
Frame segmentation
compute()
Bytecode PC positioning
inc ebx cmpl 1e, ebx jl 0x82512432 0x82512432
capture frames --- save local vars and their
type return
saved return address
Breakpoint selection
activation record of compute()
Type derivation
main()
Translation
Native code patching
...other native code... jmp start_migration
17
Just-in-Time Recompilation (target node)
Thread creation
Frame
PC
18
Just-in-Time Recompilation (target node)

• use JIT compiler to derive register
• mapping at restoration point generate code stub

Thread creation
compute()
Dynamic register patching
Register recovering stub movl 0x1234, eax movl
0x5678, ebx ...
main()
Register recovering stub movl 0x1111, eax ...
19
Just-in-Time Recompilation (target node)
Compiled methods
reg1 lt- value1 jmp restore_point1
Stack growth
compute() ... retore_point1
Thread creation
frame 1
ebp
Dynamic register patching
Ret addr
reg1 lt- value1 reg2 lt- value2 jmp restore_point0
main() ... retore_point10
Stack rebuilding
frame 0
ebp
Ret addr
trampoline frame
Linking code stub and native frames
trampoline
bootstrap frame
bootstrap() trampoline() closing handler()
ebp
20
Experimentation

• Cluster of 2GHz P4 PCs, running Linux kernel
  2.4.22
• Normal runtime overhead 0
• Migration latency hiding
• Pre-loading classes needed at target node when
  the frames have been segmented
• Overlapping class loading with stack transform at
  source node

21
Migration Overhead
DNIC Dynamic Native Code Instrumentation
approach JITR Just-in-Time recompilation
approach JITRPreload Just-in-Time recompilation
with class preloading enabled
22
Multithreaded Java Application Server Simulation
23
Parallel Mesh Refinement Simulation
24
Conclusion

• This paper presents an efficient transparent Java
  thread migration system using the technique of
  JIT recompilation without code instrumentation
• The idea of using JIT recompilation demonstrates
  a new use of the JIT compiler to gather runtime
  information

25
JESSICA2 Website
www.csis.hku.hk/clwang/projects/JESSICA2.html