The Simplest Heuristics May Be The Best in Java JIT Compilers

1
The Simplest Heuristics May Be The Best in Java
JIT Compilers

• ACM SIGPLAN 2003
• Presented by
• Mithuna Soundararaj

2
Outline

• Introduction
• Motivation
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

3
Introduction

• Simplest strategy in Java JIT compiler is to
• compile each java method the first time it is
  called.
• Better performance if selectively compiled
• Selectively compiled based on some heuristics
  like
• Frequency of method calls
• Size of the method
• More complicated heuristics do not provide any
  additional benefits.

4
Outline

• Introduction
• Motivation
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

5
Motivation

• To see how addition of selective compilation
  heuristics can improve the performance of a
  particular existing Virtual machine and JIT
  compiler.
• To speed up, the slow start up time of many
  graphical user interfaces,which may lead to users
  changing their behavior regarding the
  application.

6
Outline

• Introduction
• Motivation
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

7
Architecture of JVM and JIT
Java Class Loaded Initialization for JIT usage

Invoker Field-gtJIT compiler
When method compiled, JIT translates methods
byte code to native code
Invoker Field-gtGenerated
m/c code
When method
Re Invoked
Executes machine code or native code
8
Compiling Methods
Original VS Modified JVM

• Originally in sun versions of sunwjit
• Methods are compiled unconditionally the first
  time they are called
• Methods interpreted when
• JIT suppressed by command line
• JIT internal error while compiling
• Method belongs to primordial class
• In the modified version of sunwjit by caldera,
  Selective compilation is introduced.

9
Outline

• Introduction
• Motivations
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

10
Selective compilation

• Selectively compiling methods based on heuristics
  like
• Simple heuristics
• Based on frequency of method execution
• Based on size of methods
• Complicated heuristics
• Jit when jitted heuristic
• Square decision heuristic
• Backward branches heuristic
• core classes known to compile heuristic

11
Outline

• Introduction
• Motivations
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

12
Frequency of Method Call

• To compile methods that are likely to be executed
  most number of time.
• Methods compiled many times are likely to be
  executed many times in the future as well
• Environment variable set to some positive
• integer N
• Method will be compiled after it has executed N
  times as set in the environment variable

13
Method Size

• Heuristic depends on bytes of byte code for the
  method
• Larger method take longer to execute and compile?
• Compile a small method thats going to execute
  only once?
• Compile large methods that execute many times?

14
Boundaries of the decision

• Point where a method size is big enough that you
  want to compile it
• Even if it executes once( large methods)
• Point where method compiled, if it executes a
  certain number of times.
• Even if it is very small

15
Selective CompilationBased on Method size and
times executed
16
Outline

• Introduction
• Motivations
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

17
Results-Simple heuristics
18
Start up time results

• Startup time is important for psychological
  
• reasons(an area ripe for optimization)
• 77 of execution time java compilation over
• head, occurs in the initial 10 of program
  
• execution

19
Profiling results
20
Profiling results
21
Profiling results
22
Outline

• Introduction
• Motivation
• Background
• Selective Compilation
• Simple heuristics
• Results
• Complicated heuristics
• Conclusion

23
Complicated heuristics

• Jit when called by jitted heuristic
• Compile method X gtcompile every method X calls
• To take advantage of locality of reference
• Method X gets used a lot gt methods Y, Z etc.that
  X calls will get used a lot too.
• Compile Y,Z right away
• Tarantella object manager is 20 , SwingSet2 is
  21 slower and javac is 3 slower to startup and
  run respectively
• No test results run faster with it gt no locality
  of reference exists.

24
Complicated Heuristics

• Square decision heuristic
• Decision could be either-or if a method exceeds
  the minimum size or minimum number of times to be
  executed, it is compiled
• Shape of the graph would be a right angle between
  JIT_MIN_TIMES and JIT_MIN_SIZE
• SPEC JVM98 is 1 slower and Tarantella Object
  manager startup is 3 worse than the default
  scheme
• This scheme was found to be less flexible than
  the default scheme, hence not used

25
Complicated Heuristics
Method size in bytes
Square decision heuristic
26
Complicated heuristics

• backward branches heuristic
• Identifies backward branches in methods
• Assume the branches are loops and hot spots
• Such methods would be compiled immediately
• (as in IBM JIT compiler)
• Implementing this heuristic in SCO/Caldera JIT
  difficult
• Similar results to default scheme

27
Complicated heuristics

• core classes known to compile heuristic
• Uses data captured by off line profiling
• Decisions on whether to compile a method
  immediately- made at the beginning of the
  application execution
• Based on whether it is known that the method
  will reach its crossover point where
  compilation is sure to be beneficial.
• Optimizations no more transparent to the user
• Lot of off-line processing, user visible feedback
  loop
• Used efficiently in java core libraries that are
  used heavily in all applications
• Javac was 9 faster and SPEC JVM98 was 2 better.

28
Summary of previous work

• IBM JIT compiler
• no of times method executes modified by
  detection of loops
• HP JIT(for embedded )
• Minimum execution of 2 times before compilation
• Krintz and calder
• used metrics found by experiment
• 25 of most frequently executed methods must be
  compiled for optimum performance
• Use profiling data to identify those 25 methods
  offline

29
Conclusion

• Adding simple selective compilation heuristics,
  to an existing Java JIT compiler significantly
  improves its performance for short and
  medium-lived applications and for start-up time
  in longer-lived applications
• Results have shown that the combined decision
  strategy is superior to either one or to
  unconditionally compiling and not using JIT at
  all

30
Thank You!

• Questions??