JavaGenes and Condor: Cycle-Scavenging Genetic Algorithms

1
JavaGenes and Condor Cycle-Scavenging Genetic
Algorithms
Al Globus, CSC at NASA Ames Research Center Eric
Langhirt, AMTI at NASA Ames Research Center
Miron Livny, University of Wisconsin
Ravishankar Ramamurthy, University of Wisconsin
Marvin Solomon, University of Wisconsin Steve
Traugott
2
Abstract

• JavaGenes implements a genetic algorithm using a
  graph representation
• Stochastic
• Embarrassingly parallel
• Executed hundreds of multi-day jobs on Condor
  cycle-scavenging batch system using otherwise
  idle workstations
• Pharmaceutical drug molecules and digital
  circuits have been evolved with reasonable
  success
• Java good, but serialization may be too slow

3
Genetic algorithm

• Randomly generate a set of molecules
• Rate them with a fitness function
• Many times
• Select parent molecules at random with bias
  towards better fitness
• Randomly rip copies of each parent in two
• Mate opposite halves and rate new molecules
• Replace random molecules with bias towards worse
  fitness
• Repeat until satisfied

4
Crossover
5
JavaGenes in Action
Finding
with all-pairs-shortest-path and Tanimoto index
fitness function (0 is perfect)
6
Condor

• Cycle-scavenging batch system for single
  workstation jobs
• University of Wisconsin (http//www.cs.wisc.edu/co
  ndor)
• In production since 1986
• Unix workstations (limited Windows NT)
• 200 SGI and 50 Sun workstations at NASA Ames
• Good for
• parameter studies
• stochastic algorithms (e.g., genetic algorithms)

7
Condor and Java

• Standard universe
• link with Condor library
• automatic io redirection
• automatic checkpointing
• no kernel level threads (e.g., no fork())
• impractical for Java jobs
• Vanilla universe jobs worked
• Disk access via NFS
• Periodic application-specific checkpointing
• At startup, check for checkpoint file

8
Byte Code Checkpointing Problematic

• Condor jobs may be stopped at any arbitrary time
• Stack format not defined
• Heap format undefined and serialization slow
• JIT and optimization causes problems
• Must modify or write Java Virtual Machine
• Kaffe couldnt run JavaGenes
• There are rumors regarding checkpointable JVMs
• Pointers welcome

9
Java Universe

• Application implements Checkpointable and
  Serializable
• start(String arguments)
• restart()
• Application calls
• Checkpointer.ok()
• Checkpointer.checkpoint()
• Condor (or checkpointing thread) calls
• Checkpointer.checkpointWhenPossible()
• Checkpointer.prepareToDie()
• Checkpointer.areYouReadyToDie()
• Checkpointer.cancelDeath()

10
Java Con

• Serialization seemed unbelievably slow and big
• three hours and 25 Mbytes for 500 graphs
• plus memory leak found last month!
• two day application-specific checkpoint
  implementation lt 1Mbyte / 10 seconds - 5 minutes
• Restart from checkpoint leads to optimistic
  performance data
• class Double problem
• double foo Infinity // or NaN
• Double(foo ) throws bad format exception

11
Java Pro

• Porting trivial
• Unix SGI and Sun JDK 1.1.x
• Windows 95 Visual Cafe, Superseed, JBuilder,
  Code Warrior, JDK 1.1.x
• Bugs
• significantly less frequent than in C or C
• mostly logic bugs, easy to find and fix
• Memory Management very easy
• one clone() bug
• one memory leak quickly found by OptimizeIt

12
Summary

• Except for serialization, performance is adequate
  though not spectacular
• Wrote once ran everywhere we wanted
• No GUI
• Accomplished non-Java, non-Condor computational
  science
• "Automatic molecular design using evolutionary
  techniques," Al Globus, John Lawton, and Todd
  Wipke, Nanotechnology, Volume 10, Number 3,
  September 1999, pp. 290-299.