Genetic Algorithm-Based Motion Synthesis

1
Genetic Algorithm-Based Motion Synthesis

• Soumya Ray and Sam Williams

2
Outline

• Overview of GA and GP methods
• What people have done with them
• Spacetime Constraints Problem and Genetic
  Algorithms
• Luxo Revisited with Genetic Programming
• Evolving Virtual Creatures
• General Critique
• The Future

3
The Problem

• Optimizing physically realistic motions according
  to task criteria is NP-hard
• Search space discontinuities and multimodality
  (many local near-optimal minimums) make gradient
  descent useless
• Luckily, genetic algorithms thrive in complex
  solution spaces, such as DNA or stimulus-response
  parameters

4
Basis

• Genetic algorithms and programming are based on
  Darwinian evolution
• survival of the fittest from generation to
  generation
• mutation and crossover allow adaption over time
• competition for limited resources mates,
  processors, space
• Mendelian heritability of traits using genetic
  material

5
Genetic Algorithms 101

• Generic Genetic Algorithm
• Generate initial population of parents
• Repeat
• 1) Simulate sex between fit parents, creating
  children by gene crossover, filling up the pool.
• 2) Mutate these children randomly. (eliminate
  duplicate genes and organisms along the way)
• 3) Calculate fitness of organisms
• 4) Sort population pool on fitness and keep only
  most worthy organisms as parents for the next
  generation

6
Parameters

• Method of generating initial population
• Size of gene pool
• Selection of parents kept across generations
• Mutation rate (radiation level)
• Type of mutation (bit, byte, change, gaussian)
• Fitness function used
• Termination criteria

7
Sawtooth Improvement

• As generations increase, large improvements will
  be seen infrequently
• In between these jumps, optimizations occur by
  way of searching out a local minimum
• Yields a sawtooth shaped progression graph

8
Genetic Programming 101

• GP is an attempt to automatically create programs
  which perform a target task
• GP uses the standard genetic algorithm on a
  structured program, rather than a list of
  variables
• the programs are represented as trees, such as
  those written in lisp
• Complexity is not fixed, allowing programs to
  gain and lose complexity to match the needs of
  the problem
• Of special interest are the methods for mutation
  and crossover

9
Probabalistic Genetic Programming
10
Crossover
11
Mutation
12
Woah

• Incredible possibilities, but GP is limited in
  practice to only a few functions and a small
  number of function nodes
• The vast majority of programs created by mutation
  and crossover are not viable
• The fitness function is extremely important for
  convergence

13
Genetic Programming for Articulated Figure Motion

• L. Gritz and J.K. Hahn

14
Genetically Programmed Joint Controllers

• Tradeoff between control and automation
• key framing vs controller programs
• Benefit of GP
• Animator only needs to supply a metric of what
  motion is good, the fitness function, and the
  state and effector variables which will be used
  in the program

15
GP Vs Spacetime Constraints

• Witkin and Kass high level goals through
  physically plausible motions, minimizing energy
  consumption
• Drawbacks
• requires significant hard-coding
• can be numerically unstable, NP hard,
  discontinuous
• solution dependent on initial guess provided to
  the optimizer, easily stuck in a local minimum

16
Requirements for a Genetic Program

• a parse tree and corresponding S-expression (one
  per joint controller in their case of an
  articulated figure)
• a fitness function
• termination criteria
• population size and maximum number of
  generations, and the other standard GA parameters

17
S Expression and its Tree Representation

• ( 3.14 ( y 8))
• controller expressions yield the desired joint
  angle given the state variables

18
Operators

• operators used by Gritz and Hahn
• (, -, , )
• An additional operator, ifltz, is included to
  handle branching. It takes three subtrees as
  arguments, returning the value of the second
  subtree if the value of the first subtree is less
  than zero, and otherwise returns the value of the
  third subtree

19
Initial Controller Trees

• Numerical constants are provided by an initial
  random sprinkling of floating point values in the
  trees, called ephemeral random constants
• terminals in the controller programs correspond
  to the internal state variables and outputs of
  any sensors accessible to the agent

20
Simulation of Controller
T0 while tlt time_limit do evaluate controller,
yielding desired joint angles a1an integrate
dynamics forward for time dt ttdt endwhile
21
Constructing Fitness Measures

• Fit main_goal style_points
• the main goal is always incorporated into the
  fitness function
• the style points are initially weighted at zero,
  but as the generation number increases, they
  become more heavily weighted
• this is to ensure that the goal is achieved
  before the motion is optimized

22
Style Points in the Fitness Function

• Penalties for hitting obstacles or violating
  safety rules (dont hit your head on the ground)
• Rewards for speed and efficiency
• Rewards for remaining in control
• Minimize energy consumption
• Subgoal completion

23
Luxo Revisited

• Remember Luxo, Jr?
• Completely key framed, exhausting labor
• 4 links, 3 internally controllable degrees of
  freedom
• Gritz and Hahns fitness function
• Main goal distance between base center and goal
  point
• Style points
• time
• penalized excess movement after goal
• penalized for hitting head or falling over
• bonus for ending with joints at neutral angles

24
Parameters

• 50 generations of 250 individuals
• 90 crossover, 10 reproduction
• no mutation (found to be insignificant)

25
Result A Hopping Luxo Lamp

• State variables a0, a1, a2, vx, vz, pz, s0, t
• With controller program

(list (- (ifltz a0 pz a2) ( a1 t))
(ifltz (- (- (ifltz a0 pz a2) ( a1 t)) (ifltz
( a2 15.4963) (ifltz a0 pz a2) (a1 a2)))
(ifltz ( (0 s0 vx) (a2 s0)) ( (- s0 vx) (
a2 s0)) ( (- a1 a1) (- a1 t))) ( ( a1 vz)
ifltz ( pz a1) ( a0 pz) ( px a0)))) (-
(ifltz s0 (ifltz (- ( vx a0) ( vx vx)) (- ( vx
a0) (- ( -28.43282 t) ( a1 t))) ( 16.5266
s0)) vz) (a1 vz)) )
26
Analysis

• The Luxo lamp jumps twice to reach the goal
• It has slight imperfections which give it
  organic, high frequency movement
• With the addition of an added style term, the
  Luxo lamp was taught to limbo
• Time required a couple of hours on an R4000

27
Deficiencies

• Controller programs are brittle, and are
  sensitive to initial conditions
• They are designed for a given task, not a general
  skill. Ie go to this point, not walk
• The method is very slow and has difficult
  convergence for figures with many degrees of
  freedom

28
Evolving Virtual Creatures

• Karl Sims

29
Growing Creatures

• Sims explored how to use genetic programming to
  alter both the morphology and neural systems of
  virtual creatures
• these creatures are evolved to swim, walk, jump,
  or move towards a light source, by simple
  modifications of the fitness function
• Creature segments, shapes, and joints are added,
  switched, and mutated

30
New considerations

• Structures and control programs evolve together.
  When a structure is copied, its control program
  comes with it
• There is a new relationship genotype to
  phenotype. Morphology is defined as a directed
  graph of components and connections, allowing
  repeated structures and controlers

31
Morphology Parameters

• Dimensions of segment (height, width, depth)
• Joint-Type
• rigid, revolute, twist, universal, bend-twist,
  twist-bend, or spherical
• Joint limits (restoring spring forces)
• Recursive limit (determines how many instances of
  a self-linked segment will appear in the
  phenotype)
• Set of connections to other nodes

32
Child Structure Placement

• Position, orientation, scale, and reflection
  parameters
• Reflection causes negative scaling, so similar
  but symmetrical subtrees can be described (ie
  right and left arms)

33
Creature Control

• The brain accepts input sensor values and
  provides output effector values in the form of
  torques at the degrees of freedom of the bodys
  joints
• Sensors
• Joint angle (like Luxo)
• Contact sensors (self contact and environmental)
• Photosensor normalized light source direction
  (for creatures learning light-follow)

34
Neurons

• Many Operators
• , /, , if (Gritz and Hahns)
• also added threshold, greater-than, sign-of,
  min, max, abs, interpolate, sin, cos, atan, log,
  expt, sigmoid, integrate, differentiate, smooth,
  memory, oscillate-wave, and oscillate-saw
• Inputs to the neurons are state variables and
  other neurons

35
Combining Morphology and Control

• Blocks of neural code are copied as determined by
  their genotype description, allowing copies and
  local control of joints
• A centralized collection of neurons, not
  associated with any given part are also created,
  for global synchronization and centralized
  control

36
Physical Simulation

• Connected parts are allowed to interpenetrate
  until their remote ends make contact, but
  otherwise, normal collision, friction, and
  articulated body dynamics are used
• An optional viscosity effect is added for
  underwater simulations

37
Fitness Measures Determine Task

• Swimming- point to point in zero gravity, viscous
  environment. Speed and strait swim as fitness
• Walking- point to point. Speed and balance as
  fitness
• Jumping- maximum height of lowest segment
• Following- photosensors enabled. Time for
  several trials with different point sources are
  summed for fitness.

38
Creature Evolution

• Initial population is random
• Survival ration of 1/5, population size 300
• For each generation, 1/5 most fit individuals
  reproduce and their mutated offspring fill the
  culled spots
• Probability of reproduction proportional to
  fitness

39
Mutation in directed graphs

• 1) internal parameters and operators of neurons
  may alter
• 2) a new random physical node may be added
• 3) physical connections may alter
• 4) new random connections may be added or removed
• 5) unconnected elements are garbage collected

40
Mating in Directed Graphs

• Grafts - two genotypes are aligned in the order
  they are stored, nodes past a certain point are
  swapped. Connections into and out of the swapped
  sections are severed
• Crossover operator- two successive graft-type
  operations
• Sims used 40 asexual, 30 crossover, 30 grafting

41
Further parameters

• Pop size 300
• 100 generations
• Due to varying complexities of creatures, the
  physical simulations varied widely in time
• Took about 3 hours on a CM-5 with 32 processors

42
Results

• Swimming paddling, tail wagging, and skulling
  developed. Made heavy use of sinusoidal
  functions.
• Walking shuffle, hobble, wag-and-rock, push and
  pull, inchworm, crawl, hopping most effective
• Jumping hammer toss, crunch, and spring
• Light-following steering fins and paddle angle
  strategies. Few creatures could consistently
  follow at different locations

43
Conclusions
44
Why Bother?

• Save animator time
• local control programs and stimulus-response
  allows complexity of movements
• solutions suggest novel and useful movement
  strategies
• results seem alive and plausible, very organic
  solutions

45
Problems

• control vs automation tradeoff
• stochastic- random chance is a factor in success
  or failure
• limited robustness
• sensitive to initial conditions
• does not produce generalized movements like walk
• time consuming (several hours on a CM5 to get
  from point A to B)

46
Extensions

• More difficult tasks
• Fights and social interactions
• Adjust the language to describe robots which
  could actually be built
• Additional shapes and materials
• Move from single-goal based creatures to jack-of
  all trades. Ex use many strategies to light
  follow, swim, walk, jump