Genetic Programming

1
Genetic Programming
2
Genetic Algorithms

• 1. Randomly initialize a population of
  chromosomes.
• 2. While the terminating criteria have not been
  satisfied
• a. Evaluate the fitness of each chromosome
• i. Construct the phenotype (e.g. simulated robot)
  corresponding to the encoded genotype
  (chromosome).
• ii. Evaluate the phenotype (e.g. measure the
  simulated robots walking abilities), in order to
  determine its fitness.
• b. Remove chromosomes with low fitness.
• c. Generate new chromosomes, using certain
  selection schemes and genetic operators.

3
Genetic Algorithms

• Fitness function
• Selection scheme
• Genetic operators
• Crossover
• Mutation

4

• Crossover and mutation.

5
Agenda

• What is Genetic Programming?
• Background/History.
• Why Genetic Programming?
• How Genetic Principles are Applied.
• Examples of Genetic Programs.
• Future of Genetic Programming.

6
Genetic Programming
Please review about Behavioral systems, Genetic
Algorithms and Genetic Programming from the
book. Chapters 20, 21, 22, 23.

• John Koza, 1992
• Evolve program instead of bitstring
• Lisp program structure is best suited
• Genetic operators can do simple replacements of
  sub-trees
• All generated programs can be treated as legal
  (no syntax errors)

7
What is Genetic Programming(GP)?
ROBOTICS
Artificial Intelligence
Machine learning
evolutionary
GP
EP
GA
ES
Artificial Intelligence
8
Genetic Algorithms

• Most widely used
• Robust
• uses 2 separate spaces
• search space - coded solution (genotype)
• solution space - actual solutions (phenotypes)

Genotypes must be mapped to phenotypes before the
quality or fitness of each solution can be
evaluated
9
Evolutionary Strategies

• Like GP no distinction between search and
  solution space
• Individuals are represented as real-valued
  vectors.
• Simple ES
• one parent and one child
• Child solution generated by randomly mutating the
  problem parameters of the parent.
• Susceptible to stagnation at local optima

10
Evolutionary Strategies (contd)

• Slow to converge to optimal solution
• More advanced ES
• have pools of parents and children
• Unlike GA and GP, ES
• Separates parent individuals from child
  individuals
• Selects its parent solutions deterministically

11
Evolutionary Programming

• Resembles ES, developed independently
• Early versions of EP applied to the evolution of
  transition table of finite state machines
• One population of solutions, reproduction is by
  mutation only
• Like ES operates on the decision variable of the
  problem directly (ie Genotype Phenotype)
• Tournament selection of parents
• better fitness more likely a parent
• children generated until population doubled in
  size
• everyone evaluated and the half of population
  with lowest fitness deleted.

12
General Architecture of Evolutionary Algorithms
13
Genetic Programming

• Specialized form of GA
• Manipulates a very specific type of solution
  using modified genetic operators
• Original application was to design computer
  program
• Now applied in alternative areas eg. Analog
  Circuits
• Does not make distinction between search and
  solution space.
• Solution represented in very specific
  hierarchical manner.

14
Background/History

• By John R. Koza, Stanford University.
• 1992, Genetic Programming Treatise - Genetic
  Programming. On the Programming of Computers by
  Means of Natural Selection. - Origin of GP.
• Combining the idea of machine learning and
  evolved tree structures.

15
Why Genetic Programming?

• It saves time by freeing the human from having to
  design complex algorithms.
• Not only designing the algorithms but creating
  ones that give optimal solutions.
• Again, Artificial Intelligence.

16
What Constitutes a Genetic Program?

• Starts with "What needs to be done"
• Agent figures out "How to do it"
• Produces a computer program - Breeding Programs
• Fitness Test
• Code reuse
• Architecture Design - Hierarchies
• Produce results that are competitive with human
  produced results

17
How are Genetic Principles Applied?

• Breeding computer programs.
• Crossovers.
• Mutations.
• Fitness testing.

18
Computer Programs as Trees

• Infix/Postfix
• (2 a)(4 - num)

-

2
a
4
num
19
Breeding Computer Programs
Hmm hmm heh. Hey butthead. Do computer programs
actually score?
20
Breeding Computer Programs

• Start off with a large pool of random computer
  programs.
• Need a way of coming up with the best solution to
  the problem using the programs in the pool
• Based on the definition of the problem and
  criteria specified in the fitness test, mutations
  and crossovers are used to come up with new
  programs which will solve the problem.

21
The Fitness Test

• Identifying the way of evaluating how good a
  given computer program is at solving the problem
  at hand.
• How good can a program cope with its environment.
• Can be measured in many ways, i.e. error,
  distance, time, etc

22
Fitness Test Criteria

• Time complexity a good criteria.
• i.e. n2 vs. nlogn.
• Accuracy - Values of variables.
• Combinations of criteria may also be tested.

23
Mutations in Nature
Properties of mutations

• Ultimate source of genetic variation.
• Radiation, chemicals change genetic information.
• Causes new genes to be created.
• One chromosome.
• Asexual.
• Very rare.

Before acgtactggctaa After
acatactggctaa
24
Genetic Programming

• 1. Randomly generate a combinatorial set of
  computer programs.
• 2. Perform the following steps iteratively until
  a termination criterion is satisfied
• a. Execute each program and assign a fitness
  value to each individual.
• b. Create a new population with the following
  steps
• i. Reproduction Copy the selected program
  unchanged to the new population.
• ii. Crossover Create a new program by
  recombining two selected programs at a random
  crossover point.
• iii. Mutation Create a new program by randomly
  changing a selected program.
• 3. The best sets of individuals are deemed the
  optimal solution upon termination

25
Mutations in Programs

• Single parental program is probabilistically
  selected from the population based on fitness.
• Mutation point randomly chosen.
• the subtree rooted at that point is deleted, and
• a new subtree is grown there using the same
  random growth process that was used to generate
  the initial population.
• Asexual operations (mutation) are typically
  performed sparingly
• with a low probability of,
• probabilistically selected from the population
  based on fitness.

26
Crossovers in Nature

• Two parental chromosomes exchange part of their
  genetic information to create new hybrid
  combinations (recombinant).
• No loss of genes, but an exchange of genes
  between two previous chromosomes.
• No new genes created, preexisting old ones mixed
  together.

27
Crossovers in Programs

• Two parental programs are selected from the
  population based on fitness.
• A crossover point is randomly chosen in the first
  and second parent.
• The first parent is called receiving
• The second parent is called contributing
• The subtree rooted at the crossover point of the
  first parent is deleted
• It is replaced by the subtree from the second
  parent.
• Crossover is the predominant operation in genetic
  programming (and genetic algorithm) research
• It is performed with a high probability (say, 85
  to 90).

28
Applications of GP in robotics

• Wall-following robot Koza
• Behaviors of subsumption architecture of Brooks.
  Evolved a new behavior.
• Box-moving robot Mahadevon
• Evolving behavior primitives and arbitrators
• for subsumption architecture
• Motion planning for hexapod Fukuda, Hoshino,
  Levy PSU.
• Evolving communication agents Iba, Ueda.
• Mobile robot motion control Walker.
• for object tracking
• Soccer
• Car racing

Population sizes from 100 to 2000
29
This is a very very small subset of Lisp
30

• More functions
• Atom, list, cons, car, cdr, numberp, arithmetic,
  relations. Cond.
• Copying example

31
A very important concept Lisp does not
distinguish data and programs
32
Programs in Lisp are trees that are evaluated
(calculated)
Tree-reduction semantics
33
Lisp allows to define special languages in itself
34
(No Transcript)
35

• This subset of Lisp was defined for a mobile
  robot
• You can define similar subsets for a humanoid
  robot, robot hand, robot head or robot theatre

36
As an exercise, you can think about behaviors of
all Braitenberg Vehicles described by such
programs
37

• You can define much more sophisticated mutations
  that are based on constraints and in general on
  your knowledge and good guesses (heuristics)

38
(No Transcript)
39
EyeSim simulator allows to simulate robots,
environments and learning strategies together
with no real robot
In our project with teens the feedback is from
humans who evaluate the robot theatre performance
(subjective)
In our previous project with hexapod the feedback
was from real measurement in environment
(objective)
Evolution here takes feedback from simulated
environment (simulated)
40
Evolution principles are the same for all
evolutionary algorithms.

• Each individual (Lisp program) is executed on the
  EyeSim simulator for a limited number of program
  steps.
• The program performance is rated continually or
  when finished.

41
Sample Problem Ball Tracking
A single robot is placed at a random position and
orientation in a rectangular driving area closed
by walls. A colored ball is also placed at a
random position. Using its camera, the robot has
to detect the ball, drive towards it, and stop
close to it. The robot camera is positioned at an
angle so the robot can see the wall ahead from
any position in the field. However, the ball is
not always visible.

• Find ball in environment
• Drive towards it
• Stop when close to ball

• Similar other tasks that we solved
• Collect cans
• Shoot goal in soccer
• Robot sumo
• Robot fencing

42
Idea for solving

• In the loop, grab an image and analyze it as
  follows
• Convert the image from RGB to HSV.
• Use the histogram ball detection routine from
  section 8.6 of the book (this returns a ball
  position in the range 0..79 (left.. Right) or
  no ball and a ball size in pixels 0..60)
• If the ball height is 20 pixels or more, then
  stop and terminate (the robot is close enough to
  the ball)
• Otherwise
• If no ball is detected or the ball position is
  less than 20, turn slowly left.
• If the ball position is between 20 and 40, drive
  slowly straight
• If the ball position is greater than 40, turn
  slowly right.

43

• ColSearch returns the x-position of the ball or
  -1 if not detected and the ball height in pixels.
• The statement VWDriveWait following either
  VWDriveTurn or a VWDriveStraight command suspends
  execution driving or rotation of the requested
  distance or angle has finished.

44
The evolution can start from random data or from
good hand-coded solutions from humans
Bear in mind that obj_size and obj_pose are in
fact calls to the image processing subroutine.
45
Evolution of tracking behavior

• Koza suggests the following steps for setting up
  a genetic programming system
• 1. Establish an objective.
• 2. Identify the terminals and functions used in
  the inductive programs
• 3. Establish the selection scheme and its
  evolutionary operations
• 4. Finalize the number of fitness cases
• 5. Determine the fitness function and hence the
  range of raw fitness values
• 6. Establish the generation gap G, and the
  population M
• 7. Finalize all control parameters.
• 8. Execute the Genetic Paradigm.

46
EyeSim simulator allows to simulate robots,
environments and learning strategies together
with no real robot
In our project with teens the feedback is from
humans who evaluate the robot theatre performance
(subjective)
In our previous project with hexapod the feedback
was from real measurement in environment
(objective)
Evolution here takes feedback from simulated
environment (simulated)
47
Summary on GP

• Field of study in Machine Learning.
• Created by John Koza in 1992.
• Save time while creating better programs.
• Based on the principles of genetics.
• Symbolic Regression/Circuit Design.

48

• Example of using Genetic Programming in Robotics
• Use of simulation

49
Robots view and driving path in EyeSim simulator
Robot trajectory
50
You want to maximize this fitness function
51
(No Transcript)
52
(No Transcript)
53
Fitness grows in next populations (in general)
Highest fitness
54
What are the evolved trajectories of the robot to
meet the ball?
The best trajectory
55

• Use of improved GP
• Use of parallel processing
• Use of FPGA and evolvable hardware
• Use of quantum computers in future.

56
Behavior-Based Systems

• Rodney Brooks, 1986
• Ronald Arkin, 1998
• Instead of single, monolithic program, use a
  number of behaviors ( parallel processes)
• Improve changeability
• Utililize emergence
• Each behavior has access to all sensors and
  actuators
• Problem behavior selection

There are now many variants of behavior-based
robots. Braitenberg is just one of them.
57
Good old-fashioned hierarchical software
architecture for a robot
58
Modern behavioral software architecture for a
robot
59
Behavior Selection

• Clearly, we cannot have two different behaviors
  in control of actuators
• (e.g. obstacle-avoidance wants to drive left,
  ball-detection wants to drive right)
• Solution 1 Always select one behavior as active
• Solution 2 Add actuator output of all behaviors,
  multiplied with certain coefficients

60
Behavior Selection

• Remaining problem
• How to determine which behavior to select or
  which coefficients to use
• Solution ??
• How about using GA to evolve a solution!

61
Behavior Selection
Parameters
Evolution of controller parameters
Direct inputs
camera
Ball detection
Processed sensor input
Schema weighting
Move to ball
Avoid obstacles
Schemas using direct and processed inputs
schemas
Trial result
Actuator output
Single chromosome
62

• Many tools of this type are created
• In this class the projects are about animatronix
  editor and state based editor
• Another possibility is to use Artificial Neural
  Network

63

• This slide shows hierarchical editor for
  subsumption-type architecture

64
Sample Problem Ball Tracking

• Problem
• Find ball in environment
• Drive towards it
• Stop when close to ball
• Behavior selection
• Use NN
• Evolve NN weights using GA

65
Behavior Selection with NN and GA

• Fitness function
• fitness initDist - b_distance()
• if (fitness lt 0.0) fitness 0.0

66
(No Transcript)
67
(No Transcript)
68
Both approaches can provide increase in Fitness
function. The problem is how good?
69
Trajectories found by behavioral approaches
70
(No Transcript)
71
AI in robotics (first summary)

• Neural Networks
• Learning requires complete test set with correct
  results
• Genetic Algorithms
• Evolving solutions based on fitness function can
  be used in real robot or in simulation (often
  easier/faster)
• Genetic Programming
• Requires algorithm coding/interpretation (e.g.
  Lisp)
• Behavior-Based Systems
• Combining parallel processing approach with
  intelligent selection scheme

72
Other areas - Examples of Genetic Programs

• 1. Symbolic Regression -
• the process of discovering
• the functional form of a target function
• and all of its necessary coefficients,
• or at least an approximation to these.
• 2. Analog circuit design
• Embryo circuit is an initial circuit which is
  modified to create a new circuit according to
  functionality criteria.

73
Genetic Programming in the Future

• Speculative.
• Only been around for 10 years.
• Is very successful.
• Discovery of new algorithms in existing projects.
• Can be combined with constructive induction,
  LISP-based systems, evolutionary hardware, neural
  networks, fuzzy logic, quantum logic and many
  other ideas and methods.

Mr. Roboto
74
End of Show
Hey Butthead. That kicked ass.
Oh yeah. Hm hm yeah yeah hm. It sucked.
Shut up Buttmunch. That sucked.
75
Sources

• Braunl
• Dan Kiely
• Ran Shoham
• Brent Heigold
• CPSC 533, Artificial Intelligence