Genetic Algorithm and Genetic Programming

1
Genetic Algorithm and Genetic Programming
2
Crossovers in Nature

• Two parental chromosomes exchange part of their
  genetic information to create new hybrid
  combinations (recombinant).
• No loss of genes, but an exchange of genes
  between two previous chromosomes.
• No new genes created, preexisting old ones mixed
  together.

3
What are evolutionary algorithms and Genetic
Algorithm (GA)?
ROBOTICS
Artificial Intelligence
Machine learning
evolutionary
GP
EP
GA
ES
Artificial Intelligence
4
Genetic Algorithms
5
Genetic Algorithms

• 1. Randomly initialize a population of
  chromosomes.
• 2. While the terminating criteria have not been
  satisfied
• a. Evaluate the fitness of each chromosome
• i. Construct the phenotype (e.g. simulated robot)
  corresponding to the encoded genotype
  (chromosome).
• ii. Evaluate the phenotype (e.g. measure the
  simulated robots walking abilities), in order to
  determine its fitness.
• b. Remove chromosomes with low fitness.
• c. Generate new chromosomes, using certain
  selection schemes and genetic operators.

6
Genetic Algorithms

1. Fitness function
2. Selection scheme
3. Genetic operators
4. Crossover
5. Mutation
6. Create bitstrings
7. Evaluate bitstrings (genotype)
8. Transform bitstrings to robot behaviors
   (phenotype)

7

• Crossover and mutation.

8
Agenda

• What is Genetic Programming?
• Background/History.
• Why Genetic Programming?
• How Genetic Principles are Applied.
• Examples of Genetic Programs.
• Future of Genetic Programming.

9
Genetic Algorithms

• Most widely used
• Robust
• uses 2 separate spaces
• search space - coded solution (genotype)
• solution space - actual solutions (phenotypes)

Genotypes must be mapped to phenotypes before the
quality or fitness of each solution can be
evaluated
10
Evolutionary Strategies
11
Evolutionary Strategies

• Like GP no distinction between search and
  solution space
• Individuals are represented as real-valued
  vectors.
• Simple ES
• one parent and one child
• Child solution generated by randomly mutating the
  problem parameters of the parent.
• Susceptible to stagnation at local optima

12
Evolutionary Strategies (contd)

• Slow to converge to optimal solution
• More advanced ES
• have pools of parents and children
• Unlike GA and GP, ES have these properties
• ES Separates parent individuals from child
  individuals
• ES Selects its parent solutions deterministically

13
Evolutionary Programming
14
Evolutionary Programming

• Resembles ES, developed independently
• Early versions of EP applied to the evolution of
  transition table of finite state machines
• One population of solutions, reproduction is by
  mutation only
• Like ES operates on the decision variable of the
  problem directly (ie Genotype Phenotype)
• Tournament selection of parents
• better fitness more likely a parent
• children generated until population doubled in
  size
• everyone evaluated and the half of population
  with lowest fitness deleted.

15
General Architecture of Evolutionary Algorithms
16
Genetic Programming
17
Genetic Programming
Please review about Behavioral systems, Genetic
Algorithms and Genetic Programming from the
book. Chapters 20, 21, 22, 23.

• John Koza, 1992
• Evolve program instead of bitstring
• Lisp program structure is best suited
• Genetic operators can do simple replacements of
  sub-trees
• All generated programs can be treated as legal
  (no syntax errors)

18
Genetic Programming

• Specialized form of GA
• Manipulates a very specific type of solution
  using modified genetic operators
• Original application was to design computer
  programs
• Now applied in alternative areas eg. Analog
  Circuits
• Does not make distinction between search and
  solution space.
• Solution represented in very specific
  hierarchical manner.

19
Background/History

• By John R. Koza, Stanford University.
• 1992, Genetic Programming Treatise - Genetic
  Programming. On the Programming of Computers by
  Means of Natural Selection. - Origin of GP.
• Combining the idea of machine learning and
  evolved tree structures.

20
Why Genetic Programming?

• It saves time by freeing the human from having to
  design complex algorithms.
• Not only designing the algorithms but creating
  ones that give optimal solutions.
• Again, Artificial Intelligence.

21
What Constitutes a Genetic Program?

• Starts with "What needs to be done"
• Agent figures out "How to do it"
• Produces a computer program - Breeding Programs
• Fitness Test
• Code reuse
• Architecture Design - Hierarchies
• Produce results that are competitive with human
  produced results

22
How are Genetic Principles Applied?

• Breeding computer programs.
• Crossovers.
• Mutations.
• Fitness testing.

23
Computer Programs as Trees

• Infix/Postfix
• (2 a)(4 - num)

-

2
a
4
num
24
Breeding Computer Programs
Hmm hmm heh. Hey butthead. Do computer programs
actually score?
25
Breeding Computer Programs

• Start off with a large pool of random computer
  programs.
• Need a way of coming up with the best solution to
  the problem using the programs in the pool
• Based on the definition of the problem and
  criteria specified in the fitness test, mutations
  and crossovers are used to come up with new
  programs which will solve the problem.

26
The Fitness Test
27
The Fitness Test

• Identifying the way of evaluating how good a
  given computer program is at solving the problem
  at hand.
• How good can a program cope with its environment.
• Can be measured in many ways, i.e. error,
  distance, time, etc

28
Fitness Test Criteria

• Time complexity a good criteria.
• i.e. n2 vs. n logn.
• How long to find the solution?
• Accuracy - Values of variables.
• How good is the solution?
• Combinations of criteria may also be tested.

29
Mutations
30
Mutations in Nature
Properties of mutations

• Ultimate source of genetic variation.
• Radiation, chemicals change genetic information.
• Causes new genes to be created.
• One chromosome.
• Asexual.
• Very rare.

Before acgtactggctaa After
acatactggctaa
31
Genetic Programming

• 1. Randomly generate a combinatorial set of
  computer programs.
• 2. Perform the following steps iteratively until
  a termination criterion is satisfied
• a. Execute each program and assign a fitness
  value to each individual.
• b. Create a new population with the following
  steps
• i. Reproduction Copy the selected program
  unchanged to the new population.
• ii. Crossover Create a new program by
  recombining two selected programs at a random
  crossover point.
• iii. Mutation Create a new program by randomly
  changing a selected program.
• 3. The best sets of individuals are deemed the
  optimal solution upon termination

32
Mutations in Programs

• Single parental program is probabilistically
  selected from the population based on fitness.
• Mutation point randomly chosen.
• the subtree rooted at that point is deleted, and
• a new subtree is grown there using the same
  random growth process that was used to generate
  the initial population.
• Asexual operations (mutation) are typically
  performed sparingly
• with a low probability of mutations,
• probabilistically selected from the population
  based on fitness.

33
Crossovers in Programs
34
Crossovers in Programs

1. Two parental programs are selected from the
   population based on fitness.
2. A crossover point is randomly chosen in the first
   and second parent.
3. The first parent is called receiving
4. The second parent is called contributing
5. The subtree rooted at the crossover point of the
   first parent is deleted
6. It is replaced by the subtree from the second
   parent.
7. Crossover is the predominant operation in genetic
   programming (and genetic algorithm) research
8. It is performed with a high probability (say, 85
   to 90).

35
Applications of GP in robotics

• Wall-following robot Koza
• Behaviors of subsumption architecture of Brooks.
  Evolved a new behavior.
• Box-moving robot Mahadevan
• Evolving behavior primitives and arbitrators
• for subsumption architecture
• Motion planning for hexapod Fukuda, Hoshino,
  Levy PSU.
• Evolving communication agents Iba, Ueda.
• Mobile robot motion control Walker.
• for object tracking
• Soccer
• Car racing

Population sizes from 100 to 2000
36
Subset of LISP for Genetic Programming
37
This is a very very small subset of Lisp
LISP Famous first language of Artificial
Intelligence and Robotics
38

• More functions
• Atom, list, cons, car, cdr, numberp, arithmetic,
  relations. Cond.
• Copying example

39
A very important concept Lisp does not
distinguish data and programs
40
Programs in Lisp are trees that are evaluated
(calculated)
Tree-reduction semantics
41
Lisp allows to define special languages in itself
42
(No Transcript)
43
Robot-Lisp cont

• This subset of Lisp was defined for a mobile
  robot
• You can define similar subsets for a humanoid
  robot, robot hand, robot head or robot theatre

44
As an exercise, you can think about behaviors of
all Braitenberg Vehicles described by such
programs
45

• You can define much more sophisticated mutations
  that are based on constraints and in general on
  your knowledge and good guesses (heuristics)

46

1. You can define your own languages in Lisp
2. You can write your own Lisp-like language
   interpreter in C
3. Many on Web

47
EyeSim simulator allows to simulate robots,
environments and learning strategies together,
with no real robot
In our project with teens the feedback is from
humans who evaluate the robot theatre performance
(subjective)
In our previous project with hexapod the feedback
was from real measurement in environment
(objective)
Evolution here takes feedback from simulated
environment (simulated)
48
Evolution principles are the same for all
evolutionary algorithms.

• Each individual (Lisp program) is executed on the
  EyeSim simulator for a limited number of program
  steps.
• The program performance is rated continually or
  when finished.

49
Sample Problem Ball Tracking
50
Sample Problem Ball Tracking
GOALS

• Find ball in environment
• Drive towards it
• Stop when close to ball

1. A single robot is placed at a random position and
   orientation in a rectangular driving area closed
   by walls.
2. A colored ball is also placed at a random
   position.
3. Using its camera, the robot has to detect the
   ball, drive towards it, and stop close to it.
4. The robot camera is positioned at an angle so the
   robot can see the wall ahead from any position in
   the field.
5. However, the ball is not always visible.

• Similar other tasks that we solved
• Collect cans
• Shoot goal in soccer
• Robot sumo
• Robot fencing

51
Idea for solving

• In the loop, grab an image and analyze it as
  follows
• Convert the image from RGB to HSV (Hue,
  Saturation and Value)
• Use the histogram ball detection routine from
  section 8.6 of the book (this returns a ball
  position in the range 0..79 (left.. Right) or
  no ball and a ball size in pixels 0..60)
• If the ball height is 20 pixels or more, then
  stop and terminate (the robot is close enough to
  the ball)
• Otherwise
• If no ball is detected or the ball position is
  less than 20, turn slowly left.
• If the ball position is between 20 and 40, drive
  slowly straight
• If the ball position is greater than 40, turn
  slowly right.

52

• ColSearch returns the x-position of the ball or
  -1 if not detected and the ball height in pixels.
• The statement VWDriveWait following either
  VWDriveTurn or a VWDriveStraight command suspends
  execution driving or rotation of the requested
  distance or angle has finished.

53
The evolution can start from random data or from
good hand-coded solutions from humans
Bear in mind that obj_size and obj_pose are in
fact calls to the image processing subroutine.
54
Evolution of tracking behavior

• Koza suggests the following steps for setting up
  a genetic programming system
• Establish an objective.
• Identify the terminals and functions used in the
  inductive program
• Establish the selection scheme and its
  evolutionary operations
• Finalize the number of fitness cases
• Determine the fitness function and hence the
  range of raw fitness values
• Establish the generation gap G, and the
  population M
• Finalize all control parameters.
• Execute the Genetic Paradigm.

55
EyeSim simulator allows to simulate robots,
environments and learning strategies together,
with no real robot
In our project with teens the feedback is from
humans who evaluate the robot theatre performance
(subjective)
In our previous project with hexapod the feedback
was from real measurement in environment
(objective)
Evolution here takes feedback from simulated
environment (simulated)
56
Summary on GP

• Field of study in Machine Learning.
• Created by John Koza in 1992.
• Save time while creating better programs.
• Based on the principles of genetics.
• Used in
• Symbolic Regression
• Circuit Design.
• Many others

57

• Example of using Genetic Programming in Robotics
• Use of simulation

58
Robots view and driving path in EyeSim simulator
Robot trajectory
59
You want to maximize this fitness function
60
(No Transcript)
61
(No Transcript)
62
Fitness grows in next populations (in general)
Highest fitness
63
What are the evolved trajectories of the robot to
meet the ball?
The best trajectory
64

• Use of improved GP
• Use of parallel processing
• Use of FPGA and evolvable hardware
• Use of quantum computers in future.

65
Behavior-Based Systems
66
Behavior-Based Systems

• Rodney Brooks, 1986
• Ronald Arkin, 1998
• Instead of single, monolithic program, use a
  number of behaviors ( parallel processes)
• Improve changeability
• Utililize emergence
• Each behavior has access to all sensors and
  actuators
• Problem behavior selection

There are now many variants of behavior-based
robots. Braitenberg is just one of them.
67
Good old-fashioned hierarchical software
architecture for a robot
68
Modern behavioral software architecture for a
robot
69
Behavior Selection

• Clearly, we cannot have two different behaviors
  in control of actuators
• (e.g. obstacle-avoidance wants to drive left,
  ball-detection wants to drive right)
• Solution 1 Always select one behavior as active
• Solution 2 Add actuator output of all behaviors,
  multiplied with certain coefficients

70
Behavior Selection

• Remaining problem
• How to determine which behavior to select or
  which coefficients to use
• Solution ??
• How about using GA to evolve a solution!

71
Behavior Selection
Parameters
Evolution of controller parameters
Direct inputs
camera
Ball detection
Processed sensor input
Schema weighting
Move to ball
Avoid obstacles
Schemas using direct and processed inputs
schemas
Trial result
Actuator output
Single chromosome
72

• Many tools of this type are created
• In this class the projects are about animatronix
  editor and state based editor
• Another possibility is to use Artificial Neural
  Network

73

• This slide shows hierarchical editor for
  subsumption-type architecture

74
Sample Problem Ball Tracking

• Problem
• Find ball in environment
• Drive towards it
• Stop when close to ball
• Behavior selection
• Use NN
• Evolve NN weights using GA

75
Behavior Selection with NN and GA
76
Behavior Selection with NN and GA

• Fitness function
• fitness initDist - b_distance()
• if (fitness lt 0.0) fitness 0.0

77
(No Transcript)
78
(No Transcript)
79
Both approaches can provide increase in Fitness
function. The problem is how good?
80
Trajectories found by behavioral approaches
81
(No Transcript)
82
AI in robotics (first summary)

• Neural Networks
• Learning requires complete test set with correct
  results
• Genetic Algorithms
• Evolving solutions based on fitness function can
  be used in real robot or in simulation (often
  easier/faster)
• Genetic Programming
• Requires algorithm coding/interpretation (e.g.
  Lisp)
• Behavior-Based Systems
• Combining parallel processing approach with
  intelligent selection scheme

83
Other areas - Examples of Genetic Programs

• 1. Symbolic Regression -
• the process of discovering
• the functional form of a target function
• and all of its necessary coefficients,
• or at least an approximation to these.
• 2. Analog circuit design
• Embryo circuit is an initial circuit which is
  modified to create a new circuit according to
  functionality criteria.

84
Genetic Programming in the Future

• Speculative.
• Only been around for 10 years.
• Is very successful.
• Discovery of new algorithms in existing projects.
• Can be combined with constructive induction,
  LISP-based systems, evolutionary hardware, neural
  networks, fuzzy logic, quantum logic and many
  other ideas and methods.

Mr. Roboto
85
End of Show
Hey Butthead. That kicked ass.
Oh yeah. Hm hm yeah yeah hm. It sucked.
Shut up Buttmunch. That sucked.
86
What you can do with these evolutionary ideas?

1. Generation of gaits for a biped robot
2. Generation of hand gestures for a humanoid robot
3. Generation of path for a mobile robot
4. Generation of mouth motions for a speaking robot
   head
5. Generation of a state machine for a behavioral
   robot Braitenberg Vehicles.
6. Generation of arbitrary controller represented as
   Neural Net, Boolean Function, Fuzzy function or
   Multiple-Valued Function.
7. Many many other applications in our projects be
   creative.
8. You can find many papers by Google search under
   Evolutionary Robotics.

87
Sources

• Braunl
• Dan Kiely
• Ran Shoham
• Brent Heigold
• CPSC 533, Artificial Intelligence