Genetic Programming

1
Genetic Programming

• Reference Negnevitsky p 243 ff
• a recent development in evolutionary
  computation early 1990s, John Koza.
• the goal of a genetic algorithm is to evolve a
  fit bit representation of a problem whereas the
  goal of evolutionary programming is the evolve a
  fit representation of a solution to a problem (in
  the form of a computer program).
• LISP (list processing) is the language chosen
  for genetic programming.
• LISP programs can be represented as a tree
  which makes them easy to encode as a chromosome.

2
Genetic Programming

• LISP
• one of the oldest programming languages John
  McCarthy, late 1950s
• highly symbolic.
• it is an intepretative language.
• basic data structures are atoms and lists.
• atom --- smallest indivisible element e.g.
  symbol X, number 32 etc.
• list --- is an object that composed of atoms
  and/or other lists. Parentheses are used to order
  the lists and atoms.
• Both atoms and lists are referred to as
  symbolic expressions or s-expressions
• eg (- ( A B) C) is a s-expression. ( A B)
  is a list that calls the multiplication function
  with two arguments, A and B. (- ( A B) C) is a
  list that calls the subtraction function with two
  arguments , ( A B), C.

3
Genetic Programming

• s-expressions can represented as a tree

for example ( A B)
or A B
or (- ( A B) C)
i.e. (A B) - C

-
C

B
A
A
B
a function
Key
a terminal
4
Genetic Programming

• Genetic Programming a very simple example
• 1. determine set of terminals
• 2. select set of primitive functions
• 3. define fitness
• 4. specify parameters for the run
• 5. choose the method for designating a result for
  a run
• the example
• we will attempt to evolve a (LISP) program that
  works out pythagoras's theorem i.e.

5
Genetic Programming

• Remember we don't know the answer i.e. the
  theorem
• all we know are some sample answers (inputs gt
  output)
• e.g.
• 3.0, 4.0 gt 5.0
• 3.0, 5.0 gt 5.830952
• 8.0, 14.0 gt 16.124515
• 18.0, 2.0 gt 18.110770
• ltetcgt
• 1. Select a set of terminals (these will be the
  inputs to the program)
• a, b
• 2. Select the primitive functions (some choice
  here)
• e.g. sqrt, , -, , / (some basic arithmetic
  operators)

6
Genetic Programming

• 3. Define fitness
• fitness will be how well the program calculates
  the hypotenuse
• compare program results against known results
  (i.e fitness cases)
• a, b gt program outcome compared with expected
  outcome (shown below)
• 3.0, 4.0 gt 5.0
• 3.0, 5.0 gt 5.830952
• 8.0, 14.0 gt 16.124515
• 18.0, 2.0 gt 18.110770
• ltetcgt
• evaluate the sum of the absolute errors across
  the fitness cases, the closer the sum to zero the
  better.

7
Genetic Programming

• 4. GPs use the idea of population size, number of
  trails (or generations).
• 5. The result is the best program i.e. the one
  where fitness' is closest to 0.
• In genetic programming the chromosomes' i.e.
  LISP s-expressions can be of differing sizes.
• What does the chromosome look like?
• It is a program tree using the functions and
  terminals defined for the problem. The terminals
  are the leaves. The root node is a function
  (primitive)

8
Genetic Programming

• here is an example

s-expression
program tree
( / ( - sqrt ( ( a a)(- a b))) a )( a b))
/
bit string
-

sqrt
a
a
b

the function

-
a
a
a
b
9
Genetic Programming

• How might crossover work?
• any point in a tree can be choosen for
  cross-over a terminal or a function
• instead of exchanging bits (as in a genetic
  algorithm) sub-trees or terminals are exchanged

/
sqrt
-

-
sqrt
b
sqrt
a
a
b
/

-
a
b
a

-

( ( - ( sqrt ( - ( b b ) a ) ) b ) (sqrt ( /
a b )))
a
a
a
b
b
b
10
Genetic Programming

• How might crossover work?
• any point in a tree can be choosen for
  cross-over a terminal or a function
• instead of exchanging bits (as in a genetic
  algorithm) sub-trees or terminals are exchanged

/
sqrt
-

-
sqrt
b
sqrt
a
a
b
/

-
a
b
a

-

a
a
a
b
b
b
11
Genetic Programming

• How might crossover work?
• any point in a tree can be choosen for
  cross-over a terminal or a function
• instead of exchanging bits (as in a genetic
  algorithm) sub-trees or terminals are exchanged

/
sqrt
-
-
sqrt
b

sqrt
a
/
-

a
b
a
a
b


-
b
b
a
a
a
b
12
Genetic Programming

• How might mutation work?
• any function or terminal might mutate.
• a function will mutate to another function, a
  terminal may mutate to another terminal.
• remember for a given run a set of functions and a
  set of terminals have been defined.

sqrt
-

-
b
b


/
/
a
a
b
b
a
a
b
b
13
Genetic Programming

• Genetic Programming a summary from John Koza
• 1. assign max no. of generations to run and
  probabilities for cloning, crossover and mutation
  the total should be 1.
• 2. generate an initial population by combining
  randomly selected functions and terminals.
• 3. execute each program in the initial population
  and calculate fitness. Identify the best one.
• 4. select a genetic operator based on their
  assigned probabilities
• IF cloning is selected.
• select one program (individual) and copy it
  into the new population
• IF crossover is selected.
• select a pair of programs, perform crossover
  and place the offspring into the new population.
• IF mutation is selected.
• select one program and perform mutation, place
  the mutant in the new population.
• All selections are based on fitness.
• 5. Repeat step 4 until new population size is
  equal to old population size.
• 6. replace old population with new population
• 7. Go to step 3 and repeat the process until the
  termination criteria is reached
• most likely something based on the the best so
  far' program

14
Genetic Programming

• Genetic Programming --- final comments
• variable length chromosome' a limitation of
  the basic GA approach.
• useful if you can conceptualise the problem as
  a computer program'.
• extensive computing resources required for
  complex problems complexity is related to the
  number of functions and terminals that are
  specified and/or required.
• the choice of the fitness cases is crucial to
  success.

15
Genetic Programming and Genetic Algorithms

• Similarities
• problem' or 'program' are represented as a
  string of binary digits (bits).
• populations rather than individuals form the
  basis of the search.
• populations consisting of fitter' individuals
  evolve over time.
• operators such as crossover and mutation are
  applied.
• Differences
• Genetic programming evolves a solution' (as a
  function) to a problem eg a program which takes
  input(s) and delivers an output.
• Genetic algorithms evolve a fit representation'
  of a problem eg a timetable, a maintenance
  schedule.
• The chromosome in a GA is fixed in length.
• The chromosome in a GP can vary in size.
• In a GP fitness is evaluated using a series of
  fitness cases whereas in a GA it is calculated by
  applying an algorithm to the chromosome.