Artificial Intelligence IV

1
Artificial Intelligence IV

• 2006

2
Genetic Algorithms

• More Advanced Concepts a bit of a challenge ?

3
Interesting Facts

• The Bots in Quake3 were evolved using GAs
• GAs were used to optimize a fuzzy controller,
  whose job is to decide for the Bots what to do
  next, i.e. 86 pick up better gun, 53 pick up
  armor
• NASA often uses GAs
• Calculate low altitude satellite orbits
• Calculate the positioning of the Hubble telescope

4
The Traveling Salesman Problem

• Given a collection of cities, the traveling
  salesman must determine the shortest route that
  will enable him to visit each city precisely once
  and then return him to his starting point.

5
TSP
6
TSP

• Class NP-Complete
• As more cities are added the processing power to
  solve this increases exponentially
• To find the optimum solution you need to test all
  permutations
• i.e.
• to find the optimum for 4 cities ? testing 4 x 3
  x 2 x 1 24 permutations
• 10 cities ? 3 628 800 permutations
• 15 ? 1 307 674 368 000 permutations
• 100 ? ridiculously huge amounts of work!

7
TSP

• This TYPE of problem frequently occurs when
  coding AI for strategy games, etc.
• Many logistical problems are similar
• This is a great problem to tackle with GAs
  (start with about 20 cities)
• Improving algorithms for more and more cities can
  also be an additive exercise

8
TSP

• Although the problem arises in transportation
  applications, its most important applications
  arise in optimizing the tool paths for
  manufacturing equipment.   
• For example, consider a robot arm assigned to
  solder all the connections on a printed circuit
  board. The shortest tour that visits each solder
  point exactly once defines the most efficient
  path for the robot.
• A similar application arises in minimizing the
  amount of time taken by a graphics plotter to
  draw a given figure.

9
TSP

• The world-record-setting traveling salesman
  program is by Applegate, Bixby, Chvatal, and Cook
  D. Applegate, R. Bixby, V. Chvatal, and W. Cook.
  Finding cuts in the TSP (a preliminary report).
  Technical Report 95-05, DIMACS, Rutgers
  University, Piscataway NJ, 1995. , which has
  solved instances as large as 7,397 vertices to
  optimality.  
• At this time, the program is not being
  distributed. However, the authors seem willing to
  use it to solve TSPs sent to them.
• In their paper, they describe this work as
  neither theory nor practice, but sport - an
  almost recreational endeavor designed principally
  to break records. It is a very impressive piece
  of work, however.

10
THINK

• How would YOU tackle this using a GA?
• What would be different?

11
TSP So whats different?

• All solutions MUST be a viable permutation! (A
  tour of all cities)
• A tour of 8 cities can be encoded using 3 bits
  per city i.e.
• 3,4,,0,7,2,5,1,6 011 100 000 111 010 101 001
  110
• You cant randomly generate a starting population
• 3,1,2,3,1,5,6,7 Would not be valid for 8 cities
• Crossover would have to change
• 3,4,0,7,2,5,1,6 (Valid parent)
• 2,5,0,3,6,1,4,7 (Valid parent)
• 3,4,0,7,6,1,4,7 (INVALID child)
• Mutation would make a VALID child invalid by
  default!

12
TSP So whats different?

• In this case using binary strings will simply
  make coding more difficult, so use an array of
  integers
• Well need a different type of cross-over (MUST
  produce only valid children)
• Well need a different type of mutation

13
Crossover

• Many types of crossover can handle permutations
• Partially-Mapped Crossover
• Order Crossover
• Alternating-Position Crossover
• Maximal-Preservation Crossover
• Position-Based Crossover
• Edge-Recombination Crossover
• Subtour-Chunks Crossover
• Intersection Crossover

14
Partially Mapped Crossover (PMX)

• P1 2,5,0,3,6,1,4,7
• P2 3,4,0,7,2,5,1,6
• Choose 2 random crossover points (i.e. after 3
  cities and after 6)
• Now look at mapping between parents for the
  centre section
• 3 maps to 7
• 6 maps to 2
• 1 maps to 5
• Create new copies of chromosomes (Children)

15
Partially Mapped Crossover (PMX)

• C1 2,5,0,3,6,1,4,7 (Initially identical to P1)
• C2 3,4,0,7,2,5,1,6
• Loop through children and swap genes listed in
  mapping
• Swap for 3 and 7
• C1 2,5,0,7,6,1,4,3
• C2 7,4,0,3,2,5,1,6
• Swap for 6 and 2
• C1 6,5,0,7,2,1,4,3
• C2 7,4,0,3,6,5,1,2
• Swap for 1 and 5
• C1 2,1,0,7,6,5,4,3
• C2 7,4,0,3,2,1,5,6

16
How do you mutate?

• Swap any two genes!
• Choose two random positions
• i.e. position 2 and 7 and swap entries
• 7,4,0,3,2,1,5,6
• Becomes
• 7,5,0,3,2,1,4,6

17
Fitness

• We need a fitness function that gives shorter
  routes a higher fitness
• Also important We must have a reasonable spread
  of fitness values between the best and worst case
• Thus simply using the actual tour length is a
  bad idea. i.e. (tour length)/(pop total tour
  length)
• Better
• Keep track of the worst length,
• For each genome calculate difference with this
  worst
• NOW use normal roulette wheel with ELITSM
• (worst tour length-current tour length)/(total
  pop tour length)

18
Internal representation (MAP) For playing around
19
TSP

• The solution thus far presented actually get
  stuck a LOT!
• The key to GAs (and NNs) is too develop a feel
  for them
• This feel can be developed by playing with
  different operators

20
Alternative Mutation Operators

• Scramble Mutation (SM)
• Choose two random points and scramble entries
  between them
• 0,1,2,3,4,5,6,7 becomes 0,1,2,5,6,3,4,7
• Displacement Mutation (DM)
• Select two random points
• Grab the chunk between them and MOVE it somewhere
  else
• 0,1,2,3,4,5,6,7 becomes 0,3,4,5,1,2,6,7

21
Alternative Mutation Operators

• Insertion Mutation (IM)
• Same as DM, but only one gene gets moved
• 0,1,2,3,4,5,6,7 becomes 0,1,2,4,5,3,6,7
• Inversion Mutation (IVM)
• Select two random points
• Reverse the entries between them
• 0,1,2,3,4,5,6,7 becomes 0,1,2,6,5,4,3,7
• Displaced Inversion Mutation (DIVM)
• Select two points
• Invert then Displace
• 0,1,2,3,4,5,6,7 becomes 0, 6,5,4,3,1,2,7

22
Alternative Permutation Crossover Operators

• Order Based Crossover (OBX)
• Randomly choose nodes (cities) from one parent
• Impose the order of those nodes (cities) on the
  other parent
• P1 2,5,0,3,6,1,4,7
• P2 3,4,0,7,2,5,1,6
• C1 3,4,5,7,2,0,1,6
• C2 Choose nodes in P2 and impose on P1

23
Alternative Permutation Crossover Operators

• Position-Based Crossover (PBX)
• Similar to OBX, instead of order, the position is
  imposed
• P1 2,5,0,3,6,1,4,7
• P2 3,4,0,7,2,5,1,6
• Step 1 Copy order of P1 across
• C1 ,5,0,,,1,,
• Step 2 Fill the rest in sequence from P2
• C1 3,5,0,4,7,1,2,6
• REPEAT for C2 using P2 for order

24
Selection Techniques

• Elitism
• Make X copies of the current generations fittest
  genome(s) into next generation
• Steady State Selection
• Extreme elitism ?
• Keep entire generation, except for the worst X
  genomes

25
Selection Techniques

• Fitness Proportionate Selection
• Roulette wheel selection
• (Bad for small populations)
• Stochastic Universal Sampling
• Instead of spinning the roulette wheel N times,
  spin it once but select using N evenly spaced
  pointers

26
Selection Techniques

• Tournament Selection
• Choose n individuals at random (nltpopulation
  size)
• Take the fittest of those selected and add to the
  new generation
• DO NOT remove selected ones
• Repeat until full generation

27
Scaling

• Raw (un-scaled) fitness scores CAN give a useable
  value BUT in general GA performance can be
  improved by scaling fitness to avoid outliers
• Rank Scaling
• Rank genomes, the rank is the new fitness
• I.e. if population size 5 the fittest becomes
  fitness 5, second becomes fitness 4, etc
• This helps prevent premature convergence (on i.e.
  a local minima)

28
Scaling

• Sigma Scaling
• Raw fitness often converge too fast, rank fitness
  often converge too slow
• Sigma scaling tries too smooth things out (keeps
  selection pressure steady)
• If stddev 0 then newf 1 else newf
  (oldf-avg(fit))/2stddev
• Avg(fit) å Fit(x)/popsize
• Stddev ÖVAR
• VAR (å (Fit(x) avg(fit))2) / popsize

29
Scaling

• Boltzmann Scaling
• Sometimes you might want to vary selection
  pressure
• i.e. Start with low pressure to maintain
  diversity in population BUT increase it every
  generation so that fitter individuals have a
  higher change towards the end
• Newfitness (oldF / temperature) /
  (avg(fit) / temperature)
• Temperature is a value that start fairly high and
  get decreased gradually each generation

30
Niching

• Niching is a way of preserving population
  diversity by grouping similar individuals
  together
• Explicit Fitness Sharing
• Group members with similar genomes together
• (i.e. two genomes may have an exact overlap in 5
  places ? compatibility score 5)
• Test each genome against some sample and then
  group
• Share the fitness between similar genomes
• If three are grouped together, each has its
  fitness divided by three
• This penalize individuals if there are too many
  similar ones, THUS increasing diversity
• Other niching techniques do exist

31
So What?

• The only way to truly develop a feel for
  Genetic Algorithms (and Neural Networks) is to
  PLAY with them
• The techniques outlined earlier should be
  experimented with
• Some of them will make your pathfinder
  practical better (Its not about finding the exit,
  its about finding it in better ways!)
• TSP ? Is a VERY nice (and well studied)
  optimization problem. If you want to get really
  good at GAs, its an excellent testing ground

32
Next Practical

• Write a GA to solve the TSP
• Should generate a random OR a circular list of
  cities
• Solve TSP for a fixed number of generations or
  until convergence or until user says stop

33
?