Archivebased Cooperative Coevolutionary Algorithms

1
Archive-based Cooperative Coevolutionary
Algorithms

• Paper written by
• Liviu Panait
• Sean Luke
• Joseph Harrison
• Published in the Proceedings of the 8th annual
  conference on Genetic and evolutionary
  computation (GECCO '06)
• Presentation by
• Ronald Nussbaum

2
Coevolutionary Algorithm

• Breaks the problem into components
• Can have any number of components
• Each component is represented by a separately
  evolved population
• Fitness Function
• A solution is composed of one individual from
  each component
• The fitness of an individual is the maximum
  reward it received of all evaluations it
  participated in
• Collaborators Individuals from the other
  components in a solution

3
Competitive Coevolution

• Easy to envision as a predator-prey relationship
  between components
• Given a fitness function f(x, y)
• One component seeks to maximize the value of f
• The other component seeks to minimize the value
  of f

4
Cooperative Coevolution

• Biologically, we can imagine this as a symbiotic
  relationship between organisms
• Given a fitness function f, all components seek
  to maximize the value of f

5
Why Cooperative Coevolution?

• Goal is to simplify the search space
• Imagine if every component in a problem was
  completely independent of one another
• Instead of an O(an) search space, we would have
  n separate problems with a search space of O(a)
• We wouldn't need a coevolutionary algorithm at
  all.
• In real problems, components will have
  dependencies
• However, some of the search space will consist of
  independent regions
• Cooperative coevolutionary algorithms seek to
  exploit this in order to reduce the search space

6
Pitfalls of Cooperative Coevolution

• The fitness of individuals is sensitive to the
  components with which they are teamed
• Cooperative coevolutionary algorithms may
  gravitate towards suboptimal solutions
• Computing the fitness of an individual is
  expensive!
• The cCCEA algorithm evaluates every individual
  against every possible combination of
  collaborators
• The rCCEA algorithm evaluates five random
  combinations, plus the fittest individuals from
  the other populations from the previous generation

7
Archive-based Cooperative Coevolutionary
Algorithms

• Attempt to overcome the downfalls of a regular
  CCEA by maintaining an archive of individuals
  from each component which are good collaborators
• Members of the population are only tested against
  individuals in the archive
• Like everything else in evolutionary computation,
  the archive can be implemented in multiple ways
• The authors examine two archive-based algorithms,
  Bucci and Pollack's pCCEA 1, and their own iCCEA

8
Domination

• Take two individuals x1 and x2
• x1 dominates x2 if
• f(x1, y) gt f(x2, y) for all y
• f(x1, y) gt f(x2, y) for at least one y

9
pCCEA

• pCCEA employs an elitist archive in each
  population which... were effective in assisting
  some member of the other population
• In other words, the archive is the set of
  non-dominated individuals
• Algorithm
• 1. Evaluate every individual with every member
  of the archive
• 2. Assemble the set of non-dominated individuals
  (new archive)
• 3. Fill rest of population by breeding random
  individuals from the entire population
• Select two random individuals
• Breed them together if neither dominate the other

10
Drawbacks of of pCCEA

• There's always a downside, isn't there?
• No attempt is made to keep the archive small
• pCCEA does not use fitness to select individuals
  an individual may be selected if it collaborates
  marginally better with just one other partner
• These issues tend to slow evaluation
• Even worse, an infinite Pareto frontier leads to
  the archive consuming the entire population

11
iCCEA

• At the beginning of a run, the archive is the
  entire population
• Algorithm
• 1. Like pCCEA, iCCEA begins with evaluating each
  individual against every member of the archive
• 2. iCCEA then builds the new archive with
  collaborators that produce the same rank-ordering
  of fitness of individuals in the other population
• If a collaborator cannot increase the rank of
  some individual, it is not included in the
  archive
• Goal is to select an archive which would produce
  the same rank as evaluating against the entire
  population
• 3. Remainder of the population is then filled
  out using tournament selection to fill the rest
  of the population

12
Experiments

• Tested problems with multimodal domains (MTQ,
  Griewangk, Rastrigin), as well as problems with
  infinite Nash equilibra (OneRidge, Rosenbrock,
  Booth), as well as a problem with multimodal
  domains with infinite Nash equilibria (SMTQ)
• Each algorithm with given a budget of 51200
  evaluations
• Each experiment was repeated 250 times for
  statistical significance
• pCCEA, cCCEA, rCCEA, and iCCEA with different
  settings for minimum distance were tested

13
Experiment Maximum of Two Quadratics Function

• This is a simple two peak domain
• One global optimum
• One local optimum
• The height and width of the peaks affect the
  problem difficulty
• Performance
• iCCEA with a small minimum distance was the best
• cCCEA, pCCEA, and iCCEA with smaller values were
  in the second tier
• rCCEA came in last, frequently converging to the
  local optimum

14
Experiment Griewangk Function

• f(x, y) -1 x² / 4000 y² / 4000
  cos(x)cos(y / v2)
• Global optimum at f(0, 0) 0
• Many local optima
• Performance
• iCCEA with a small minimum distance was the best
• cCCEA did almost as well
• rCCEA and pCCEA were significantly worse

15
Experiment Rastrigin Function

• f(x, y) -20 x² 10cos(2px) y² 10cos(2py)
• Similar to the Griewangk function, Rastrigin
  contains one global optimum at f(0, 0) 0 and
  many local optima
• Performance
• iCCEA and rCCEA performed the best
• cCCEA and pCCEA did a bit worse

16
Experiment OneRidge Function

• f(x, y) 1 2min(x, y) max(x, y)
• f(1, 1) 2 is the global optimum
• There are an infinite amount of Nash equilibria
• Since any f(x, y) where x y is a Nash
  equilibria
• This means that both populations must
  simultaneously change to a new equilibrium in
  order to improve
• Performance
• rCCEA found the global optimum every trial, as
  did iCCEA when the minimum distance was greater
  than 0
• cCCEA always got near the global optimum
• pCCEA never came close to the global optimum

17
Experiment Rosenbrock Function

• f(x, y) -(100(x² y²)² (1 x)²)
• Global optimum is f(1, 1) 0
• There are an infinite number of ridges (Nash
  equilbria) along the hill
• Performance
• rCCEA and iCCEA did the best
• cCCEA and iCCEA with a very low minimum distance
  were close behind
• pCCEA did the worst

18
Experiment Booth Function

• f(x, y) -(x 2y 7)² - )2x y 5)²
• Global optimum is f(1, 3) 0
• Similar to the previous two problems
• Performance
• rCCEA performed the best
• ICCEA and cCCEA performed decently
• pCCEA was a bit behind

19
Experiment - SMTQ

• For the final experiment, the researchers devised
  a function like Griewangk and Rastrigin, having
  many modalities, with each peak containing
  infinite Nash equilbria
• f(x, y) insert 6 lines of nasty equations
• Performance
• iCCEA performed the best
• cCCEA came close
• rCCEA often got stuck on local suboptima
• pCCEA just got stuck in random places

20
Conclusions

• iCCEA outperformed pCCEA
• All problem domains tested against had an
  infinite number of Pareto non-dominated points,
  which lead to pCCEAs archive consuming the entire
  population
• Comparing mean performance instead of median
  performance closes the gap somewhat
• My observations
• cCCEA always did moderately well
• rCCEA had very good or very bad performance
• iCCEA had the best performance and pCCEA the
  worst
• While promising, the paper is not entirely
  convincing that an archive is actually needed
• It is clear that if an archive is used, it needs
  to be kept small

21
References

• 1 A. Bucci and J. Pollack. On identifying
  global optima in cooperative coevolution. In
  Hans-Georg Beyer et al. 2, pages 539-544.
• 2 Hans-Georg Beyer et al., editor.
  Proceedings of the Genetic and Evolutionary
  Computation Conference (GECCO) 2005. ACM, 2005.
• The graphs in slides 14, 15, 17, and 18 can be
  found at A. Hedar's homepage at Kyoto University,
  http//www-optima.amp.i.kyoto-u.ac.jp/member/stude
  nt/hedar/Hedar_files/TestGO_files/Page364.htm