Parallel Evolutionary Algorithms

1
Parallel Evolutionary Algorithms
2
Goals of Parallelism

• Speed The same results in less time
• Robustness The same results in at most the same
  time
• Quality Better results in the same time

3
Non-panmictic Population Models

• Classification after Gorges-Schleuter (1991)
• Island Model
• Stepping Stone Model
• Neighborhood Model

4
Non-panmictic Population Models

• Island Model
• Seperated (panmictic) subpopulations
• Genetic material randomly exchanged between
  subpopulations

5
Non-panmictic Population Models

• Stepping Stone Model
• Seperated (panmictic) subpopulations are placed
  in a topology (e.g., ring, torus)
• Exchange of genetic material only between
  adjacent subpopulations

6
Non-panmictic Population Models

• Neighborhood Model
• A single spatially structured population
• Individuals only interact with neighbors ?
  local reproduction

7
Non-panmictic Population Models

• Neighborhood Model

8
Different Terminology

• Migration Model Stepping Stone or Island Model
• Diffusion Model, Plant Pollination Model
  (Goldberg) Neighborhood Model

9
Different Terminology

• Computational view
• Coarse grained Stepping Stone or Island Model
• Fine grained Neighborhood Model

10
Different Terminology

• Exchange of genetic material
• Migration Information is moved
• Pollination Information is copied

11
Migration ModelOutline of the Algorithm

• FOREACH population
• BEGIN PARALLEL
• WHILE NOT stop-condition
• BEGIN
• select parents
• produce offspring
• select emigrants
• send emigrants
• receive immigrants
• integrate immigrants
• evaluate population
• END
• END PARALLEL

12
Migration ModelDiscussion

• Minor changes to the original algorithms
• Scalable parallelism
• - Additional parameters

13
Migration ModelDiscussion

• Experimental results
• Short isolation times, high connectivityExploita
  tion of part of SS Tuned for speed and accuracy
• Long isolation times, low connectivityExploratio
  n of SSTuned for global optimization

14
Migration Model Emigration

• Which individuals should emigrate
• The best? ? Danger of premature stagnation
• The worst? ? Small chance to survive in target
  population
• Randomly chosen? ? Good compromise

15
Migration Model Immigration

• How can immigrants be integrated?
• SimpleJust adding them to population Mule
  effect
• Keep them alive for some number of
  generationsChance to establish

16
Mule effect

• Subpopulation search different attractive areas
  of SS
• Recombined offspring of two individuals is likely
  to be worse than either of parentsNo chance to
  produce offspring (like mules)

17
Neighborhood ModelOutline of the Algorithm

• FOREACH individual
• BEGIN PARALLEL
• WHILE NOT stop-condition
• BEGIN
• select parents from neighborhood
• produce offspring
• evaluate offspring
• replace local individual
• END
• END PARALLEL

18
Neighborhood ModelDiscussion

• Self-organizing forming of subpopulation
• isolation by distancemule effectniching
• - Local selection differs completely from
  traditional EAs
• - Parallelism not scalable

19
Neighborhood ModelDiscussion

• Experimental results
• Local selection causes clustering of individuals
  heading for the same optimum
• Large NeighborhoodsExploitation
• Small NeighborhoodsExploration

20
Local Reproduction

• Reproduction is restricted to a
  neighborhoodBiological model demes
• Local selectionderived from the panmictic
  operators, e.g.
• Local (m, l) or (m l) selection
• Local proportional selection
• Local tournament selection
• Basic principleApply the original operators as
  if the deme were the whole population

21
ExampleLocal Proportional Selection

• v individual to be replaced
• Nv set of indices of adjacent individuals
• v-local relative fitness

22
ExampleLocal Proportional Selection

• v-local cumulative relative fitness
• Draw a random number from 0, 1) and choose
  individual k with

23
Mixed-Model Approaches

• Internally spatially structured subpopulation
• External neighborhood relation between
  populations
• Common approach
• Individuals live in topology, e.g., a torus
• Parts of the population (e.g., slices of the
  torus) are mapped to tasks

24
Mixed-Model Approaches

• Best of both models
• Scalable parallelism
• - Many external parameters- Local population
  sizes- Exchange rates- Isolation time

25
Conclusions

• Parallel EAs are a not only parallel variants of
  traditional EAs, but a new class of algorithms
• Premature stagnation is avoided by mechanisms
  such as isolation and niching
• Parallel EAs can be tuned for either exploitation
  or exploration