Geometric Interpretation of Crossover

1
Geometric Interpretation of Crossover
BCTCS 2005

• Alberto Moraglio
• amoragn_at_essex.ac.uk

2
Contents

• I Quick Preliminaries
• II Geometric Interpretation of Crossover
• Extremely quick overview of its implications
• III Unification of Major Representations
• IV Crossover Principled Design
• V Is Biological Recombination Geometric?
• VI Unity of Evolutionary Search

3
I. Quick Preliminaries
4
Evolutionary Algorithms

• Are function optimizers
• Mimic biological evolution
• Are robust, hence preferred for real world
  problems
• Have little theory to explain how and why they
  work
• There are various flavours

5
Evolutionary Algorithm Template

• Problem representation independent

6
Standard representations EAs flavours/dialects

• Binary strings (genetic algorithms, the classic)
• Real code vectors (evolution strategies,
  continuous optimization)
• Permutations (order-based GAs, combinatorial
  optimization)
• Parse trees (genetic programming, evolution of
  computer programs)
• Algorithmically irrelevant differences
  name/authorship/solution interpretation/domain of
  application
• Algorithmically relevant differencessolution
  representation/genetic operators

7
What is crossover?
8
Mutation Crossover for binary strings

• Mutation bit flip at random position
• 101001 ? 101101
• Crossover selection crossover point at random
  swap tails
• 101001 ? 101000
• 111000 ? 111001
• 1100 1100
• All offspring match the parent schema

9
II. Geometric Interpretation of Crossover
10
Genetic operators Neighbourhood structure

• Forget the representation and consider the
  neighbourhood structure ( search space
  structure)
• Mutation offspring are close to their parent ?
  in the direct neighbourhood

11
Direct Neighbour Mutation
Representation Binary String Move Bit
Flip Neighbourhood Hamming Representation
Move Neighbourhood
100
101
000
001
111
110
?
010
011
Mutation Offspring in the direct
neighbourhood What is crossover?
12
Neighbourhood and Crossover

• Crossover idea combining parents genotypes to
  get children genotypes somewhere in between
  them
• Topologically speaking, somewhere in between
  somewhere on a shortest path
• Why on a shortest path?

13
Shortest Path Crossover
Parent1 011101 Parent2 010111 Children
0111
Children are on shortest paths More than one
shortest path in general
14
Interpretation Generalization

• Traditional mutation crossover have a natural
  interpretation in the neighbourhood structure in
  terms of closeness and betweenness
• Given any representation plus a notion of
  neighbourhood (move), mutation crossover
  operators are well-defined

15
From graphs to geometry

• Forget the neighbourhood structure and consider
  the metric space ( space with a notion of
  distance)
• The distance in the neighbourhood is the length
  of the shortest path connecting two solutions
• Mutation ? Direct neighbourhood ? Ball
• Crossover ? All shortest paths ? Line Segment

16
Balls Segments

• In a metric space (S, d) the closed ball is the
  set of the form
• where x belongs to S and r is a positive real
  number called the radius of the ball.
• In a metric space (S, d) the line segment or
  closed interval is the set of the form
• where x and y belong to S and are called extremes
  of the segment and identify the segment.

17
Squared balls Chunky segments
18
Uniform Mutation Uniform Crossover

• Uniform topological crossover
• Uniform topological e-mutation

Genetic operators have a geometric nature
19
Representation independentand rigorous
definition ofcrossover and mutation in the
neighbourhood seen as a geometric space
20
This is cheating! I have generalized from a
single example of solution representation!
21
III. Unification of Major Representations
Operators
22
Minkowski spaces real vectors
Representation real vectors Neighbourhoods
continuous (3 types) Distances Minkowski
distances Implementation algebraic manipulation
of real vector (equation of line passing through
two points) Pre-existing recombination
operators- both blend crossovers and discrete
crossovers fit geometric definition- extended
blend crossovers do not fit
23
Hamming spaces binary strings
Representation binary/multary strings Neighbourho
ods bit-flip/site substitution Distances
Hamming distances Implementation symbolic
manipulation of multary strings (mask-based
crossovers) Pre-existing recombination
operators- all binary crossovers fit the
geometric definition
24
Cayley spaces - permutations
Representation permutations Neighbourhoods adj.
swap, swap, reversal, insertion Distances
corresponding distances Implementation minimal
permutation sorting by X move algorithms- adj.
swap bubble sort- swap selection sort -
insertion insertion sort - reversal
approximated MPS by reversals (NP-Hard))
Pre-existing recombination operatorsvarious
pre-existing crossover operators are sorting
algorithm in disguise (because sorting
permutations is easier than sorting vectors of
other items)
25
Syntactic tree spaces
Representation syntactic tree (lisp
expression) Neighbourhood weighted sub-tree
neighbourhood Distance structural
distance Implementation - sub-tree swap
crossover - common region mask based crossover
Pre-existing recombination operators-
traditional crossover (non-geometric)-
homologous crossover - the geometric framework
can help to clarify what is the landscape and
distance related to homologous crossover and a
distance connected with a geometric crossover
which traditional crossover is an approximation
26
Significance of Unification

• Most of the pre-existing crossover operators for
  major representations fit geometric definition
• Established pre-existing operators have emerged
  from experimental work done by generations of
  practitioners over decades
• Geometric crossover compresses in a simple
  formula an empirical phenomenon

27
IV. Crossover Principled Design
28
Crossover Principled Design

• Domain specific solution representation is
  effective
• Problem for non-standard representations it is
  not clear how crossover should look like
• But given a combinatorial problem you may know
  already a good neighbourhood structure
• Geometric Interpretation of Crossover ? Give me
  your neighbourhood definition and I give you a
  crossover definition

29
Crossover Design Example

?
30
Non-labelled graph neighbourhood
MOVE Insert/remove an edge Fixed number of
nodes
31
Offspring
32
V. Is Biological Recombination Geometric?
33
Levenshtein spaces sequences
Representation multary sequences (DNA/amino
acids) Neighbourhood insertion deletion
substitution (compound edit move) Distance
Levenshtein distance Implementation inexact
sequence alignment (dynamic programming) and
sites exchange (crossover mask) Pre-existing
recombination operators- none- it could be a
good crossover for linear GP- it could be a
better model of biological crossover to study
molecular evolution because it keeps into account
the inexact alignment due to molecular annealing
of DNA strands that producesevolution of size
variation
Parent1AGCACACA Parent2ACACACTA best inexact
alignment (with gaps) AGCACAC-A ?
Child1AGCACACTA A-CACACTA ? Child2ACACACA
34
A simple model of (homologous) biological
recombination fits the geometric definition under
a DNA distance used in bioinformatics
35
VI. Unity of Evolutionary Search
36
Example of evolutionary search
37
Abstract convex evolutionary search

• Main result an evolutionary algorithm using
  geometric crossover with any probability
  distribution, any kind of representation, any
  problem, any selection and replacement mechanism,
  does the same search convex search
• Proof based on abstract convexity (axiomatic
  geodesic convexity) and axiomatization of search
  process (abstract search process)

38
Nearly Over!
39
Future work
THEORY Generalizing and accommodating
pre-existent theories into geometric framework
(schema theorem, fitness landscapes,
representation theories) PRACTICE Testing
crossover principled design on important problems
with non-standard representation (problem domain
representation)
40
Questions?