Biologically Inspired Computing: Indirect Encodings, and Hyperheuristics

1
Biologically Inspired Computing Indirect
Encodings, and Hyperheuristics

• This is DWCs lecture 7 for
• Biologically Inspired Computing
• Contents
• direct vs indirect encodings / hyperheuristics

2

• Encodings
• (aka representations)
• Direct vs Indirect

3
Encoding / Representation

• Maybe the main issue in (applying) EC
• Note that
• Given an optimisation problem to solve, we need
  to find a way of encoding candidate solutions
• There can be many very different encodings for
  the same problem
• Each way affects the shape of the landscape and
  the choice of best strategy for climbing that
  landscape.

4
Direct encoding for exam timetabling
- Assume 8 exams, E1E8 - 16 slots (mon9am to
thur4pm) - fitness function knows the
student data i.e. which students take which
exams
mon tue wed thur
900 E4, E5 E2 E3, E7
1100 E8
200
400 E1, E6

4, 5, 13, 1, 1, 4, 13, 2
Exam2 in 5th slot
Exam1 in 4th slot
Etc

• Generate any string of 8 numbers between 1 and
  16,
• and we have a timetable!
• Fitness may be ltclashesgt ltconsecsgt etc ,
  and
• is to be minimised

5
Mutating a directly-encoded timetable
mon tue wed thur
900 E4, E5 E2 E3, E7
1100 E8
200
400 E1, E6

4, 5, 13, 1, 1, 4, 13, 2
Using straightforward single-gene mutation
Choose a random gene
6
Mutating a directly-encoded timetable
mon tue wed thur
900 E5 E2 E4 E3, E7
1100 E8
200
400 E1, E6

4, 5, 13 , 9, 1, 4, 13, 2
Using straightforward single-gene mutation
One mutation changes position of one exam
7
Alternative ways to do it

• This is called a direct encoding. Note that
• A random timetable is likely to have lots of
  clashes.
• The EA is likely (?) to spend most of its time
  crawling through clash-ridden areas of the search
  space.
• Is there a better way?

8
An indirect encoding for exam timetabling
Mon Tue wed thur
900
1100
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

E1 is the first one scheduled, so there are no
possible clashes 4th slot is also 4th
clash-free slot
Use the 4th clash-free slot for exam1
9
An indirect encoding for exam timetabling
Mon Tue wed thur
900
1100 E2
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

Suppose the problem data tells us that E1 and E2
would clash. 5th clash-free slot is therefore
slot 6
Use the 5th clash-free slot for exam2
Would clash with E2
No clash with E2
10
An indirect encoding for exam timetabling
Mon Tue wed thur
900 E3
1100 E2
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

Suppose the problem data tells us that E3 has no
clashes with E1 or E2, then 13th clash-free slot
is therefore slot 13
Use the 13th clash-free slot for exam3
11
An indirect encoding for exam timetabling
Mon Tue wed thur
900 E4 E3
1100 E2
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

Nothing before now in slot 1, so this
is Obviously the 1st clash-free slot
Use the 1st clash-free slot for exam4
12
An indirect encoding for exam timetabling
Mon Tue wed thur
900 E4 E3
1100 E5 E2
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

Suppose E4 and E5 have students in common, so 1st
clash-free slot for E5 is slot 2
Use the 1st clash-free slot for exam5
13
An indirect encoding for exam timetabling
Mon Tue wed thur
900 E4 E3
1100 E5 E2,E6
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

Suppose E6 would clash with E4 and E1 so 4th
clash-free slot for E6 is slot 6
Use the 4th clash-free slot for exam6
14
An indirect encoding for exam timetabling
Mon Tue wed thur
900 E4 E3,E7
1100 E5 E2,E6
200
400 E1
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2

Suppose E7 has no students in common with any
other exam, so 13th slot is 13th clash-free slot
Use the 13th clash-free slot for exam7
15
An indirect encoding for exam timetabling
Mon Tue wed thur
900 E4 E3,E7
1100 E5 E2,E6
200
400 E1,E8
The encoded solution tells us how to build the
timetable step by step

4, 5, 13, 1, 1, 4, 13, 2
Suppose E8 clashes with E4 and E5 but not E1

Use the 2nd clash-free slot for exam8
16
Mutating an indirectly-encoded timetable
Mon Tue wed thur
900 E4 E3,E7
1100 E5 E2,E6
200
400 E1,E8

4, 5, 13, 1, 1, 4, 13, 2
Using straightforward single-gene mutation
Choose a random gene
17
Mutating an indirectly-encoded timetable
Mon Tue wed thur
900 E5 E6 E4 E3,E7
1100 E2
200 E8
400 E1

4, 5, 13, 9, 1, 4, 13, 2
Using straightforward single-gene mutation
One mutation causes considerable change
18
Direct encoding an indirect encoding
4, 5, 13, 1, 1, 4, 13, 2
4, 5, 13, 1, 1, 4, 13, 2

mon tue wed thur
900 E4, E5 E2 E3, E7
1100 E8
200
400 E1, E6
Mon Tue wed thur
900 E4 E3,E7
1100 E5 E2,E6
200
400 E1,E8

There are clashes at Mon 9am and Mon 4pm
No clashes, and good slot utilisation
19
Direct encoding Indirect encodings

• The genotype provides the
• parameters of and/or inputs to
• an algorithm that builds the
• Phenotype (the solution). There is
• a very indirect mapping
• between them.
• This means small changes in the
• genotype could have huge effects
• on the phenotype, that are difficult
• to understand in advance
• But
• often means even the initial
• random solutions are quite good,
• and the evolution process focusses
• only on a much smaller search space
• of good candidates.

• The genotype (the data structure
• that encodes a solution) is
• basically the same as the
• Phenotype (the solution). There is
• a simple and direct mapping
• between them.
• This can help with design and
• understanding of operators and
• landscapes. You basically know
• what type of effect a mutation
• will have,
• But
• often means very bad solutions
• can be easily generated. Lots of
• dross to evolve past.

20
Hyper-heuristics a specific type of indirect
encoding

• Indirect encoding
• Dont encode a candidate solution directly
• instead encode parameters/features for a
  constructive algorithm that builds a candidate
  solution
• Hyperheuristic encoding
• Usually, this is an encoding where each gene is
  a constructive algorithm (usually a simple one)
  which builds one part of the solution

21

• In next few slides we will look at examples of
  the simple heuristics that hyperheuristic
  encodings use
• Then we will look at a simple hyperheuristic
  encoding.

22
HH have been most commonly researched for
bin-packing problems
23
Including 2D bin-packing (also called
stock-cutting) given collection of shapes,
find how to cut them out of large lamina leaving
the minimum possible wastage.
24
Example using First-Fit Ascending (FFA)
constructive algorithm for bin-packing
FFA
FFA means we pack them one by one, Each time
taking the smallest unpacked item
25

First-fit Ascending
26

First-fit Ascending
27

First-fit Ascending
28

First-fit Ascending
29

First-fit Ascending
30
Example using First-Fit Descending
First-fit Descending
FFA means we pack them one by one, Each time
taking the largest unpacked item
31

First-Fit Descending
32

First-Fit Descending
33

First-Fit Descending
34

First-Fit Descending
35

First-Fit Descending
36
Other bin-packing - heuristics
FFA would choose this
37
Other bin-packing - heuristics
FFD would choose this
38
Other bin-packing - heuristics
But, this one fits exactly into the partially
full bin, moving it to capacity
39
Other bin packing heuristics

• FFA - pack smallest remaining into 1st
  available bin it will fit in
• FFD - pack largest remaining into 1st available
  bin it will fit in
• BF - 1. consider all gaps in partially full
  bins find an item that exactly
• fills a gap. 2. If no such item
  currently available, do FFA
• BF(2) - 1. consider all gaps in partially full
  bins find an item that exactly
• fills a gap. 2. If no such item, find
  a pair of items that will together
• exactly fill a gap. 3. Otherwise, do
  FFA
• FFAH(p) - if the number of Huge items remaining
  is lt p, use FFA, otherwise use BF
• FFAS(p) - if the number of Small items remaining
  is lt p, use FFD, otherwise use BF(2)
• . Etc many other possibilities

Huge (item is gt 2/3 of bin capacity) small
( item is lt 1/3 of bin capacity)
40
hyper-heuristics

• is a research area which is based on the idea
  of using indirect encodings for logistics
  problems (mainly), where the genotype is (often)
  a list of heuristics Used for timetabling,
  scheduling, bin-packing, stock-cutting, vehicle
  routing, etc ...
• Note at least two main kinds of indirect encoding
  that make use of heuristics. Consider
  timetabling
• A. 1, 2, 4, 1, 7, 6 . Use H1 with
  parameter 1, use H1 with parameter 2,
• use H1 with parameter 4, use H1 with
  parameter 1, etc (our indirect
• encoding for timetabling was like
  this)
• B. 1, 2, 4, 1, 7 ,6 Use H1, then
  use H2, then use H4, then use H1,
• (a hyper-heuristic encoding see
  next 2 slides)
• and of course there are approaches that mix up
  styles A and B
• It gets called a hyper-heuristic approach if
  there is a strong type B element.

41
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 8 heuristic choices
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1

42
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1 pack
  first item with BF

0.5
43
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1 pack
  next item with FFD

5
44
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1 pack next
  item(s) with BF(2)

5
5
45
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1 pack next
  item with FFD

4
5
5
46
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1 pack next
  item(s) with BF(2)

5
5
5
47
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2,1 pack next
  item(s) with FFA

5
1.5
5
5
48
Example simple hyper-heuristic encoding for bin
packing

• weights 2 1 3 1.5 0.5 3.5
  4 4.5 bin capacity 5
• encoding
• 1 FFA, 2 FFD, 3 BF, 4 BF(2)
• Genotype is a list of 7 heuristic choices, (why
  7 ?),
• each either 1, 2, 3 or 4
• Lets interpret 3,2,4,2,4,1,2 ,1 only one
  item left, use FFD

5
5
5
5
49
The original HH paper open-shop scheduling
Hsiao-Lan Fang, Peter Ross, David Corne. "A
promising hybrid GA/heuristic approach for
open-shop scheduling problems." In Proc. 11th
European Conf. on Artificial Intelligence, 1994.

List of choices of these heuristics
50
Notes constructive algorithms

• Each heuristic we have seen is a simple
  constructive algorithm it performs the next
  step in constructing a solution to the problem
• There are many constructive heuristics for bin
  packing, timetabling, scheduling, etc .... often
  called dispatch heuristics
• There are well-known constructive heuristics for
  many other problems, e.g. Prims algorithm for
  building the minimal spanning tree (see an
  earlier lecture), Djikstras shortest path
  algorithm for networks, etc
• Suitably engineered, Prims algorithm can be
  used as part of an indirect encoding for solving
  hard constrained spanning-tree problems ---
  similar for Djikstras algorithm and network-path
  problems.
• And so on

51
Direct vs Indirect Encodings summary other
notes

• Direct
• straightforward genotype (encoding) ? phenotype
  (actual solution) mapping
• Easy to estimate effects of mutation
• Fast interpretation of chromosome (hence speedier
  fitness evlaluation)
• Indirect/Hybrid
• Easier to exploit domain knowledge (e.g. use
  this in the constructive heuristic)
• Hence, possible to encode away undesirable
  features
• Hence, can seriously cut down the size of the
  search space
• But, interpretation of genotype could become
  computationally expensive
• Neighbourhoods tend to be highly rugged.
• ? tradeoff between size of search space and
  ruggedness
• - sensible engineering of the indirect
  encoding often beats the direct
• - however, direct encodings with smarts
  built-in to the operators can be
• the best choice thats for further
  reading or an advanced module

52
Mandatory reading

• slides for
• - Operators (typical mutation and crossover
  operators for different types of encoding)
• - Selection (various standard selection
  methods)
• - More encodings

53
About CW2
54

55
About CW2

• Pamela Hardaker, Benjamin N. Passow and David
  Elizondo. Walking State Detection from
  Electromyographic Signals towards the Control of
  Prosthetic Limbs UKCI 2013
• got signals like this, but on her
• thigh just above the knee

running
walking
standing
56

• Current knee-joint prospects need manual
  intervention to change between standing/walking/ru
  nning modes (the wearer presses a button)
• Can we train a Neural Network to automatically
  detect when to change, on the basis of nerve
  signals from the last 30ms ?

57
Neural Network ?

• You will learn later exactly what it is
• For now

Neural Network This simply applies a complex
mathematical function to the inputs, which
depends on the weight parameters Your EA will
evolve the weight parameters
Input a set of real numbers (e.g.
characterising the last 30ms of
neuromuscular signals)

• Output a decision,
• In this case, either
• standing
• walking
• running

58
About CW2

• Snapshot of Pamelas data
• Time signal state

004.206 6.064453124 standing
004.207 5.693359374 standing
004.208 4.946289061 standing
004.209 4.609374999 standing
004.210 4.589843749 standing
004.211 4.482421874 standing
004.212 4.809570311 standing
004.213 5.102539061 standing
004.214 5.507812499 standing
004.215 5.922851562 standing
004.216 5.561523436 standing
004.217 5.610351561 standing
004.218 5.605468749 standing
004.219 5.551757811 walking
004.220 5.952148437 walking
004.221 6.542968749 walking
004.222 6.923828124 walking
004.223 6.845703124 walking
004.224 6.958007812 walking
004.225 6.484374999 walking
004.226 6.118164062 walking
004.227 5.424804686 walking
004.228 4.023437499 walking
004.229 3.696289061 walking
004.230 3.798828124 walking
004.231 3.579101561 walking
59
About CW2
Max signal strength in last 30ms Max signal
strength in last 20ms Max signal strength
in last 10ms range of sig in last
30ms range last 20ms
range last 10ms
mean last 30ms
mean last 20ms
mean last 10ms

• Snapshot of Pamelas data
• Time signal state

004.206 6.064453124 standing
004.207 5.693359374 standing
004.208 4.946289061 standing
004.209 4.609374999 standing
004.210 4.589843749 standing
004.211 4.482421874 standing
004.212 4.809570311 standing
004.213 5.102539061 standing
004.214 5.507812499 standing
004.215 5.922851562 standing
004.216 5.561523436 standing
004.217 5.610351561 standing
004.218 5.605468749 standing
004.219 5.551757811 walking
004.220 5.952148437 walking
004.221 6.542968749 walking
004.222 6.923828124 walking
004.223 6.845703124 walking
004.224 6.958007812 walking
004.225 6.484374999 walking
004.226 6.118164062 walking
004.227 5.424804686 walking
004.228 4.023437499 walking
004.229 3.696289061 walking
004.230 3.798828124 walking
004.231 3.579101561 walking
1.37207031 1.37207031 1.25488281 2.39746 2.39746 1.7041 0.44873 0.338135 0.674805 0 0 1
1.37207031 1.25488281 0.971679685 2.39746 1.7041 1.37207 0.355469 0.532471 0.390137 0 0 1
2.294921873 2.294921873 2.294921873 2.75391 2.75391 2.75391 0.51709 0.438232 0.486328 0 0 1
2.324218748 2.324218748 2.324218748 3.17383 3.17383 3.17383 0.482422 0.528564 0.570801 0 0 1
2.324218748 2.324218748 2.084960936 3.17383 3.17383 2.69043 0.507975 0.518799 0.466797 0 1 0
2.324218748 2.084960936 1.860351561 3.17383 2.69043 2.37793 0.484701 0.44165 0.416504 0 1 0
2.084960936 1.860351561 1.674804686 2.72949 2.50488 2.31934 0.328939 0.26001 0.103516 0 1 0
Outputs 1 0 0, 0 1 0 or 0 0
1 (standing) (waking)
(running)
60
About CW2
Max signal strength in last 30ms Max signal
strength in last 20ms Max signal strength
in last 10ms range of sig in last
30ms range last 20ms
range last 10ms
mean last 30ms
mean last 20ms
mean last 10ms

• Snapshot of Pamelas data
• Time signal state

004.206 6.064453124 standing
004.207 5.693359374 standing
004.208 4.946289061 standing
004.209 4.609374999 standing
004.210 4.589843749 standing
004.211 4.482421874 standing
004.212 4.809570311 standing
004.213 5.102539061 standing
004.214 5.507812499 standing
004.215 5.922851562 standing
004.216 5.561523436 standing
004.217 5.610351561 standing
004.218 5.605468749 standing
004.219 5.551757811 walking
004.220 5.952148437 walking
004.221 6.542968749 walking
004.222 6.923828124 walking
004.223 6.845703124 walking
004.224 6.958007812 walking
004.225 6.484374999 walking
004.226 6.118164062 walking
004.227 5.424804686 walking
004.228 4.023437499 walking
004.229 3.696289061 walking
004.230 3.798828124 walking
004.231 3.579101561 walking
1.37207031 1.37207031 1.25488281 2.39746 2.39746 1.7041 0.44873 0.338135 0.674805 0 0 1
1.37207031 1.25488281 0.971679685 2.39746 1.7041 1.37207 0.355469 0.532471 0.390137 0 0 1
2.294921873 2.294921873 2.294921873 2.75391 2.75391 2.75391 0.51709 0.438232 0.486328 0 0 1
2.324218748 2.324218748 2.324218748 3.17383 3.17383 3.17383 0.482422 0.528564 0.570801 0 0 1
2.324218748 2.324218748 2.084960936 3.17383 3.17383 2.69043 0.507975 0.518799 0.466797 0 1 0
2.324218748 2.084960936 1.860351561 3.17383 2.69043 2.37793 0.484701 0.44165 0.416504 0 1 0
2.084960936 1.860351561 1.674804686 2.72949 2.50488 2.31934 0.328939 0.26001 0.103516 0 1 0
CW2 evolve a neural network that predicts the
state.
61
What you will do

• From me, you get
• Working NN code that does the job already
• What you will do
• Implement new mutation and crossover operators
  within my code, and test them on Pamelas data
• Write a report comparing the performance of the
  different operators

62
If time, a look at the key bits of those operator
slides
63
Operators for real-valued k-ary encodings

• Here the chromosome is a string of k real
  numbers, which each may or may not have a fixed
  range (e.g. between -5 and 5).
• e.g. 0.7, 2.8, -1.9, 1.1, 3.4, -4.0,
  -0.1, -5.0,
• All of the mutation operators for k-ary
  encodings, as previously described in these
  slides, can be used. But we need to be clear
  about what it means to randomly change a value.
  In the previous slides for k-ary mutation
  operators we assumed that a change to a gene mean
  to produce a random (new) value anywhere in the
  range of possible values

64
Operators for real-valued k-ary encodings

• thats fine we can do that with a real
  encoding, but this means we are choosing the new
  value for a gene uniformly at random.

65
Mutating a real-valued gene using a uniform
distrution

• Range of allowed values for gene 010
• New value can be anywhere in the range, with any
  number equally likely.

66
But, in real-valued encodings, we usually use
Gaussian (Normal) distributions

• so that the new value is more likely than not
  to be close to the old value.
• 
  typically we generate a perturbation
• 
  from a Gaussian distribution, like this
• 
  one, and add that perturbation to the
  old
• 
  value.

0.3
0.2
Probability of choosing this perturbation
0.1
0
-1 -0.5 0 0.5 1
perturbation
67
Mutation in real-valued encodings

• Most common is to use the previously indicated
  mutation operators (e.g. single-gene, genewise)
  but with Gaussian perturbations rather than
  uniformly chosen new values.

68
Crossover in real-valued encodings

• All of the k-ary crossover operators previously
  described can be used.
• But there are other common ones that are only
  feasible for real-valued encodings

69
Box and Line crossover operators for real-valued
chromosomes figure is from this paper
http//cdn.intechopen.com/pdfs/30229/InTech-The_ro
les_of_crossover_and_mutation_in_real_coded_geneti
c_algorithms.pdf

• 
  Treat the parents like vectors
  
  
  
• Fig assumes you are crossing over two 2-gene
  parents, x (x1,x2) and y (y1,y2)
• Box child is
• (x1 u1 (y1 x1), x2 u2(y2 x2) )
• Where u1 and u2 are uniform random numbers
  between 0 and 1
• Line child is x u (y x)
• or (x1 u(y1 x1) , x2 u (y2 x2)
  )
• Where u is a random number between 0 and 1

70
Box and Line crossover general form

• Parent1 x1, x2, x3, ., xN
• Parent2 y1, y2, y3, , yN
• given parameter a (typically values are 0, 0.1
  or 0.25)
• General form
• Child is ((x1 a) u (y1 x1), ((x2 a)
  u (y2 x2), etc)
• Where u is a uniform random number between 0
  and 12a
• Line crossover a 0 u is generated once
  for each crossover, and is the same for every
  gene.
• Extended line crossover a gt 0, u is generated
  once for each crossover, and is the same for
  every gene.
• Box crossover a 0 u is different for
  every gene
• Extended box crossover a gt 0 u is different
  for every gene

71
(No Transcript)