An Introduction to Genetic Algorithms

1
An Introduction to Genetic Algorithms

• Lecture 1
• September 20, 2006
• Ivan Garibay
• igaribay_at_cs.ucf.edu

2
Motivation learn from nature
Introduction

• Nature evolve strikingly complex organisms in
  response to complex environmental adaptation
  problems with apparent ease
• Localize and extract principles from nature
• Apply them to design algorithms

3
Evolution

• Charles Darwin (1859) On the origin of species
  by means of natural selection
• Reproduction does not produce a perfect copy,
  always minor variations (mutations)
• Some variations are advantageous some are not
• Individuals with advantageous variations are more
  likely to survive and reproduce (natural
  selection, or the survival of the fittest)
• The variations and inheritable
• Species are continuously adapting to their
  environment

4
Genetics

• Science of heredity
• Gregor Mendel (1865) units of inheritance Genes
  (traits)
• Organisms form by cells
• Each cell has information necessary to construct
  a new organism genome
• Genome set of chromosomes
• Chromosome set of genes
• Genes are DNA segments associated with a
  characteristic (i.e. eye color)
• Allele is a particular gene value (blue, black,
  etc)

5
DNA Information
Rethinking Evolutionary Computation

• DNA molecule is an information structure
• Store information digitally (chain of
  nucleotides)
• Nucleotide deoxyribose sugar phosphate
  Nitrogenous base
• Nitrogenous bases Adenine, Thymine, Cytosine,
  Guanine
• DNA is an amazingly efficient, highly specialized
  structure for information storage, replication,
  expression and evolution

6
Historical perspective
Evolutionary Computation

• Evolutionary Strategies
• Rechenberg, 1965
• Population of two
• Only mutation
• Real value parameter optimization

• Evolutionary Programming
• Fogel, Owens, and Walsh, 1966
• Only mutation
• Evolving Finite State Machines

• Genetic Algorithms
• Holland, 1975
• Population based
• Crossover and mutation
• Study adaptation
• Schema Theorem

7
GA terminology from biology
Chromosome (string)
gene
Population
individual
Fitness based Selection
Crossover Mutation
Genetic Operators
Generation i
Generation i1
8
Simple Genetic Algorithm
procedure GA begin initialize
population while termination condition not
satisfied do begin evaluate current
population members select parents from
current population apply genetic operators to
selected parents set offspring equal to
current population end end
9
Genetic Algorithm Components

• Population of individuals
• Fitness Function
• Selection Function
• Genetic Operators

10
Individuals
gene
allele
0 1 0 0 0 1 1 1 0 0 1 1 0 1 0 1 1
1 1 0 0 0 1 1

• Each individual represent a candidate solution
• String of 1s and 0 (binary representation)
• Could take any other form (tree, integers, etc)
• Needs to be decoded to have meaning Genotype to
  Phenotype

11
Problem Representation
Rethinking Evolutionary Computation

• Problem specific
• Different representations are different problems
  for a GA
• Map a string (structure) into a instance of a
  solution
• Representation is very important
• Define the space to be explored
• Define the space structure variations are
  meaningful

Genotype to Phenotype
Genome (DNA) Organisms Computational
Instance of Evolutionary Problem Structure
Solution Bit String Ordering of cities
for TSP Logo instructions Antena
12
Binary Representation

• Example encoding 4 parameters
• Param1 value 1000 8
• Param2 value 1011 11
• Etc.,

13
Fitness function

• Problem specific component
• Function takes as input an individual
  (chromosome)
• Function return a numerical value that determines
  how good the individual is
• Natural Selection fitness function environment
• Genetic Algorithm fitness function is user
  defined
• Typically higher is better

14
Selection

• Survival of the fittest
• Select the best individuals
• Based on fitness function
• Drives exploitation exploit good genes found so
  far
• Multiple Types
• Proportional
• Rank
• Tournament (most used)

15
Fitness proportional Selection

• Holland, 1975.
• Expected number of times an individual is
  selected to reproduce is proportional to its
  fitness relative to the total population fitness.
• where f(i) is the fitness of individual i and f
  is the sum of fitness of all individuals in a
  pop.
• Actual number of offspring may be far from
  expected number

Ps(i) f(i) / fsum
16
Rank Selection

• Similar to Proportional
• Proportional to their rank instead
• Rank selection is weaker than proportional in
  diverse populations
• Rank is stronger than proportional in converged
  populations

Ps(i) r(i) / rsum
17
Tournament Selection

• Select two individuals
• Generate a random number, r, 0 r 1
• If r lt k, select the better of the 2 individuals
• else, select the worse of the 2 individuals
• where k is a parameter.

• Computationally efficient.
• Previous methods require 2 passes
• Compute sum
• Calculate expected number of offspring.
• Rank selection also requires a sort.

18
Genetic Operators

• Crossover
• Biologically inspired
• Combine genes from two individuals to form an
  off-spring (sexual reproduction)
• Mutation
• Biologically inspired
• DNA is copied with errors mutations
• Most of the time mutation problem
• Some times advantage

19
One-point Crossover

• Simplest form of crossover
• Advantage Fairly large change in individuals
  with very little disruption of information

20
Other Crossover Ops

• Two point select two points and exchange middles
• Uniform with probability px exchange or not each
  bit

21
Mutation

• Single parent operator
• Mutation rate (M) is per bit
• Mutation rate per individual M L (individual
  length)
• As a start M 1/L per bit
• Issues
• Low mutation rate minimal exploration
• High mutation rate too disruptive

22
Initialization

• Initial Populations are randomly generated
• Binary case are all randomly generated binary
  strings

23
GA Convergence
24
Termination Criteria

• Found solution
• Number of generations
• Stagnation no more fitness improvement

25
A GA by hand
Onemax problem Maximize the number of ones
Population (0)
Fitness
(0) 11001111 (1) 00100010 (2) 11100100 (3)
10011000 (4) 01100100 (5) 00001001
6 2 4 3 3 2 20/6 3.33
26
A GA by hand
Onemax problem Maximize the number of ones
Population
Fitness
Selected parents
(0) 11001111 (1) 00100010 (2) 11100100 (3)
10011000 (4) 01100100 (5) 00001001
6 2 4 3 3 2
(2) 11100100 4 (0) 11001111 6 (0) 11001111
6 (4) 01100100 3 (0) 11001111 6 (1)
00100010 2
27
A GA by hand
Onemax problem Maximize the number of ones
X
After crossover
Selected parents
M
3 6 5
11101111 11000100 11001100 01100111 11001010 00100
111
(2) 11100100 4 (0) 11001111 6 (0) 11001111
6 (4) 01100100 3 (0) 11001111 6 (1)
00100010 2
7 3 - 2 5 0
28
A GA by hand
Onemax problem Maximize the number of ones
After crossover
M
After mutation
Fitness
6 4 4 4 5 5 28/6 4.67
11101111 11000100 11001100 01100111 11001010 00100
111
7 3 - 2 5 0
11101110 11010100 11001100 01000111 11001110 10100
111
29
A GA by hand
Onemax problem Maximize the number of ones
Population (0)
Fitness
Population (1)
Fitness
6 4 4 4 5 5 28/6 4.67
(0) 11001111 (1) 00100010 (2) 11100100 (3)
10011000 (4) 01100100 (5) 00001001
6 2 4 3 3 2 20/6 3.33
11101110 11010100 11001100 01000111 11001110 10100
111
30
Next Class

• Problem Representation
• Search Spaces
• Fitness Landscapes