Genetic Algorithms: An Introduction

1
Institute of FIeld roBOtics KMUTT Research and
Development Center for Intelligent Systems KMITNB
2
Genetic Algorithms An Introduction
Nachol Chaiyaratana BEng(Hons), PhD, MIEE,
MIEEE Assistant Professor of Electrical
Engineering
3
What is a Genetic Algorithm?

• A stochastic search or optimisation technique in
  which its search mechanisms are based on the
  Darwinian concept of the survival of the fittest
• Developed by Holland (1975) and his colleagues in
  order to model the mechanisms of natural
  selection.

4
Mechanisms behind a GA

• The use of a parallel search where solutions in
  one search iteration or generation undergo a
  number of transformations in order to achieve
  better solutions in the next generation.

5
Mechanisms behind a GA

• Transformations on solutions or individuals
  within the population in one generation are done
  to explore the search space and promote the
  propagation of fit or desired characteristics
  within one generation to the next.
• The optimal solution found is represented by the
  fittest individual obtained.

6
Characteristics of a GA

• A genetic algorithm search is done in the search
  space containing decision variables of the
  optimisation problem which has been coded in a
  specific way.
• A genetic algorithm searches from a population of
  search points, not a single one.

7
Characteristics of a GA

• A genetic algorithm uses payoff information which
  is based on the objective function of the
  optimisation problem, not derivatives or other
  auxiliary knowledge, to guide the search
  direction.
• A genetic algorithm uses probabilistic
  transmission rules, not deterministic ones, in
  its search mechanisms.

8
Terminology

• Comparison between technical terms used in a
  genetic
• algorithm and their equivalent counterparts

9
Simple Genetic Algorithm

• 1. Create an initial population of random
  chromosomes.
• 2. Decode the chromosome of every individual in
  order to obtain decision variables.
• 3. Based on the decision variables obtained,
  calculate the objective value of each solution in
  the population.

10
Simple Genetic Algorithm

• 4. Calculate the fitness of each individual,
  using the objective value obtained.
• 5. Create a new population from the current
  population, using the fitness value as an
  indicator on how to select individuals from the
  current population.

11
Simple Genetic Algorithm

• 6. Perform a transformation on the new
  population, using crossover and mutation
  operations.
• 7. Go back to step 2 until a convergence is
  observed from the solutions found or a fixed
  number of iterations is reached. Note that one
  loop from steps 2 to 6 is called one generation
  of a genetic algorithm run.

12
Five Main Operations

• Chromosome coding
• Fitness evaluation
• Selection
• Crossover
• Mutation

13
Chromosome Coding

• The most common form of code which can be used to
  represent decision variables is binary code.
• Each solution will contain a set of genes where
  each gene can have either allele 0 or allele 1.
• Other variations of code which have binary form -
  Gray code

14
Chromosome Coding

• Schematic diagram of chromosome coding using
• binary representation

15
Fitness Evaluation

• The fitness of each individual in the population
  will have a direct correlation to the objective
  value of its corresponding solution.
• To obtain the fitness value, the chromosome of
  each individual has to be decoded into decision
  variables.

16
Fitness Evaluation

• Then the objective value of each solution can be
  calculated from its decision variables, using the
  cost or profit function of the optimisation
  problem concerned.
• Maximisation Fitness Profit
• Minimisation Fitness Constant-Cost

17
Selection

• Fitness selection helps promoting the propagation
  of fit individuals from one generation to the
  next.
• A new population will be created by
  proportionally reproducing fit individuals from
  the current population according to their fitness.

18
Selection

• The ratio between each individual fitness and the
  total sum of fitness from all individuals in the
  current population will indicate the aimed
  proportion of each current individual in the new
  population.
• An individual with high fitness will take a large
  proportion of the desired new population.

19
Selection

• An individual with low fitness will be given a
  small proportion of the same new population.
• Various techniques e.g. roulette wheel selection
  and stochastic universal sampling selection.

20
Selection

• Schematic diagram showing roulette wheel
  selection
• and stochastic universal sampling

21
Crossover

• Two individuals (parent individuals) from the
  pool of selected individuals, obtained after
  fitness selection, are randomly picked out.
• Genes from both parent individuals are passed on
  to two offspring individuals according to some
  defined rules.

22
Crossover

• The possibility that a crossover will occur after
  two parent individuals are selected from the
  population is given by the crossover probability.
• Usually, the crossover probability is set to a
  value in the range of 0.7 - 0.9.
• Various techniques e.g. n-point crossover and
  uniform crossover

23
n-Point Crossover

• 1-point and 2-point crossovers

24
Uniform Crossover

• Uniform crossover

25
Mutation

• A mutation operation can be thought as a small
  perturbation on the chromosome of an individual.
• Mutation leads to a search at a neighbouring
  point from the original search point, dictated by
  the structure of the chromosome.

26
Mutation

• In the case of a binary chromosome, a mutation
  can be achieved by reversing the allele value of
  a gene.
• The possibility of a mutation occurring on one
  particular gene on the chromosome is given by the
  mutation probability.
• Generally, mutation probability is set to a value
  in the range of 0 to 0.1.

27
Mutation

• Bit-flipped mutation

28
Summary of Simple GA

• Pseudo code for a simple genetic algorithm

29
Evolutionary Algorithm

• An evolutionary algorithm is a modified genetic
  algorithm which utilises genotypic representation
  other than binary chromosomes.
• In certain cases, genetic operators are also
  modified so that they will become more suitable
  for use in a particular problem.

30
Comparison between GA and EA

• Difference between a genetic algorithm and an
• evolutionary algorithm

31
Multi-Objective Optimisation GA

• In a number of applications, the optimisation
  problem contains a set of conflicting objectives
  where all objectives cannot be optimised
  simultaneously.
• Such an optimisation problem is referred to as a
  multi-objective or multi-criteria optimisation
  problem.

32
Multi-Objective Optimisation GA

• Early attempt
• Schaffer (1984) - Vector Evaluated Genetic
  Algorithm (VEGA)
• Two Approaches
• Aggregating approach
• Pareto-based approach

33
Aggregating Approach

• All optimisation objectives are weighted and
  added together in order to form a single
  objective.
• Then a genetic algorithm is used to solve the
  newly formed single-objective problem in the
  usual way.
• One best compromised solution is obtainable.

34
Aggregating Approach

• Proven to be popular due to its simplicity
• It has also been used in the situation where the
  objectives or constraints of the problem can only
  be defined in fuzzy terms (Trebi-Ollennu and
  White, 1997)

35
Pareto-Based Approach

• The true solutions to the multi-objective
  optimisation problem will be non-dominated
  solutions or Pareto optimal solutions.
• For a minimisation problem, solution x dominates
  solution y (x ltp y) when the following conditions
  occur,

36
Pareto-Based Approach

• The level of Pareto optimality can be determined
  from objective values.
• Pareto optimality can be used to define the
  fitness of solutions
• Non-dominated solutions are said to be fitter
  than dominated ones.

37
Pareto-Based Approach

• Examples
• Multi-Objective Genetic Algorithm
• (Fonseca and Fleming, 1993)
• Niched Pareto Genetic Algorithm
• (Horn and Nafpliotis, 1993)

38
Fitness Assignment in MOGA
 

• Two-objective minimisation problem

Fonseca and Fleming (1998) Goldberg (1989)
39
Genetic Programming

• Genetic programming is a special form of
  evolutionary algorithm in which a tree-structured
  chromosome representation is used (Koza, 1992).
• The evaluation of an expression from the
  chromosome is done in a hierarchical manner.

40
Tree-Structured Chromosome

• Chromosome contains two components
• Function Set
• e.g. arithmetic functions, transcendental
  functions, logical functions and conditional
  statements
• Terminal Set
• e.g. variables and constants

41
Example of a Chromosome

• Chromosome representing an XOR function
• Function Set, F AND, OR, NOT
• Terminal Set, T D0, D1
• Symbolic Expression
• (OR (AND (NOT D0) (NOT D1)) (AND D0 D1))

42
Chromosome for XOR Function
Five internal points are members of function
set Four external points are members of terminal
set
43
Tree Generation Techniques

• Grow Method
• Non-uniform structured tree
• Full Method
• Uniform structured tree
• Ramped Half-and-Half
• Grow method Full method

44
Grow Method
Non-uniform structured tree
45
Full Method
Uniform structured tree
46
Crossover in Genetic Programming

• Parent Symbolic Expression
• (OR (NOT D1) (AND D0 D1))
• (OR (OR D1 (NOT D0))
• (AND (NOT D0) (NOT D1)))
• Offspring Symbolic Expression
• (OR (AND (NOT D0) (NOT D1))
• (AND D0 D1))
• (OR (OR D1 (NOT D0)) (NOT D1))

47
Parent Trees
48
Offspring Trees
49
Mutation in Genetic Programming

• Change from one terminal to another terminal
• Change from a terminal to a non-terminal node
  (sub-tree)
• Change from a non-terminal node to a terminal
• Change from one function to another function