Advanced Examples and Ideas

1

• Advanced Examples and Ideas

2
Three Layer Evolutionary Approach
Local perceptions, such as bald head or long
beard
Encoded behaviors or internal states
Time intervals
Evolve Behaviors
Evolve Motions
Evolve Perceptions
Motions as timed sequences of encoded actions,
for instance RFRFLL
Global perceptions, possibly encoded such as
narrow Corridor or beautiful Princess
Behaviors such as go forward until you find a
wall, else turn randomly right or left
3
Evolve in hierarchy

• Together or separately
• Feedback from model or from real world
• First evolve motions and encode them.
• Then evolve behaviors.
• Finally develop perceptions.

Go to the end of the corridor and then look for
food
If you see a beautiful princess go to her and bow
low.
If you see a dragon escape
4
Evolve in hierarchy
avoid obstacles
Execute optimal motions
Save energy
Look for energy sources in advance
Execute actions that you enjoy
What if robot likes to play soccer and sees the
ball but is low on energy?
5
Optimizing a motion

• Parking a Truck

6
Find the control
Solving this analytically would be very difficult
7
Question How to represent the chromosomes?
Here you see several snapshots of a movie about
parking a truck, stages of the solution process.
8

• t is time u

9
Another example

• Learning Obstacle Avoiding

10
Similar to Braitenberg Vehicle but has 8 sensors
11
how
Input and output data are some form of MV logic

• How would you represent chromosomes?
• Design Crossovers?

• Robot can move freely but has to avoid obstacles
• This can be like the lowest level of behaviors in
  subsumption or other behavioral architecture for
  all your robots

12
Remember the goal when you create the fitness
function
The key to success is often in fitness function
13
Number of collisions
When you train longer you decrease the number of
collisions

• Time of learning

14

• Applications and Problems

15
General GA Schema
16
Evolutionary Methods

• Optimization problems
• Single objective optimization problems
• Multi-Objective optimization Problems

17
More examples of problems in which we use
evolutionary algorithms and similar methods.

• Search Problems (Path search)
• Optimal multi-robot coordination
• Multi-task optimization
• Optimal motion planning of robot arms (Trajectory
  planning of manipulators )
• Motion optimization (optimization of controller
  parameters - morphology in different control
  schemas)
• PID (PI)
• Fuzzy
• Neural
• Hybrid (neuro-fuzzy)
• Path planning and tracking (mobile robots)
• Optimal motion planning of robot arms
• Trajectory planning of manipulators
• Vision computational optimization

18
What are these other algorithm?

• Evolutionary Algorithms - Related techniques
• Ant colony optimization (ACO)
• Particle swarm optimization
• Differential evolution
• Memetic algorithm (MA)
• Simulated annealing
• Stochastic optimization
• Tabu search
• Reactive search optimization (RSO)
• Harmony search (HS)
• Non-Tree Genetic programming (NT GP)
• Artificial Immune Systems (AIS)
• Bacteriological Algorithms (BA)

• You can try them in your homework 1 if GA or GP
  is too easy for you.
• Using them gives you higher possibility of
  creating a successful superior method for a new
  problem

19
GA-operators

• Selection
• Roulette
• Tournament
• Stochastic sampling
• Rank based selection
• Boltzmann selection
• Nonlinnear ranking selection
• Crossover
• One point
• Multiple points
• Mutation

Read in Auxiliary Slides about these methods. Or
invent your own operators for your problem.
20
Your design parameters to be decided

• Genotype length
• Fixed length genotype
• Variable-length genotype
• Population
• Fixed population
• Variable population
• Species inside population
• Geometrical separation

21
Drawbacks of GA

• time-consuming when dealing with a large
  population
• premature convergence
• Dealing with multiple objective problems

Solutions

• Niches
• Islands
• Pareto approach
• Others

22
More examples of using GA in robotics

• Trajectory Planning Problems

23
GA and Trajectory Planning

• GA techniques for robot arm to identify the
  optimal trajectory based on minimum joint torque
  requirements (P. Garg and M. Kumar, 2002)
• path planning method based on a GA while adopting
  the direct kinematics and the inverse dynamics
  (Pires and Machado, 2000)
• point-to-point trajectory planning of flexible
  redundant robot manipulator (FRM) in joint space
  (S. G. Yue et al., 2002)
• point-to-point trajectory planning for a 3-link
  (redundant) robot arm, objective function is to
  minimizing traveling time and space (Kazem,
  Mahdi, 2008)

Projects last years
24
Optimal path generation of robot manipulators

• Control Schema
• Robotic arm kinematic model
• Controller type
• Objective function - optimal path
• Optimization algorithm (method)
• GA use smooth operators and avoids sharp jumps in
  the parameter values.

25

• Adaptive Control Schema Track Control error
  function between outputs of a real system and
  mathematical model
• What we optimize?
• Which parameters must be optimized?
• How many objectives (single objective or
  multiobjective)?
• Collision free? (How to model collision in GA?)

26

• Three join Manipulator
• A three-joint robotic manipulator system has
  three inputs and three outputs.
• The inputs are the torques applied to the joints
  and the outputs are the velocities of the joints
• No ripples

27
Design of robotic controllers

• For n-DOF we will have n inputs ui, i1n, (ui ?
  ?i)
• Controller
• PID (PI)
• Neural network (multilayer perceptron, recurrent
  NN, RBF based NN)
• Fuzzy
• Neuro-Fuzzy (hybrid)

28
Use of Neural Networks

• NN We must to adapt the weights and eventually
  the bias
• The chromosome
• Adapt the weights

29
FUZZY LOGIC

• Fuzzy Logic
• Aggregation of rules
• defuzzification
• free-of-obstacles workspace (Mucientes, et. al,
  2007)
• wall-following behavior in a mobile robot

30
Learning FUZZY LOGIC Controllers

• Learning of fuzzy rule-based controllers
• Find a rule for the system
• Step 1 evaluate population
• Step 2 eliminate bad rules and fill up
  population
• Step 3 scale the fitness values
• Step 4 repeat NI iterations for Step 4 to Step
  9
• Step 5 select the individuals of the
  population
• Step 6 crossover and mutate the
  individuals
• Step 7 evaluate population
• Step 8 eliminate bad rules and fill up
  population
• Step 9 scale the fitness values.
• Step 10 Add the best rule to the final rule
  set.
• Step 11 Penalize the selected rule.
• Step 12 If the stop conditions are not
  fulfilled go to Step 1

31
Encoding fuzzy controls

• The chromosome encode the rules
• Sn is constant in this application but it can be
  also variable to be optimized
• wall-following behavior of the robot
• the robot is exploring an unknown area
• moving between two points in a map
• Requirements
• maintain a suitable distance from the wall that
  is being followed
• to move at a high velocity whenever the layout of
  the environment is permitting
• avoid sharp movements (progressive turns and
  changes in velocity)

32
Path-based robot behaviors

• The requirements are encoded in Universes of
  discourse and precisions of the variables
• right-hand distance (RD)
• the distances quotient (DQ), based on left-hand
  distance
• Orientation
• linear velocity of the robot (LV)
• Linear acceleration
• Angular velocity
• Path of the robot (simulated environments)

33
Fast, reliable, no harm to robot or to environment

• This is useful for out PSU Guide Robot
• Do not harm humans
• Do not harm robot

34

• Fixed points the desired Cartesian path Pt is
  given the problem is to find the set of joint
  paths P? in order to minimize the cumulative
  error between desire and real path during
  trajectory
• Pk is the kinematic model
• Free end points case

Find the set of joint paths, next smooth it
Minimize the cumulative error
35
Weighted Global Fitness

• fitness function (minimization)
• Global fitness Linear function of individual
  objectives
• Fot excessive driving (sum of all maximum
  torques), fq the total joint traveling distance
  of the manipulator, fc - total Cartesian
  trajectory length, tT - total consumed time for
  robot motion
• Penalty function
• Population initialization (probability
  distribution)
• Random uniform
• Gaussian

36
example

• Drug Delivery Problem

37
Drug delivery using microrobots (Tao, et. al,
2005)

• (GA)based area coverage approach for robot path
  planning.
• Drawbacks of most currently available drug
  delivery methods are that the drug target area,
  delivery amount, and
• release speed are hard to be precisely
  controlled.
• It is very difficult or impossible to eliminate
  side effects.
• Open issues
• actively control the delivery process
• Access to appropriate areas that cannot be
  reached using traditional devices
• Current Issues
• On-line path planning (solve unexpected obstacles
  problem)
• Optimal path planning (efficiency, path planning)

38

• microcontroller is used to guide the robot
  movement
• GA-based approach uses fine grid cell
  decomposition for area coverage
• Because the robot will move cell by cell, the
  start point of chromosomes has to be changed
  dynamically whenever the robot reaches the center
  of a cell
• The end point of a chromosome is not fixed and
  needs to be determined by applying GA operators.
• The robots may move from the center of a cell to
  its 8 adjacent cells along 8 directions.
• some obstacles are unknown before drug delivery
  (the robot discover these obstacles during the
  motion)

39

• Expandable chromosomes
• Deleting the path
• Crossover operator

40

• New mutation operators
• Travel further
• Delete
• Reverse delete
• Stretch
• Shortcut
• The algorithm keep mind the visited nodes
• Extension to operational research?

41
Other applications using evolutionary algorithms

• Autonomous mobile robot navigation - Path
  planning using ant colony optimization and fuzzy
  cost function evaluation (Garcia, et. al, 2009).
• Legged Robots and Evolutionary Design
• Optimal path and gait generations (Pratihar,
  Debb, and Gosh, 2002) 0/1 absence or presence
  of rule
• six-legged robot
• collision-free coordination of multiple robots
  (Peng and Akela, 2005)

42
What if you want to optimize two parameters at
the same time?

• Pareto Optimization

43
Pareto Evolutionary Methods
44
What is better this or this?

• We want to optimize both functions f1 and f2

45
Biobjective means two objectives to reach
Pareto solutions for different algorithms
Pareto Front

• We have x and y, two objectives here

46
Pareto front

• The single objective optimisation problem (SOP)
  conduct to a minimization (or maximization) of
  one cost function, less or more complex, that is
  a single objective is taken into account.
• Conversely, the multi-objective optimization
  problem takes into account two or more objective
  that has to be minimized (or maximized)
  simultaneously.
• Some objectives can be in competition, so a
  simultaneous minimization is not possible, but
  only a trade-off among them.
• Some time, the number of objectives can be high,
  like 16 objectives or more that make the
  multi-objective optimization problem (MOP) and
  interesting and challenging area of research

47
Example of Pareto Optimization of two parameters

• Optimization of Airplane Wings

48

• Two objectives Maximize lift, and minimize drag

49
In most of the design space the red method is
better than the blue method It is good to use
many Pareto methods and modify parameters

• Two objectives Maximize lift, and minimize drag

50
Multi-Pareto

• We optimize many parameters,
• We may switch between subsets of them.
• Subsets can have two elements each.

51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55

• Three-dimensional Minimization Problem

56
(No Transcript)
57

• Pareto Front

58
General multiobjective optimization problem

• The multiobjective optimization problem could be
  generally formulated as minimization of vector
  objectives Jt(x) subject to a number of
  constraints and bounds

59
Pareto-optimal set

• In the case of competing objectives a trade-off
  is involved such a problem usually has no unique
  solution.
• Instead, we can admit a set of solutions, equally
  valid non-dominated as a set of alternative
  solutions known as Pareto-optimal set
• In what follows we assume without loss of
  generality that all the function objectives must
  be minimized.
• If we have a maximization case fi we simply
  minimize the function -fi.
• For any two points that are usually named
  candidate solutions V1,V2??, V1 dominates V2 in
  the Pareto sense (P-dominance) if and only if the
  following condition hold

60
The Pareto set

• The Pareto set is the set of PO (Pareto-Optimal)
  solution in design domain and the Pareto Front
  (PF) is the set of PO solutions in the objective
  domain.
• The most popular way to solving the MOP (Multi
  Objective Optimization Problem) is to reduce the
  minimization problem to a scalar form by
  aggregating the objectives in weighted sum, with
  the sum of weights constant
• The weighted sum method has a serious drawback,
  the method usually fail in the case of nonconvex
  PF.

61
Example of a clear picture of Pareto points
62
Nice properties

• GA can provide an elegant solution for tradeoff
  among different minimization of cost function for
  each variable versus total cost or other
  variable.
• Non-convex solutions
• Immigrants, possible solution for jump from
  local minima.
• Dealing with many variables (e.g. 16 variables)

63
Multi-Robots

• Pareto optimal multi-robot coordination with
  acceleration constraints (Jung and Ghrist, 2008)
• collection of robots sharing a common environment
  
• each robot constrained to move on a roadmap in
  its configuration space
• each robot wishes to travel to a goal while
  optimizing elapsed time considering vector-valued
  (Pareto) optima
• all illegal or collision sets are removed.

64
Conclusions

• GA is not a universal panacea to optimization
  problems.
• Coding the problem into a genotype is the most
  important challenge!
• The best selection schema of individuals for
  crossover operator is difficult to be chosen
  apriori (tournament selection seems to be more
  promising)
• A number of parameters are determined
  empirically
• Size of population
• pc and pm even often values inspired from biology
  are given
• Other parameters in hybrid or more sophisticated
  GA

65
Good properties

• One of the most important element in the design
  of a decoder-based evolutionary algorithm is its
  genotypic representation.
• The genotype-decoder pair must exhibit
  efficiency, locality, and heritability to enable
  effective evolutionary search
• locality, and heritability
• small changes in genotypes should correspond to
  small changes in the solutions they represent,
• and
• solutions generated by crossover should combine
  features of their parents

66
(No Transcript)
67
Sources

• Dragos Arotaritei

68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73

• example

74
(No Transcript)
75
(No Transcript)
76
(No Transcript)