Genetic and developmental computing architectures: Modular robotics, Lecture 2

1
Genetic and developmental computing
architecturesModular robotics, Lecture 2
Auke J. Ijspeert Biologically Inspired Robotics
Group (BIRG) Swiss Federal Institute of
Technology, Lausanne
16 January 2006
2
Content of lecture 2 control of locomotion

• A review of animal locomotion
• Locomotion control in robotics
• Locomotion control in modular robotics
• Overview
• Gait tables
• Sine-based
• Neural networks
• Central pattern generators

3
Adaptive motor control in animals
Coordination of multiple degrees of freedom
Modulation
Visuomotor coordination Switching between motor
tasks
Learning new skills
4
Animal Locomotion
Large diversity of different types of
locomotionswimming, crawling, walking, hoping,
burrowing, flying,
5
Multiple redundancies

• Control of locomotion is a difficult and
   ill-posed  problem

• Requires good coordination (right frequencies,
  phases, signal shapes,) of multiple degrees of
  freedom,
• despite the multiple redundancies
• Many possible end-point trajectories
• Many possible postures for a given end-point
• Many possible muscle activations for a given
  posture
• Many possible motor unit activations for a given
  muscle activations

6
Additional difficulties

• In addition to the problem of redundancies,
  locomotion control is complex because it
  requires
• The possibility to adjust speed and direction
• Adapting to terrain
• Optimizing the gaits (finding the fastest, most
  efficient,...)
• Dealing with lesions, and changes in body
  properties (fatigue, aging)
• Good control of balance
• Trying to satisfy multiple constraints
  simultaneously
• Stabilizing head, maintaining equilibrium,
  moving forward, avoiding obstacles,.
• Note legged robots face the same problems!

7
Animal Locomotion Control

• In vertebrates, three main ingredients
• Central pattern generators (CPGs),
• Reflexes, and
• Command signals from higher control centers
  (cerebellum, basal ganglia, motor cortex)

8
Neural control of movement
Caudate
Thalamus
Cerebral cortex
SC
IC
Cerebellum
Brain Stem
Spinal Cord
9
Pattern generators

• Pattern generator simple inputs ? complex
  outputs
• Central pattern generators neural networks
  capable of producing oscillatory patterns without
  oscillatory inputs
• Found in many animals invertebrates
• and vertebrates (e.g. lamprey)
• Locomotion CPGs in spinal cord

• Relatively simple control signals from higher
  control centers to the spinal cord (Shik and
  Orlosky 1966)
• Distributed system multiple coupled oscillators,
  at least one per DOF (Grillner 1985)

10
Different types of gaits

• Salamander swimming and walking

• Biped locomotion walk (at least one leg on the
  ground at all times), running

11
Different types of gaits (continued)
12
Statically versus dynamically stable gaits

• Statically stable gait the center of mass is
  maintained at all times above the support polygon
  formed by the contacts between the limbs and the
  ground

Center of mass
Tripod gait in a hexapod robot
Leg on ground
Leg in the air
13
Content of lecture 2 control of locomotion

• A review of animal locomotion
• Locomotion control in robotics
• Locomotion control in modular robotics
• Overview
• Gait tables
• Sine-based
• Neural networks
• Central pattern generators

14
The problems of legged locomotion control

• Coordinating all the degrees-of-freedom of the
  robot means, for each dof, finding the right
• frequency u1/T,
• phase j,
• amplitude A, and
• signal shape

15
The problems of legged locomotion control

• Underactuated problem a robot cannot follow
  arbitrary motion commands
• Need to coordinate multiple degrees of freedom
• Need to adapt to the terrain
• Need to keep balance
• Need to modify the gait for different speeds and
  directions
• Obstable avoidance
• Visually-guided feet placements
• Adapting to perturbations

16
Different approaches to legged robot locomotion
control in current robots

• Three main approaches for  monolithic  robots
• Trajectory based methods,
• Heuristic control methods, and
• CPG based methods

17
Trajectory based methods

• Main idea design walking kinematic trajectories,
  and use the dynamic equations to test and prove
  that locomotion is stable
• Use a feedback controller to track those
  trajectories
• Most successful approach Zero Moment Point (ZMP)
  method (Vukobratovic 1990)

18
ZMP Approach

• Example Honda robot

19
ZMP Approach summary

• Pros
• Well-defined methodology for proving stability
• Well-suited for expensive robots that should
  never fall
• Cons
• Requires a perfect model of the robots dynamics
  and of the environment
• Requires additional online control to deal with
  perturbations
• Transitions from online control back to desired
  trajectories can be tricky
• Defining good trajectories is time-consuming

20
Virtual Model Control

• For each virtual element producing a force F, the
  joint torque needed to produce that virtual force
  can be computed with
• J is the Jacobian relating the reference frame of
  the virtual element to the robot

21
Virtual Model Control

• Example Flamingo robot at MIT Leg LAB

22
Virtual Model Control summary

• Pros
• Intuitive way of designing a controller
• Does not need an accurate model of the
  environment
• Robust against pertubations
• Cons
• Requires a perfect model of the robots dynamics
• Need to make sure that the virtual forces can
  actually be generated by the robots motors
• Finite-state machine for cycling through the
  different phases is a somewhat rigid mechanism

23
CPG-based control

• Main idea to use oscillators and to replicate
  the distributed control mechanisms found in
  vertebrates

Vestibular Sys.
Visual System
Balance Control
Visuomotor Coord.
Reflexes
CPG
Reflexes
Proprioception
Actuators
24
Concept of Limit Cycle

• A limit cycle is an oscillatory regime in a
  dynamical system
• If the limit cycle is stable, the states of the
  system will return to it after perturbations

25
Quadruped-robot controlled with a CPG-and-reflex
based controller
Kimura Lab, National Univ. of Electro-Communicatio
nsTokyo
26
Quadruped-robot controlled with a CPG-and-reflex
based controller
Kimura Lab, National Univ. of Electro-Communicatio
nsTokyo
Camera control Obstacles detection
Reflex - Knee Bending To avoid obstacles
27
CPG-based Control summary

• Pros
• Distributed control
• Limit cycle behavior (controller-body-environment)
  
• Robust against perturbations
• Smooth trajectories due to the oscillators
• Cons
• Fewer mathematical tools than other methods
• Not (yet) a clear design methodology, it is
  recommended to use learning/optimization
  algorithms

28
Content of lecture 2 control of locomotion

• A review of animal locomotion
• Locomotion control in robotics
• Locomotion control in modular robotics
• Overview
• Gait tables
• Sine-based
• Neural networks
• Central pattern generators

29
Modular robotics challenges

• Efficient locomotion despite unknown
  configurations
• Configurations that change over time
• Distributed control
• Traditional model-based control is not well
  suited
• Requires specific approaches, see next slides

30
Locomotion control in modular robotics

• Different approaches for chain-type modular
  robots (i.e. that activate their joints)
• Gait tables
• Sine-based controllers
• Neural networks
• Central pattern generators (CPGs)
• Approaches for lattice-type modular robots
• Continuous reconfiguration (we will see this in
  the next modular robotics lecture)

31
Locomotion control in modular robotics

• Different design approaches
• Hand-coding
• Evolutionary algorithms
• Other optimization algorithms

32
Gait tables
33
Gait tables

• Most modular robots use precomputed gait tables
• Idea
• Use a simplified state machine for each module
• Follow a prescribed sequence of behaviors for
  each module
• Usually implemented in a master-slave mode, with
  a master module that
• Provides a gait table to all modules, and
• Synchronizes the different phases of the gait

34
Gait tables example
35
Execution of the gait table

• The motor executes the gait table with a
  low-level motor controller (e.g. a PID control
  loop)

Desired angle
DOF i
DOF j
36
Execution of the gait table
37
Gait tables summary

• Pros
• Easy to program
• Cons
• Requires a master module for providing the tables
  and for synchronizing the steps
• Fragile single-point of failure (the master)
• Creates rigid gaits that can not adapt to the
  terrain
• Trajectories are not smooth, risk of damaging
  motors

38
Sine-based controllers
39
Role-based control

• Designed by Stoy and colleages for the CONRO
  project
• Characteristics
• Sine-based
• Hand-coded gaits
• Designed for modular robots with tree structures
  (no loops)
• Each module has the same set of roles
• Modules are interchangeable (i.e. gaits are not
  based on IDs)

40
Role-based control

• Two main algorithms
• The role playing algorithm
• The role selection algorithm

41
Role-based control

• What is a role?
• a role implements a periodic motion, and how it
  relates to neighbor modules
• It is defined by three components
• a cyclic action A(t),
• a period T
• and a set of delays di , which determine how much
  children modules should be delayed

42
Role-based control
Example
Delays
43
Role-playing algorithm

• Role playing algorithm
• Performs the action,
• Sends out synchronization signals to children
  modules
• Resets time (t0) to respect delays with parent
  modules
• Note some failures in the exchange of signals
  with neighbor is not critical because each module
  has its own timer

44
Caterpillar gait
45
Role selection algorithm

• In order to have different gaits, modules need to
  play different roles depending on the robot
  configuration.
• Need for a role selection algorithm
• Selection is based on
• the local configuration (i.e. how neighbors are
  attached)
• which roles the neighbors are playing
• Based on this, a set of rules determine which of
  different possible roles has to be played

46
Role selection algorithm
Example switching between sidewinding and
walking Four different roles are needed One
for sidewinding Sidewinder (sw), Three for
walking spine (sp), east leg (eleg), west leg
(wleg)
47
Role selection algorithm
Example Spine (sp) role
48
Role selection rules
Rules for role selection algorithm (note each
module has them)
49
Role selection algorithm
Explanation of previous figure
50
Role selection algorithm

• In words Change the sidewinder role sw into
• a spine role sp if there is a module connected
  to the east C(e)1 or the west C(w)1
• a west leg role wleg if there is a module
  connected as parent C(p)1 and the connector of
  the parent is west NC(p)w
• a east leg role eleg if there is a module
  connected as parent C(p)1 and the connector of
  the parent is east NC(p)e

51
Role selection algorithm

• In words Change the spine role sp into
• a sidewinder role sw if there is no module
  connected to the east nor the west

52
Role selection algorithm

• In words Change a leg role wleg or eleg into
• a sidewinder role sw if there is a module
  connected as parent and the role of the parent is
  sidewinder sw, OR if there is a module connected
  to the north C(n)1
• a spine role sp if there is a module connected
  as parent and the role of the parent is spine sp

53
Example of reconfiguration
54
Example of reconfiguration
55
Role-based control summary

• Pros
• Trajectories are smooth (except for the
  resetting)
• Uses local information to generate complete gaits
• Modules are interchangeable (i.e. gaits are not
  based on IDs)
• Robust against communication problems
• Deals with dynamic reconfiguration
• Cons
• The synchronization mechanism (resetting) is
  crude and leads to jumps in the trajectories
• Sine-based control does not allow easy modulation
  by sensory feedback
• Creates rigid gaits that can not adapt to the
  terrain

56
Neural networks
57
Example Karl Sims
Co-evolution of body structure and neural network
controllers in simulated modular robots Fitness
function distance covered Note the controllers
are not really neural networks, rather a set of
nodes that perform various computations sum,
product, divide,, sin, cos, atan, log,,
oscillate-wave, oscillate-saw,
58
Karl Sims
The morphology and the controller are encoded in
directed graphs with adjustable parameters
59
Karl Sims
Example of a neural controller
60
Karl Sims
Different types of resulting locomotion Movie
61
Neural networks
Many other examples of evolution of artificial
neural networks for locomotion R. Beer, F.
Gruau, J. Kodjabachian, A. Ijspeert Example
62
Neural network control summary

• Pros
• Smooth trajectories
• Allow the integration of sensory feedback
• Distributed control
• Cons
• Difficult to design
• Require (some) computation

63
Central pattern generators
64
Central pattern generators
Idea To implement a locomotion controller as a
system of coupled nonlinear oscillators (like the
CPGs in spinal cords of animals) The gait is
encoded in the limit cycle behavior of the
coupled oscillator system At least one
oscillator per degree of freedom Coupling
connections between oscillators and between
modules determine the global behavior
65
Our project CPG-based control
66
Yamor key characteristics

• One-DOF
• Autonomous each unit has its own battery and
  microprocessor (micro controller and FPGA)
• Wireless bluetooth communication
• Arbitrary connections (strong velcro)

67
YAMOR Examples
Elmar Dittrich and Rico Moeckel
68
Designing CPGs
69
Nonlinear oscillator model
Each unit is controlled by the following
oscillator
Limit cycle
70
Network of oscillators
Diffusive coupling
71
Designing CPGs
xi sent as a set point for a PD controller (servo
motor)
72
To be optimized for each oscillator
4 (or more) parameters per module
73
Different algorithms tested
Genetic algorithm
Particle Swarm opt.
Stochastic
Simulated annealing
Optimization
Heuristic
Powells method
Fitness function distance covered
74
Stochastic optimization
Yvan Bourquin
75
Typical gait
Yvan Bourquin
76
Stochastic optimization results
Yvan Bourquin
77
Online optimization using Powells method

• Multidimensional optimization method which does
  not require gradient computation
• Idea
• use Brents method for unidimensional
  optimization
• Carefully choose direction sets for
  multidimensional optimization
• Numerical Recipes in C, W.H. Press, S.A.
  Teukolsky

78
Unidimensional optimization Brents method
Combination of
Successive bracketing and parabolic
interpolation
79
Powells optimization method
Method for choosing directions for
one-dimensional opt.
80
Yamor online learning of a controller
Param. values
Speed
Time
81
Yamor online learning of a controller
Modifications of parameters without
stopping/resetting the robot
82
Yamor online learning of a controller
Time 0.0, starting from random initialization
Marbach and Ijspeert 2005, ICMA2005
83
Yamor online learning of a controller
Resulting gait after 30 minutes
Marbach and Ijspeert 2005, ICMA2005
84
Yamor online learning of a controller
Another example
Marbach and Ijspeert 2005, ICMA2005
85
Yamor online learning of a controller
The online learning tends to produce solutions
that are as good or even better than those
evolved with a GA
86
Properties of the CPG smooth modulations
87
CPG-based control summary

• Pros
• Smooth trajectories
• Allows the integration of sensory feedback
• Distributed control, easy to synchronize
• Easier to design than neural network controllers
• Fast learning with some algorithms (e.g. Powells
  method)
• Cons
• The control of direction is sometimes not trivial
  to add (this is true for all methods)
• Requires mechanisms to implement multiple gaits
  (e.g. like in role-based)

88
End of this lecture