Issues in Evolutionary Robotics

1
Issues in Evolutionary Robotics

• Harvey, Husbands, Cliff (1992)

2
Introduction

• evolutionary approach to the design of robots
  will supercede design by hand?
• exploration using an extended genetic algorithm
  to evolve control architectures
• application based on a simulated version of a
  physical robot
• viable in highly simplified simulated worlds
  also viable in the complexity of the real world?
• evolution in the form of a Behavioural Language
  vs. artificial neural networks?
• current simulations are not naïve
• based on observations of a real robot
• attempt to model the physics of its interactions
  with the world

3
Difficulties

• problems with traditional approaches implicit
  assumption of functional decomposition
• that perception, planning and action can be
  analysed independently of each other
• ? leads to fragile and computationally very
  expensive methods!
• vs. behavioural decomposition (e.g. MIT)
• analyse independent behaviours
• wire in each behaviour all the way from sensor
  input to motor output
• simple behaviours first, more complex behaviours
  on separate layers
• ?
• can suppress/inhibit earlier layers
• BUT to foresee all possible interactions with the
  environment between separate layers of the
  robot ? inherently explosive complexity!
• ? places limits on real robots doing useful
  things!

4
Evolving robots instead

• if some objective fitness function can be derived
  for any given architecture
• ? possibility of automatic evolution of the
  architecture without explicit design (e.g. in
  natural selection)
• ?
• Genetic Algorithms (GAs) use ideas borrowed from
  evolution in order to solve problems
  in highly complex search spaces
• ?
• may provide a solution in this case?
• artificial evolution approach
• maintains a population of viable genotypes
  (chromosomes), coding for cognitive architectures
• interbreed and mutate according to selection
  pressure ? task-oriented evaluation function
• i.e. the better a robot performs in its task,
  the more evolutionarily favoured is its cognitive
  architecture
• ? leads to a gradual emergence of a suitable
  system

5
continued...

• benefits of artificial evolution
• no need for any assumptions about how to achieve
  particular behaviours
• (as long as the behaviour is directly or
  implicitly included in the evaluation function)
• no need to commit to an exclusively behavioural
  decomposition
• artificial evolution allows either type of
  decomposition ? will determine which, if either,
  should characterise the cognitive architecture

6
An incremental, species approach

• animat simulated animal or autonomous
  robot
• animals are not solutions to problems, they
  merely represent species adaptations that solve
  particular problems in the short-term
• ? GAs based on evolutionary ideas should search
  the space of possible adaptations of the existing
  animat (not through the complete space of
  animats)
• ? evolution cannot choose from any adaptation
  ever, only the ones possible for the present
  organism!
• this method implies that the evolved population
  is always a genetically-converging species
• increases in genotype length (associated with
  increases in complexity) can only happen very
  gradually
• similar to other incremental approaches, except
  that evolution is used instead of design

7
Using simulations

• artificial evolution requires that members of the
  population be evaluated over many generations
• BUT this would take far too long in the real
  world!
• ? most evaluations should be done in simulation
• (though doubts about this approachs long-term
  viability)
• must keep simulations as close to reality as
  possible
• calibrate simulation at regular intervals by
  testing architectures in real robots
• base simulations of inputs to sensors and
  reactions of actuators on empirical data
• take into account noise at all levels ? use low
  resolution sensing
• use a range of unstructured, dynamic environments
  in the simulation
• ? the cognitive architecture evolved is more
  likely to be robust
• (more like real worlds)
• use of adaptive, noise-tolerant units (e.g.
  neural nets)
• ? 100 accuracy not required as discrepancies can
  be treated as noise, and the system will adapt
  and cope accordingly

8
What to evolve?

• at least 3 useful ways to implement the control
  system of an animat
• 1. an explicit control program in some high-level
  language
• 2. a mathematical expression mapping inputs to
  outputs
• 3. a blue-print for a processing structure, a
  network of simple processing elements

9
1. High-level programs

• if the language supports partial recursion,
  programs to be evaluated may never halt
• ? need to impose some arbitrary time-out
• seems reasonable to use a program, without
  partial recursion, in which genotype changes are
  restricted to small steps
• BUT
• this approach treats the brain as a
  computational system ? produces a set of motor
  outputs for any given set of sensor inputs
• vs. alternative view ? agents as dynamical
  systems, perturbed by interactions with a
  dynamical environment
• primitives manipulated by the evolutionary
  process should be at the lowest level possible,
  in contrast with the use of higher level
  languages
• (supported by simulation results)
• ? any high-level semantic groupings necessarily
  restrict the possibilities available to the
  evolutionary process (incorporates human
  designers prejudices) ? leads to a much more
  coarse-grained fitness landscape
• ( is harder to justify injecting noise into
  anything other than the lowest levels)

10
2. Polynomial transfer functions

• close relationships with artificial neural
  networks
• e.g. input-output associations of most neural
  nets can be arbitrarily closely approximated by
  a polynomial function and vice versa
• BUT
• even for modest numbers of inputs/outputs ? most
  useful transfer function may be a highly complex
  non-linear expression with many terms
• ? search space is potentially very large
• simulation suggests that the search space (except
  for low dimensions) lacks structure ? GA
  degenerates into random search
• real world robustness will demand some degree of
  adaptation (requires auxiliary systems
  identification algorithms, seriously complicating
  matters)
• OR an expression complicated enough to cover a
  wide enough range of situations (see problems
  above)

11
3. Neural nets

• approaches take various forms, but all include
  that the
• basic architecture has been defined with some
  parameters left as variables
• GA is used to tweak these parameters to optimal
  values
• evolvability of connectionist networks is clearly
  established
• can concisely specify on the genotype of
  sub-networks or modules which may be repeatedly
  used (though require a mechanism to interpret
  such specifications several times)
• artificial neural networks obviously allow for
  adaptation
• parallel nature ? enables very fast
  implementation in the appropriate hardware
• (vs. serial interpretation of a behaviour
  language)

12
Type of connectionist network

• good reasons why a generalised form of
  connectionist network could be appropriate
• 3 basic principles
• 1. brain as a physical system, with a physical
  volume and a finite number of input and output
  points on its surface
• 2. interactions within brain should be mediated
  by physical signals
• ? travel with finite velocities through its
  volume from inputs to outputs
• 3. excluding the lower limit of an undecomposable
  node, these principles apply to any physical
  subvolume of the whole brain
• (linked to incremental development of the whole
  by alterations/additions over evolutionary
  timescales)
• product is a network model with internal nodes as
  the lower limit, and signals between these and
  inputs and outputs taking finite times
• can be arbitrarily recurrent

13
Details of the network

• simplest assumptions (of a standard connectionist
  network)
• input signals weighted by a scalar quantity
• all output signals are identical when they leave
  the node (calculated from the weighted sum of the
  inputs)
• pass the weighted sum through a sigmoid or
  thresholding function
• ? produces non-linear behaviour
• timelags between nodes need to be specified
• difficult to analyse (compared with standard
  feedforward networks)
• BUT
• dont need to analyse how it works!
• ? only need to be able to assess how good the
  behaviour it elicits is!
• (with the internal complexity of the brain
  being dependent on the history of interactions
  with its world)

14
Timing

• the robot will have timing circuitry to
  synchronise sensing, control and motor activities
• ?
• as long as they operate using discrete time
  intervals, even complex recurrent networks can be
  handled straightforwardly
• (asynchronous continuous time networks are more
  difficult but pose no significant problems)
• arbitrarily complex polynomial transfer
  functions are certainly more difficult to handle,
  as are potentially non-halting high-level
  programs

15
Sensors

• autonomous robots require sensors in order to
  navigate
• exteroreceptors detect stimuli external to an
  animal e.g. light
• interoreceptors detect stimuli that arise
  inside the animal e.g. blood pressure
• successful navigation depends on finding a
  satisfactory combination of exteroreceptors and
  interoreceptors
• exteroreceptors
• proximal only provide reliable data for the
  immediate surroundings e.g. bumpers
• ? forced to employ primitive strategies e.g.
  wall following
• ? loss of contact results in sensor-blindness
• ? degradation/loss of navigation ability
• distal provide reliable data for surroundings
  further away e.g. vision
• ? necessary for more sophisticated navigation
  competences

16
continued

• animate vision the approach of mounting
  computer vision systems on mobile robotic
  platforms
• provides a rich source of information concerning
  an agents external environment
• factors to be considered
• discretization of the sampling of the optic array
  (how many pixels in the images, the geometry of
  these images etc.)
• angular extent of the vision systems field of
  view (360 etc.)
• visual angular resolution of the optics (uniform
  vs. nonuniform/foveal)
• many animals (e.g. insects) use low-resolution,
  low-bandwidth vision as a primary source
• ? worth exploring within this context of
  evolutionary robotics
• need to ensure that the simulated visual systems
  correspond in a useful manner to the physical
  visual systems that the robot will be equipped
  with
• ?
• possible in principle, but computational demands
  soon become considerable
• require availability of necessary processing
  hardware

17
continued again

• in order to equip a physical robot with a vision
  system that has variable capabilities, the
  mounted camera must offer performance at the
  upper limits of what is seen as necessary
• worse visual capabilities can then be produced
  under genetic control e.g. using subsampling of
  the image
• BUT difficult to see what is necessary for the
  robot until experimentation has been carried out
• ? require some iteration between simulation work
  and the building of real robots

18
Experimentation

• simulation of real robot assembly in Sussex (UK)
• wheels as outputs, whisker and bumper touch
  sensors as inputs
• low-resolution, insect-type visual system
• tasks leading towards the goal of navigation
  using learnt visual landmarks
• Current Paper exploring the methodology using
  careful simulations
• robot in simulation (see Figure 1)
• cylindrical, with two wheels towards the front
  and a trailing rear castor
• wheels have independent drives, with five
  settings (full speed forward etc.)
• aim to evolve neural-style networks to control
  the robot in a variety of environments

19
The neural networks used

• continuous, real-valued networks with
  unrestricted connections and time delays between
  units
• ? can support a range of behaviours and is
  highly adaptive (without requiring Hebbian-type
  learning)
• 1 input node for each sensor
• 2 output units for each motor (providing a range
  of -1.0 to 1.0 for each motor)
• each unit is a noisy linear threshold device ?
  simulates naturally occurring noise in the
  physical world (see Figure 2 for input-output
  relationship)
• 2 types of connections
• normal weighted link joining the output of one
  unit to the input of another
• veto special, infinitely inhibitory connection
  between two units (above a certain threshold) ?
  crude but effective model of natural phenomena
• some number of hidden units (not prespecified
  genotypes vary in length)

20
The genetic encoding

• specifies properties of the units and
  connections, and connection-types emanating from
  them
• genotype interpreted sequentially (see Figure 3
  for details)
• method makes sure that offspring produced by
  crossover are always legal
• process allows for any number of internal nodes

21
Simulating movement

• the continuous nature of the system was modelled
  by using a fine-time-slice simulation
• at each time step, the sensor readings are fed
  into the neural network
• simulate continuous nature of network by running
  it and then converting the outputs to motor
  signals
• (calculate new position of robot using kinematic
  equations, with noise injected into the
  calculations)
• collisions are handled as accurately as possible
  (using observations of the real system)
• ? not a perfect simulation, but realistic enough
  to produce useful results!

22
The experiments

• each experiment was run for 50 generations
• population size of 40
• crossover rate of 1.0
• mutation rate of 1 bit per genotype
• experiment in which a control network was evolved
  using an evaluation function which encouraged
  wandering in a cluttered environment (see Figure
  5)
• scored on how far away from its starting position
  it moved in a fixed time period
• see results in Figure 4
• found that more and more robust networks began to
  appear, with very good abilities for this simple,
  specific task

23
continued

• second experiment in which a control network was
  evolved using an evaluation function which
  measured the area of the enclosed polygon formed
  by the robots path over a finite time period
  (see Figure 6)
• robot started at random locations with a random
  orientation
• see results in Figure 7
• found that by taking the fitness to be the
  average of the small number of runs in the
  evaluation set, the robots produce a better
  average performance but a very poor, noisy worst
  performance
• evaluating from the worst pushes the worst and
  average closer together, providing a more robust
  solution
• see example of network in Figure 8 (looks messy
  as there is no term in the evaluation functions
  that penalises unnecessary links)
• ? experiments have clearly been successful!

24
Conclusion

• no reason to think that humans are good at
  designing systems involving many emergent
  interactions between many constituent parts (e.g.
  robust control systems for robots)
• artificial evolution seems a good way forward (as
  this paper demonstrates)
• results from realistic simulation experiments
  support the claim that an incremental artificial
  evolution is a viable methodology

25
Other information

• University of Sussex - www.sussex.ac.uk
• Informatics Centre - www.sussex.ac.uk/informatics
• To e-mail one of the authors - inmanh_at_sussex.ac.uk
  (my tutor next year!)
• Good introductory book Ive read
• An Introduction to Genetic Algorithms by
  Melanie Mitchell