Holland and Goodman

1
(No Transcript)
2
Holland and Goodman Caltech Banbury 2001
3
When an autonomous embodied system, with a
difficult animal-like mission in a difficult
environment, has a sufficiently high level of
intelligence (i.e. is able to achieve that
mission well), then it may exhibit consciousness,
either as a necessary component for achieving the
mission, or as a by-product.
Holland and Goodman Caltech Banbury 2001
4

Holland and Goodman Caltech Banbury 2001
5
Holland and Goodman Caltech Banbury 2001
6

Holland and Goodman Caltech Banbury 2001
7
Holland and Goodman Caltech Banbury 2001
8
Holland and Goodman Caltech Banbury 2001
9
Holland and Goodman Caltech Banbury 2001
10
Holland and Goodman Caltech Banbury 2001
11
Holland and Goodman Caltech Banbury 2001
12
Holland and Goodman Caltech Banbury 2001
13
Holland and Goodman Caltech Banbury 2001
14
Holland and Goodman Caltech Banbury 2001
15
Holland and Goodman Caltech Banbury 2001
16
Robots
17
A Simple Robot

• The Khepera miniature robot
• Features
• 8 IR sensors which allow it to detect objects
• two independently controlled motors.

18
Webots Khepera Embodied Simulator
Simulators allow faster operation than real
robots particularly if learning involved.
Simlulator complexity is OK for a simple robot
like the Khepera, but for more complex robots,
the simulator may be too complex or not simulate
the real word accurately.
19
A Generic Robot Controller Architecture
HIDDEN
UNITS

Sensory Inputs Including Motors and effectors

Controller outputs to motors and effectors



OUTPUT

UNITS

INPUT


UNITS
Rec
ur
r
ent
STATE

Neural

UNITS
Machine

• The controller of the robot is an artificial
  neural network with recurrent feedback, capable
  of forming internal representations of sensory
  information in the form of a neural state
  machine.
• Sensory inputs (vision, sound, smell, etc) from
  sensors are fed to this structure
• Sensory inputs also include feedback from the
  motors and effectors.
• Controller outputs drive the locomotion and
  manipulators of the robot.
• The neural controller learns to perform a task,
  using neural network and genetic algorithm
  techniques.
• But - the internal model of the controller is
  implicit and therefore hidden from us.

20
Understanding the internal model

HIDDEN

UNITS

• Introduce a second recurrent neural network,
  separate from the first system, which learns the
  inverse relationship between the internal
  activity of the controller and the sensory input
  space

Motor
effector

drive

outputs

OUTPUT

UNITS

INPUT


UNITS
Recurrent
STATE

Neural

UNITS
Machine

Outputs of inverse in same sensory space as
inputs of forward controller
OBSERVE
INVERSE



Recurrent Neural

Machine

• This mechanism will allow us to represent the
  hidden internal state of the controller in terms
  of the sensory inputs that correspond to that
  state.
• Thus we may claim to know something of what the
  robot is thinking.
• We assume that the controller be learned first,
  and that, once this is learned and reasonably
  stable, the inverse can be learned.

21
Simplified Inverse

• In this experiment, we utilize a controller model
  which is much less powerful than the recurrent
  controllers described above, but allows us to
  illustrate the principle, and in particular makes
  inversion of the forward controller extremely
  simple.
• The crucial simplification we make is that the
  controller will learn its representation directly
  in the input space. Thus there is no inverse to
  learn - the internal representation learned by
  the robot is directly visible as an input space
  vector.
• The first phase is to learn or program the
  forward model or robot controller. In this simple
  experiment we program in a simple reactive
  wall-following behavior, rather than learn a
  complex behavior. The robot starts with no
  internal model, and adaptively learns its
  internal representation in an unsupervised manner
  as it performs its wall following behavior.

22
The Learning Algorithm(based on Linaker and
Niklasson 2000 ARAVQ algorithm)

• A 10-dimensional feature space is formed from the
  8 Khepera IR sensor signals plus the 2 motor
  drive signals.
• Clusters feature-vectors by change detection, to
  form prototype feature vector models.
• Unsupervised
• Adds new models based on two criteria
• Novelty Large distance from existing models
• Stability Low variance in buffered history of
  features
• Adapts existing models over time
• We program in a simple wall following behavior
  to act as a teacher.

23
Learning in action
Colors show learned concepts Black right
wall Blue ahead wall Green 45 degree right
wall Red corridor Light Blue outside corner
24
Running with the model

• Switch off the wall follower
• The robot sees features as it moves
• Choose the closest learned model vector at each
  tick
• Use the model vector motor drive values to
  actually drive the motors.

25
Running with the model
Color indicates which is the current bestmodel
feature
26
Run the model in the real robot
27
Invert the motor signals back to sensory signals
to infer an egocentric map of the environment
as seen by the robot.
28
Keeping it Real

• Mapping with the real robot

29
Manipulating the model mentally to make a
decision - planning

• Take the sequence of learned model feature
  vectors and cluster sub sequences into
  higher-level concepts
• For example
• Blue-Green-black Left Corner
• Red Corridor
• Black right wall
• At any instant ask the robot to go to home
• Run the model forwards mentally to decide if it
  is shorter to go ahead or to go back
• Take appropriate action

30
Decision Time
Corridor corner is home Rotate Home is behind
me Flash LEDs Home is ahead of me
31
Inverse Predictor Architecture

• We now allow the inverse to be fed back into the
  controller via the switch
• Thus the controller has an image of its internal
  hidden state or self in the same feature space
  as its real sensory inputs
• Thus it can see what it itself is thinking.
• As before we can also observe what the machine
  is thinking.

32
Consequences of the architecture

• In normal mode - the controller is producing
  motor signals based on the sensory input it
  sees (including motor/effector feedback).
  Normally we expect to see what it is seeing. The
  inverse allows for detecting mismatch between a
  predicted and an actual sensory input thus
  indicating a novel experience, which in turn
  could focus attention and learning in the main
  controller. Noisy, ambiguous, and partial inputs
  can be completed.
• In thinking or planning mode the real world is
  disconnected from the controller input, and the
  mental images being output by the inverse are
  input to the controller instead. Thus sequences
  of planned action towards a goal can take place
  in mental space, and executed as action. Note
  that by switching between normal mode and
  thinking mode in some way, we can emulate the
  robot doing both reactive control and thinking at
  the same (multiplexed really) time. That is, like
  humans do when driving a car on automatic while
  thinking of something else.
• In sleeping mode we shut off the sensory input
  and allow noise to be input. Then the inverse
  will output mental images, which themselves can
  be fed back into the input (because they have the
  same representation) producing a complex series
  of imagined mental images or dreams. Note
  that we can use this sleeping mode to actually
  learn (or at least update) the inverse. The
  input noise vector is a sensory input vector
  like any other (whether it is structured
  accordingly or not), thus the inverse should be
  able to output this vector like any other from
  the state and motor signals. Thus we can use the
  error to update the inverse.
• If we do not disconnect the motors during
  dreaming we will have sleepwalking or
  twitching. If we assume that the controller is
  continually learning, then the inverse must be
  continually updated. If they get too much out of
  synchronization we could get irrational sequences
  in thinking or worse in execution mode - an
  analog of madness.

33
Wheres the Consciousness?

• Not there yet
• More complex robots
• More complex environments
• More complex architecture

SONY DREAM ROBOT
34
Increasing complexity
Environment Agent Fixed environment Movabl
e body Moving objects More sensors Movable
objects Effectors Objects with different
values Articulated body Other agents
prey Metabolic state Other agents
predators Acquired skills Other agents
competitors Tools Other agents
collaborators Imitative learning Other agents
mates Language Etc Etc
35
Multi-stage planning
At each step - what actions could it take? -
what actions should it take? - what actions
would it take? The planning system needs - a
good and current model of the world - a good and
current model of the agents abilities,
expressible in terms of their effects on the
model world - an associated executive system to
use the information generated by the
planning system
36
A framework?
Updates
Self Model
To executive
Updates
Environment Model
37
Speculation
There may be something it is like to be such a
self-model linked to such a world model in a
robot with a mission