Neurological Modeling

1
Neurological Modeling CooperationAutomatic
Acquisition of Triggered Reactions, a
Physiological Approach

• Cooperative Control of Distributed Autonomous
  Vehicles in Adversarial Environments2001 MURI
  UCLA, CalTech, Cornell, MIT
• Mao/Massaquoi/Dahleh/Feron
• May 14, 2001
• UCLA

2
Basic Route

• Impression
• Useful, complex group behavior is based on a
  combination of relatively simple, perhaps
  identical Triggered Programmed-Reactions existing
  within a collection of nominally autonomous
  agents
• Hypothesis
• The physiological basis for general behavioral
  TPRs is the same as that for TPRs used for
  elemental body movement control/postural
  regulation

3
Examples
4
Observations

• Both postural defense, herd containment and
  dancing via triggered reactions require
• Assessment of continuous (though perhaps only
  piecewise, intermittent) sensory information
• Selection of stereotyped movements (motion
  primitives) that are appropriately scaled and
  timed to project beyond the anticipated motion of
  the target
• Learning based on goals and reinforcement as
  dictated by environment and higher control levels

results
Goals, constraints, Reward/failure
Selection, Timing, Scaling
Assessment, Prediction
Multichannel sensory information
Partially pre-programmed behavior
5
Observations (ct'd)

• Presumably, scaling, timing and selection also
  automatically learn to take into account
  supportive or obstructive features of
  environment, e.g.
• Traction/motion characteristics of floor
• responsiveness of target
• Or
• Presence or absence of multiple actuators (e.g.
  ankles and hips when falling forward, hips only
  when falling backward)
• Presence or absence of other herders on one side
  vs. another
• General sensitivity to environment may be
  physiological substrate for functionally useful
  group-aware behavior

6
Modeling Assumptions

• Natural motor control system can be represented
  as a hierarchy consisting of a high level,
  largely conscious, discrete state-machine-type
  computer and a low/intermediate level, largely
  unconscious, continuous signal processing
  controller.
• In between are structures enabling the
  development of flexible, simple, semi-conscious
  motor programs (behaviors) that address/adhere
  to the goals and constraints provided by the high
  level computer

7
Natural Sensorimotor Control

• That is, our Interest
• Understand Control, Assessment and Learning at
  the interface between higher and intermediate/low
  functional levels of natural sensorimotor system

Discrete Behavior Control, Assessment
Adaptation (conscious/preconscious?)
MURI
Continuous Action Control, Assessment
Adaptation (subconscious?)
Action Production
Action Monitoring
Environment
8
Natural Sensorimotor Control

• More specifically,
• Natural Sensorimotor control Hierarchy
• High level Goals (conscious)
• e.g. win point vs. conserve energy
• Strategic Planning/Decisions (conscious)
• e.g. return to right rear baseline
• Tactical Objectives (preconscious/overlearned?)
• e.g. contact ball with racket face having
  particular orientation and velocity
• Tactical Assessment/Planning/Decisions
  (preconscious/overlearned?/development of
  motor program)
• assess/predict ball trajectory, spin, body
  location in court
• use forehand, assume particular posture,
  generate specific trajectory

MURI
9
Natural Sensorimotor Control

• Natural Control Hierarchy (contd)
• Action (force, position) generation on-line
  control (subconscious)
• Action (continuous trajectory) improvement
  (optimization?) with practice (subconscious motor
  learning)
• Behavior (discrete program, trajectory)
  improvement (optimization?) with practice
  (conscious--gt preconscious tactical motor
  learning, motor programming)
• Behavior improvement (optimization?) with
  practice (conscious strategic motor learning,
  gamesmanship)

MURI
10
Natural Sensorimotor Control

• Natural Sensorimotor Control System

SENS
MTR
(parietal) ASSOC
ST
MT
(frontal) ASSOC
BG
Interface between high and intermediate/low contro
l levels involves sensorimotor and association
cortices (especially frontal) and the Basal
Ganglia. These link automatic behavior and
reward. Cerebellum likely contributes optimization
Cbl
11
Human motor control principal information flow
(adapted from V. Brooks, 1986)
highest level PLANS (strategy)
middle level (high and intermediate) PROGRA
MS (tactics)
lower level ACTION (force, velocity)
Putamen GP
Caudate GP
Motor Servo
Brainstem or Spinal Cord Segment
Frontal Parietal Assoc Ctx
Mtr Ctx
Neural signals ------------------
executive sensory consciousness gradient
Im Ant Cbl
L Ant Cbl
Muscle tendon, Joints, skin
M. Cbl
Flocc Cbl
Body Force/ Motion
Vestib
Visual
12
MURI Goals
MURI to specifically study Programming of
Triggered Reaction Loops
high level PROGRAMS (discrete control)
(tacticstrajectories, cues)
highest level PLANS, ALGORITHMS (free assoc,
strategy)
intermediate level CONTROL (continuous
control) (stability, tracking, stiffness, scaling,
movement time)
Putamen GP, SN (Basal Ganglia)
Caudate GP (Basal Ganglia)
Motor Servo (proprioceptive)
Frontal Parietal Assoc Ctx
Frontal Parietal Peri- Sensorimotor Ctxs
Mtr Ctx
Im Ant Cbl
L Post Cerebellum
L Ant Cbl
TPR Loop Circuitry
M. Cbl
Flocc Cbl
13

• Proposed MURI project (Year 1)
• Acquisition of triggered motor reactions

Video monitor showing virtual targets and environ
ment
Robot arm Implementing virtual
targets and environment
14

• Proposed MURI project questions with respect to
  physiological structures known or suspected to be
  involved in TPRs
• (Year 1)
• What are the motion primitives?
• How are they generated, scaled, timed, triggered?
• What and how is continuous sensory information
  used?
• How is prediction performed evidence for
  internal models?
• How is reinforcement/suppression mediated?
• What is the statistical nature of the learning
  and programming?

15

• Background studies and resources
• Existing models for intermediate and low/level
  motor control based on cerebellar and
  sensorimotor cortical physiology
• Robot arm laboratory
• Access to human subjects including those with
  diseases of the basal ganglia and cerebellum

16
Beyond Year 1

• Useful, complex group behavior may emerge from
  relatively simple, perhaps identical Triggered
  Programmed-Reactions existing within a collection
  of nominally autonomous agents
• Link to emergent group behavior possible via
  experimental observations / prior and similar
  approaches in Air Traffic Control (eg Mao, Feron
  and Bilimoria, IEEE ITS, 06/01)