Biological Foundations of the Reactive Paradigm

1
Biological Foundations of the Reactive Paradigm

• Describe the three levels in a Computational
  Theory.
• Explain in one or two sentences each of the
  following terms reflexes, taxes, fixed-action
  patterns, schema, affordance.
• Be able to write pseudo-code of an animals
  behaviors in terms of innate releasing
  mechanisms, identifying the releasers for the
  behavior.
• Given a description of an animal's sensing
  abilities, its task, and environment, identify an
  affordance for each behavior.
• Given a description of an animal's sensing
  abilities, its task, and environment, define a
  set of behaviors using schema theory to
  accomplish the task.

Review Why? -comp. theory IRM Perception -Summary
2
Robots In the Hierarchical Paradigm
3
Timeline of Influences
Braitenbergs Vehicles
Arkins Schemas
Neisser
Arbibs Schemas
J.J. Gibson
Payton
Brooks Insects
Middlestat
Marrs Computational Theory
Tinbergen Lorenz von Frisch
1970
1980
1990
4
Marrs Computational Theory
5
Level 1 Existence Proof
Goal how to make line drawings of
objects? people can do this by age 10, computers
should
6
Level 2 Inputs, Outputs, Transforms
drawing
light
lines (edges)
drawing
light
retina (gradient)
7
Level 3 Implementation
-
-
-
-
-

-
-
-
Center Surround Cell in retinal ganglion
8
Class Discussion

• Give three examples of how biology has informed
  modern technology?
• ex. Wright Brothers- control flaps on airplane
  wings

9
Behavior Definition (graphical)
BEHAVIOR
Pattern of Motor Actions
Sensory Input
10
Types of Behaviors

• Reflexive
• stimulus-response, often abbreviated S-R
• Reactive
• learned or muscle memory
• Conscious
• deliberately stringing together

WARNING Overloaded terms Roboticists often use
reactive behavior to mean purely reflexive, And
refer to reactive behaviors as skills
11
Ethology Coordination and Control of Behaviors

• Nobel 1973 in physiology or medicine
• von Frisch
• Lorenz
• Tinbergen

INNATE RELEASING MECHANISMS
www.nobel.se
12
Arctic Terns

• Arctic terns live in Arctic (black, white, gray
  environment, some grass) but adults have a red
  spot on beak
• When hungry, baby pecks at parents beak, who
  regurgitates food for baby to eat
• How does it know its parent?
• It doesnt, it just goes for the largest red spot
  in its field of view (e.g., ethology grad student
  with construction paper)
• Only red thing should be an adult tern
• Closer large red

13
behavior template
BEHAVIOR
Pattern of Motor Actions
Sensory Input
14
the feeding behavior
Feeding BEHAVIOR
RED
PECK AT RED
Pattern of Motor Actions
Sensory Input
15
the releaser template
Sensory input and/or internal state
16
the feeding releaser
RED HUNGRY
Releaser
sensory input
internal state
present?
N
/dev/null
Y
Feeding BEHAVIOR
RED
PECK AT RED
17
Innate Releasing Mechanisms
Sensory input and/or internal state
BEHAVIOR
Pattern of Motor Actions
Sensory Input
18
Example Hide Behavior

• Programmed in C, ltlt 100 LOC
• shows
• taxis (oriented relative to light, wall, niche)
• fixed action pattern (persisted after light was
  off)
• reflexive (stimulus, response)
• impliciting sequencing
• use of internal state

19
Example Cockroach Hide

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out
• even if the lights are turned back off earlier

20
Reflexive Behaviors S-R

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out
• even if the lights are turned back off earlier

21
Fixed Pattern Actions

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out
• even if the lights are turned back off earlier

22
Exhibits Taxis

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out
• even if the lights are turned back off earlier

to light
to wall
to niche
23
Class Exercise

• Draw flowchart of how this works

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out
• even if the lights are turned back off earlier

24
Break into Behaviors
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

Follow- wall
hide
25
Find Releasers
Y
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

Ooops, need internal state Scared
Follow- wall
SCARED SURROUNDED
present?
N
hide
26
Internal State Set
Y
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

SCARED
BLOCKED SCARED
present?
N
Follow- wall
SCARED SURROUNDED
present?
N
hide
27
Action
steer 360, drive forward
Y
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

SCARED
BLOCKED SCARED
present?
N
steer F(dist to wall) drive forward const.
Follow- wall
SCARED SURROUNDED
present?
steer F(dist to wall) drive forward const. stop
N
hide
28
Sensory Input
IR
steer 360, drive forward
Y
encoders
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

SCARED
BLOCKED SCARED
IR
present?
N
steer F(dist to wall) drive forward const.
IR
Follow- wall
SCARED SURROUNDED
IR
present?
steer F(dist to wall) drive forward const. stop
N
IR
hide
29
How Do You Link Them?
IR
steer 360, drive forward
Y
encoders
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

SCARED
BLOCKED SCARED
IR
present?
N
steer F(dist to wall) drive forward const.
IR
Follow- wall
SCARED SURROUNDED
IR
present?
steer F(dist to wall) drive forward const. stop
N
IR
hide
30
AnalogyIRMs work on THREADS,not sequential
processing!

• Very simple modules
• Nice building blocks since not directly linked
• If one module (part of brain) fails, what happens?

31
What happens when theres a conflict from
concurrent behaviors?

• Equilbrium
• Feeding squirrels-feed, flee hesitate in-between
• Dominance
• Sleepy, hungry either sleep or eat
• Cancellation
• Sticklebacks defend, attack build a nest

?
32
Perception

• Two uses of perception (can be the same percept)
• Release a behavior
• Guide a behavior
• Action-oriented perception (Neisser)
• Planning is not needed to act
• Perception is selective

33
Gibsons Ecological Approach

• Acting and sensing co-evolved as agent survived
  in a particular environment. The environment
  affords the agent what it needs to survive.
• The perception needed to release or guide the
  right action is directly in the environment,
  not inferred or memorized
• Ex. Red on Artic Terns food
• Ex. Sound of filling containerfull
• Percepts are called affordances or said to be
  obtained through direct perception

34
Gibsonian Affordances

• How do you know youre going fast in a car? Or in
  a space movie?
• How do animals know when to mate?
• How do mosquitoes know to bite in the most tender
  areas?
• What should you do when you think youre being
  stalked by a mountain lion?
• Whats your favorite fishing lure?

35
Sittability
36
But does this really hold for everything?

• my car versus your car?
• Difference is
• where I parked it (memory)
• Semantic meaning (cars arent generic like nuts
  to a squirrel)
• Neissers Two Systems
• Direct Perception older, behavioral
• Recognition evolved later, deliberative

37
Review Questions

• What are the levels of a computational theory?
• Existence proof, inputs-outputs-transformations,
  implementation
• What is a behavior?
• A behavior is a mapping of sensory inputs to a
  pattern of motor actions
• Is sequencing normally implicit or explicit in
  IRM?
• implicit
• What is an affordance?
• A potentiality in the environment for an action

38
Schema Theory

• schema- is used in cognitive science and ethology
  to refer to a particular organized way of
  perceiving cognitively and responding to a
  complex situation or set of stimuli
• is generic, equivalent to an object in OOP
• schema specific knowledge (local data)
• procedural knowledge (methods)
• schema instantation is specific to a situation,
  equivalent to an instance in OOP
• a behavior is a schema, consists of
• perceptual schema
• motor schema

39
Behavioral Schema
alternative PS, MS sequencing logic for reactive
skills (judgment value function)
action, intensity
percept, gain
Motor Schema (MS)
Perceptual Schema (PS)
Reflexive behaviors usually just have methods,
not data
40
Ex. Fly Snapping Behavior IRM
Releaser small moving dark blob
present?
N
/dev/null
Y
snap, 100
x,y,z, 100
snap(blob)
track(blob)
41
Schema Instantiation (SI)
Releaser small moving dark blob
present?
N
/dev/null
Y
snap, 100
x,y,z, 100
snap(blob)
track(blob)
42
Schema/Schema Instantiation
43
Advantages

• modular
• can assemble new behaviors from existing schemas
• learning by experimentation
• can substitute alternatives
• reroute nerves

44
Instantiation for each eye
Releaser small moving dark blob
present?
Left eye
N
/dev/null
Y
snap, 100
x,y,z, 100
snap(blob)
track(blob)
Releaser small moving dark blob
present?
N
/dev/null
Right eye
Y
snap, 100
x,y,z, 100
snap(blob)
track(blob)
45
Wheres the MS and PS?
IR
steer 180, drive forward
Y
encoders
Flee

• light goes on, the cockroach turns and runs
• when it gets to a wall, it follows it
• when it finds a hiding place (thigmotrophic),
  goes in and faces outward
• waits until not scared, then comes out

SCARED
BLOCKED SCARED
IR
present?
N
steer F(dist to wall) drive forward const.
IR
Follow- wall
SCARED SURROUNDED
IR
present?
steer F(dist to wall) drive forward const. stop
N
IR
hide
46
General Principles

• All animals possess a set of behaviors
• Releasers for these behaviors rely on both
  internal state and external stimulus
• Perception is filtered perceive what is relevant
  to the task
• Some behaviors and associated perception do not
  require explicit knowledge representation (e.g.,
  rely on affordances)

47
Silicon v. Carbon

• Individual robots must survive, not species
• detection of non-productive behaviors
• graceful degradation
• Must be able to predict emergent behaviors
• Not clear how to learn quickly
• Robots need more alternative perceptual schemas
  since poorer understanding of the environment

48
Unresolved Issues

• How to resolve conflicts?
• behavioral arbitration/combination
• When is explicit representations, memory needed?
• How to set up or learn new sequences of behaviors
• What are the affordances for a particular ecology?

49
Take Home Thoughts

• Ideas bubbling up for robotics
• Maybe programming in terms of behaviors is better
  than STRIPS or trying to set up a complex
  hierarchy
• Intelligence has something to do with agents
  ecological niche its abilities, its tasks
  (survival), and environment
• Perception is going to be critical because it
  releases and guides actions
• IRMs, Schemas are nice ways to start thinking
  about the computational structure of programming
  a robot

50
Review Questions

• Think about the robots at the WTC. What are
  affordances of victims?
• color, motion, sound, heat
• Can schema theory represent behaviors in both
  biological and computational systems?
• yes
• A behavior schema is composed of at least the
  following
• motor schema and a perceptual schema
• What is an example of behavior-specific
  knowledge?
• sequencing in a skill, alternative PS or MS

51
(No Transcript)
52
Inhibition
while (TRUE) predator sensePredator()
//has a time delay if (predatorPRESENT) //as
long as predator persists flee() else
food senseFood() hungry
checkStateHunger() ...
Could also be done as an interrupt