Conclusions

1
Towards Biological Limbic System Models as Basic
Deliberative Architectures
Derek Harter, Dept of Computer Science and
Information Systems, Texas AM University
Commerce, TX Robert Kozma, Dept of Computer
Science, University of Memphis, TN Walter J.
Freeman, Dept of Molecular and Cell Biology,
University of California at Berkeley, CA
Computational Neurodynamics Laboratory
Experiments and Results
Conclusions
Introduction

• Our work shows that aperiodic dynamics can be
  used in autonomous agents to form perceptual
  categories and other long-term memories.
• The intrinsic aperiodic dynamics of the K-III are
  shaped through experience in our autonomous
  agents by learning mechanisms, such as hebbian
  modification and habituation.
• Different chaotic attractors come to represent
  the meanings of salient stimuli to the organism.
• Intrinsic aperiodic dynamics appear to play an
  important role in neurological mechanisms of
  biological organisms
• One advantage is that it may be much quicker to
  shape attractor basins from baseline aperioidc
  background activity. Therefore memory formation
  is fast and occurs with only one or a few
  exposures to stimuli.
• Aperiodic dynamics aid in the recognition process
  as well. Chaotic activity is not noise, it can
  quickly converge on formed attractors.
• The evolution of the primordial limbic system
  marked the formation of long-term memory
  mechanisms in biological brains.
• Introduced the first forms of deliberative
  behavior, where past experiences and memory are
  brought to bear on current problems (for example
  in order to successfully navigate a learned
  environment).
• Yet another level of plasticity in behavior
  formation, such behavior goes beyond learning to
  chain sensory/motor stimuli together, to learning
  conditioned responses based on the ontogenetic
  experience of the organism with the environment.
• Intelligent behavior is characterized by the
  flexible and creative pursuit of endogenously
  defined goals.
• One characterization of intentional behavior is
  that it is an act of observation through time and
  space, by which information is sought for the
  guidance of future action.
• Intent comprises the endogenous initiation,
  construction, and direction of behavior into the
  world.
• What we think of as deliberative behavior begins
  with this process, where ontogenetic learning and
  memory combine with endogenous needs to drive
  behavior in a directed and rational manner.
• The K-IV architecture proposes that
• Intrinsic aperiodic dynamics are important to the
  formation and recognition of objects/events/catego
  ries/memories in neuronal spatio-temporal
  mechanisms.
• Intentional and therefore deliberative actions
  were first made possible by primordial long-term
  hippocampal memory systems.
• A bottom-up understanding of the dynamics of
  neuronal populations and their contribution to
  mechanisms such as memory and perception may lead
  to a better understanding of the neurological
  basis of intentional and deliberative behavior.

Appetitive / Aversive Experiment (Harter Kozma
2004b)
Cognitive Map Formation Simulation (Harter
Kozma 2004a)
While bottom-up approaches to studying cognition
have proved insightful in many ways, top-down
approaches are still better at explaining
deliberative cognitive processes. Deliberative
actions are those that go beyond simple
sensory-motor loops and seem to require some type
of internal model, map or logical reasoning.
Examples of deliberative actions include planning
a route to navigate to a goal or performing a
chain of logical inference to determine a likely
course of action. Bottom-up approaches such as
Walters tortoise (1951) and Braitenbergs
vehicles (1984) are excellent models of how
simple sensory-motor loops can combine to produce
complex intentional behavior. Such behaviors are
still mainly of the tropic type (e.g.
phototropic, chemotropic), which rely on
detecting and following some type of perceptual
gradient in the environment. Recently, models
such as Brooks (1990) subsumption architecture
have shown us how collections of behavior
patterns can combine in relatively flexible
chains, in an emergent manner, to produce more
complex behaviors. Tropic behaviors are present
in even the simplest of single celled organisms,
while the complex collection, chaining and
combining of such sensory-motor behavior patterns
first appear with insects and simple vertebrates
like fish. Deliberative actions appear to require
the development of more long-term memory
mechanisms that allow for the storage of past
experiences and for these experiences to be
brought to bear on the current situation (Freeman
2001). Evolutionarily, the development of the
limbic system in simple vertebrates, such as
amphibians, marks the first appearance of
primitive hippocampal structures (Kozma, Freeman,
Erdi, 2003). The hippocampus plays the role of
forming and recalling longer-term representations
of experiences. It is known to participate in
the formation of episodic memory, logical
reasoning and cognitive maps (Arbib, Érdi
Szentágothai, 1997). Building deliberative
systems in a bottom-up whole-system approach
would therefore potentially benefit from a more
complete understanding of the biological limbic
system and how its dynamics contribute to the
formation and execution of deliberative behavior.

• Task learn edible / poisonous distinction using
  KA-III
• Environment 6 food sources, 3 edible 3 poisonous
• Results categories formed in KA-III PC layer,
  trigger avoidance behavior before reaching
  poisonous food source.

• Task form representation of environmental
  locations using a KA-III
• Environment 8 locations, distance and direction
  information to 4 landmarks
• Results 8 distinct aperiodic attractors form in
  layers of the KA-III
• Right middle, contour maps of 500ms of
  spatio-temporal activity in the CA1 layer
• Cluster analysis, right bottom, confirms
  attractors formed that correspond to the 8
  locations.

Environment Key
Edible food source
1
Poisonous food source
1
Architecture used in the cognitive map formation
simulation. The CA1,CA2 and CA3 layers form a
KA-III. CA1 and CA3 are an 8x8 array of KA-II
units.
Limbic System Architecture
K-Models
The K-IV architecture is a model of what
biologists believe is the simplest neural system
capable of basic intentional and deliberative
actions, the limbic system (Kozma, Freeman
Érdi, 2003 Harter Kozma, submitted). The
purpose of the K-IV is to model a complete
autonomous organism, in a bottom-up manner, to
understand better the neurodynamical mechanisms
involved in intentional and deliberative
behavior. The K-IV uses a neural population
model (called K-sets) to describe the activity of
large populations of neurons (as opposed to
single unit or more abstract ANN models). It is
a highly-recurrent multi-layer model of the
important neurological structures of the basic
limbic system. We have developed pieces of the
K-IV for use as control mechanisms in autonomous
vehicles for exploration and navigation problems
for NASA. We use both continuous and discrete
versions of the K-set neural population models to
develop autonomous agent simulations (Harter
Kozma, submitted).

• 3.5 billion years single-cell entities
• 550 million years fish vertebrates
• 430 million years insects
• 370 million years reptiles
• 330 million years dinosaurs
• 250 million years mammals
• 120 million years primates
• 18 million years great apes
• 2.5 million years man
• 5000 years writing

• Basic Limbic System
• Primitive Hippocampus
• Long-term memory
• Beyond stimulus/response
• Episodic Memory
• Cognitive Maps
• Intentional Behavior Deliberative Actions

K-0 Continuous ODE Neural Population Model
KA-0 Discrete Difference Equation
Transfer Function

• The basic hypothesis captured by the K-IV model,
  shown to the left, is that intrinsic aperiodic
  dynamics, like those observed in some perceptual
  processes, may play important roles in the
  hippocampal memories and value systems of
  biological organisms.
• The biological limbic system, and thus the K-IV
  model, is composed of four basic areas.
  Perceptual areas which are mainly involved in
  determining the identity and type of perceptual
  stimuli. The hippocampus which forms more
  plastic long-term memories and can be thought of
  as involved in orienting the organism within its
  spatio-temporal environment. The midline
  forebrain which receives internal sensations and
  is implicated in the regulation of the organisms
  needs, drives and goals. And motor areas which
  regulate actions of the organism out into the
  environment.
• In the K-IV model, the What, Where and Why
  systems are modeled by K-III hierarchical groups,
  capable of producing intrinsic aperiodic basil
  activity. The following properties are also
  important to the K-IV model

References

• K models are population models that capture the
  dynamics of populations of neurons.
• Continuous models first developed by Freeman (see
  Freeman 1991)
• Discrete model described in (Harter Kozma 2004)
• Hypothesis Intrinsic aperiodic dynamics are
  important to perceptual and memory processes.
• K-II produce oscillatory behavior, damped
  oscillation shown bottom left.
• K-III produce intrinsic aperiodic dynamics, shown
  in bottom middle and right.

K-III
Arbib, M.A., Érdi, P., and Szentágothai, J.
(1997). Neural Organization Structure, Function
and Dynamics. The MIT Press, Cambridge, MA.
Braitenberg, V. (1984). Vehicles Experiments
in Synthetic Psychology. The MIT Press,
Cambridge, MA. Brooks, R.A. (1995). Intelligence
without reason. in Building Agents out of
Autonomous Behavior Systems, L. Steels, ed.
83-121. Freeman, W.J. (2001). The neurodynamics
of intentionality in animal brains may provide a
basis for constructing devices that are capable
of intelligent behavior. NIST Workshop on
Metrics for Intelligence Development of Criteria
for Machine Intelligence, National Institute of
Standards and Technology, Gaithersburg, MD.
Freeman, W.J. (1991). The physiology of
perception. Scientific American, 264(2),
78-85. Harter, D., and Kozma, R. (submitted).
Chaotic neurodynamics for autonomous agents. IEEE
Transactions on Neural Networks. Harter, D. and
Kozma, R. (2004a). Navigation and cognitive map
formation using aperiodic neurodynamics. From
Animals to Animats 8 The Eighth International
Conference on the Simulation of Adaptive
Behavior, 246-254. Los Angeles, CA. Harter, D.
and Kozma, R. (2004b). Aperiodic dynamics for
appetitive/aversive behavior in autonomous
agents. Proceedings of the 2004 IEEE
International Conference on Robotics and
Automation (ICRA), 2142-2152. New Orleans,
LA. Kozma, R., Freeman, W.J. and Érdi, P. (2003).
The KIV model nonlinear spatio-temporal
dynamics of the primordial vertebrate forebrain.
Neurocomputing, 52-54 819-826. Walter, G.
(1951). A machine that learns. Scientific
American, August 60-63.
K-Ie
K-II
E1
E2
K-Ii
I1
I2

• Far from equilibrium, thermodynamic systems
• Mechanisms of self-organization
• competition
• cooperation
• autocatalytic loops
• hierarchy mesh
• Aperiodic dynamics
• Expectation or reafference
• Embodiment
• environment/organism coupling
• Small worlds type divergent-convergent projecting
  connections

Results of measured lyapunov exponent of a KA-III
when varying the projecting weight between the 3
layers from 0 to 100 of some initial
configuration. Above 3 figures shows dynamics at
0, 0.25 and 1.0 scaling factors.
Supported by
NCC-2-1244
EIA-01-30352
(Harter Kozma, submitted Kozma, Freeman
Erdi, 2003)