Computational Discovery of Communicable Knowledge

1
A Unified Cognitive Architecture for Physical
Agents
Pat Langley Dongkyu Choi Computational Learning
Laboratory Center for the Study of Language and
Information Stanford University, Stanford,
California USA http//cll.stanford.edu/
Thanks to K. Cummings, N. Nejati, S. Rogers, S.
Sage, and D. Shapiro for their many
contributions. This talk reports research. funded
by grants from DARPA IPTO, which is not
responsible for its contents.
2
Psychological Ideas as Design Heuristics
To develop intelligent systems, we must constrain
their design, and findings about human behavior
can suggest

• how the system should represent and organize
  knowledge
• how the system should use that knowledge in
  performance
• how the system should acquire knowledge from
  experience

This approach has led to many new insights,
starting with Newell, Shaw, and Simons (1956)
work on the Logic Theorist.
3
Cascaded Integration in ICARUS
In this talk I will use ICARUS ? a unified
cognitive architecture ? to illustrate the value
of ideas from psychology.
learning
problem solving
skill execution
conceptual inference
ICARUS adopts a cascaded approach to system
integration in which lower-level modules produce
results for higher-level ones.
4
Representing and Using Concepts
E.g., psychology makes claims about conceptual
knowledge

• concepts are distinct cognitive entities that
  support both categorization and inference
• the majority of human concepts are grounded in
  perception and action (Barsalou, 1999)
• many human concepts are relational in nature,
  describing connections among entities (Kotovsky
  Gentner, 1996)
• concepts are organized in a hierarchical manner,
  with more complex categories defined in terms of
  simpler ones.

ICARUS adopts these assumptions about conceptual
memory.
5
ICARUS Concepts for In-City Driving
((in-rightmost-lane ?self ?clane) percepts
( (self ?self) (segment ?seg) (line ?clane
segment ?seg)) relations ((driving-well-in-segme
nt ?self ?seg ?clane) (last-lane ?clane) (not
(lane-to-right ?clane ?anylane)))) ((driving-well
-in-segment ?self ?seg ?lane) percepts ((self
?self) (segment ?seg) (line ?lane segment ?seg))
relations ((in-segment ?self ?seg) (in-lane
?self ?lane) (aligned-with-lane-in-segment ?self
?seg ?lane) (centered-in-lane ?self ?seg
?lane) (steering-wheel-straight
?self))) ((in-lane ?self ?lane) percepts
( (self ?self segment ?seg) (line ?lane segment
?seg dist ?dist)) tests ( (gt ?dist -10)
(lt ?dist 0)))
6
Structure and Use of Conceptual Memory
ICARUS organizes conceptual memory in a
hierarchical manner.
Conceptual inference occurs from the bottom up,
starting from percepts to produce high-level
beliefs about the current state.
7
Representing Short-Term Beliefs/Goals
(current-street me A) (current-segment me
g550) (lane-to-right g599 g601) (first-lane
g599) (last-lane g599) (last-lane
g601) (at-speed-for-u-turn me) (slow-for-right-tur
n me) (steering-wheel-not-straight
me) (centered-in-lane me g550 g599) (in-lane me
g599) (in-segment me g550) (on-right-side-in-segme
nt me) (intersection-behind g550
g522) (building-on-left g288) (building-on-left
g425) (building-on-left g427) (building-on-left
g429) (building-on-left g431) (building-on-left
g433) (building-on-right g287) (building-on-right
g279) (increasing-direction me) (buildings-on-righ
t g287 g279)
8
Skills and Execution
Psychology also makes claims about skills and
their execution

• the same generic skill may be applied to distinct
  objects that meet its application conditions
• skills support the execution of complex
  activities that have hierarchical organization
  (Rosenbaum et al., 2001)
• humans can carry out open-loop sequences, but
  they can also operate in closed-loop reactive
  mode
• humans can deal with multiple goals with
  different priorities, which can lead to
  interrupted behavior.

ICARUS embodies these ideas in its skill
execution module.
9
ICARUS Skills for In-City Driving
((in-rightmost-lane ?self ?line) percepts
((self ?self) (line ?line)) start
((last-lane ?line)) subgoals ((driving-well-in-s
egment ?self ?seg ?line))) ((driving-well-in-seg
ment ?self ?seg ?line) percepts ((segment
?seg) (line ?line) (self ?self)) start
((steering-wheel-straight ?self)) subgoals
((in-segment ?self ?seg) (centered-in-lane ?self
?seg ?line) (aligned-with-lane-in-segment ?self
?seg ?line) (steering-wheel-straight
?self))) ((in-segment ?self ?endsg) percepts
((self ?self speed ?speed) (intersection ?int
cross ?cross) (segment ?endsg street ?cross
angle ?angle)) start ((in-intersection-fo
r-right-turn ?self ?int)) actions ((?steer
1)))
10
ICARUS Skills Build on Concepts
ICARUS stores skills in a hierarchical manner
that links to concepts.
concepts
Each concept is defined in terms of other
concepts and/or percepts. Each skill is defined
in terms of other skills, concepts, and percepts.
skills
11
Skill Execution in ICARUS
Skill execution occurs from the top down,
starting from goals to find applicable paths
through the skill hierarchy.
This occurs repeatedly on each cycle to support
reactive control with a bias toward persistence
of initiated skills.
12
Ideas about Problem Solving and Learning
Psychology also has ideas about problem solving
and learning

• humans often resort to means-ends analysis to
  solve novel problems (Newell Simon, 1961)
• problem solving often occurs in a physical
  context and is interleaved with execution
  (Gunzelman Anderson, 2003)
• efforts to overcome impasses during problem
  solving leads to incremental acquisition of new
  skills (Anzai Simon, 1979)
• structural learning involves monotonic addition
  of symbolic elements to long-term memory
• learning can transform backward-chaining
  heuristic search into informed forward-chaining
  execution (Larkin et al., 1980).

ICARUS reflects these ideas in its problem
solving and learning.
13
ICARUS Interleaves Execution and Problem Solving
Skill Hierarchy
Problem
Reactive Execution
?
no
impasse?
Primitive Skills
Executed plan
yes
Problem Solving
This organization reflects the psychological
distinction between automatized and controlled
behavior.
14
ICARUS Learns Skills from Problem Solving
Reactive Execution
no
impasse?
Primitive Skills
Executed plan
yes
Problem Solving
Skill Learning
15
Learning Skills for In-City Driving
We have trained ICARUS to drive in a simulated
in-city environment. We provide the system with
tasks of increasing complexity. Learning
transforms the problem-solving traces into
hierarchical skills. The agent uses these skills
to change lanes, turn, and park using only
reactive control.
16
Similarities to Previous Architectures
ICARUS has much in common with other cognitive
architectures like Soar (Laird et al., 1987) and
ACT-R (Anderson, 1993)

• Short-term memories are distinct from long-term
  stores
• Memories contain modular elements cast as
  symbolic structures
• Long-term structures are accessed through pattern
  matching
• Cognition occurs in retrieval/selection/action
  cycles
• Learning involves monotonic addition of elements
  to memory
• Learning is incremental and interleaved with
  performance

These ideas all have their origin in theories of
human memory, problem solving, and skill
acquisition.
17
Distinctive Features of ICARUS
However, ICARUS also makes assumptions that
distinguish it from these architectures

• Cognition is grounded in perception and action
• Categories and skills are separate cognitive
  entities
• Short-term elements are instances of long-term
  structures
• Inference and execution are more basic than
  problem solving
• Skill/concept hierarchies are learned in a
  cumulative manner

Some of these assumptions appear in Bonasso et
al.s (2003) 3T, Freeds APEX, and Sun et al.s
(2001) CLARION architectures.
These ideas have their roots in cognitive
psychology, but they are also effective in
building integrated intelligent agents.
18
Directions for Future Research
Future work on ICARUS should incorporate other
ideas about

• progressive deepening in forward-chaining search
• graded nature of categories and category learning
• model-based character of human reasoning
• persistent but limited nature of short-term
  memories
• creating perceptual chunks to reduce these
  limitations
• storing and retrieving episodic memory traces

These additions will increase further ICARUS
debt to psychology.
For more details, see http//cll.stanford.edu/res
earch/ongoing/icarus/
19
End of Presentation