Some Information

1
Some Information

• Parking is free in Stanford lots over the
  weekend.
• There is no registration fee thanks to ONR and
  NSF.
• Participation in the meeting is by invitation
  only.
• We will provide lunch on both days to registered
  participants.
• We will provide dinner on Saturday to registered
  participants.
• Restrooms are in the hall to the right of the
  Cordura lobby.

Some Requests

• Speakers Please limit your talks to the allotted
  35 minutes.
• Audience Please ask speakers only one question
  at a time.
• Please recycle bottles, drinks, and paper in
  labeled receptacles.
• Please do not smoke in or near the CSLI buildings
  or events.

2
Reasoning and Learning in Cognitive Systems
Pat Langley Center for the Study of Language and
Information Stanford University, Stanford,
California http//cll.stanford.edu/langley langle
y_at_csli.stanford.edu
The views contained in these slides are the
authors and do not represent official policies,
either Expressed or implied, of the Defense
Advanced Research Projects Agency or the DoD.
3
Motivation for the Symposium
A number of factors encouraged us to organize
this symposium

• Reasoning and learning are both central aspects
  of intelligence,
• but the two research groups have become nearly
  disjoint.
• There exist substantial results on reasoning and
  learning,
• but many have forgotten or never learned about
  them.
• There are growing needs for integrated
  intelligent systems,
• but research focuses primarily on component
  technologies.
• DARPA now wants cognitive systems that reason and
  learn.

We hope this meeting can help build a community
of researchers that can respond to these problems
and opportunities.
4
Elements of Machine Learning
performance element
environment
knowledge
learning element
5
Learning to Improve Reasoning
We can state the general task of learning to
improve reasoning as

• Given Initial knowledge elements for a
  particular domain
• Given A performance system that can compose
  these elements dynamically to solve problems or
  accomplish goals
• Given Traces of the performance systems
  behavior or advice about how to solve problems in
  the domain
• Find New or revised knowledge elements that
  improve system performance on novel problems.

Much of the early research on machine learning
can be cast in just these terms.
6
Some Systems that Reason and Learn
STRIPS (1972)
ACT-F (1981)
Anzai (1978)
LEX (1981)
SAGE (1982)
UPL (1983)
Soar (1984)
MORRIS (1985)
LEAP (1985)
MacLearn (1985)
Eureka (1989)
Prodigy/E (1988)
Bagger (1990)
PRIAR (1990)
Daedalus (1991)
Prodigy/A (1993)
Cascade (1993)
SCOPE (1996)
7
Characteristics of Early Research

• The performance system engaged in multi-step
  reasoning by dynamic composition of knowledge
  elements.
• Learning methods were typically incremental and
  integrated with the performance system.
• Learning was relatively rapid and took at least
  some domain knowledge into account.
• Learning was embedded in a problem-solving
  architecture that made representational and
  performance assumptions.
• Research emphasized support of cognitive
  abilities, such as planning and reasoning, rather
  than perception and execution.
• Researchers looked to psychology and logic for
  ideas, rather than to statistics and operations
  research.

8
Some Historical Developments

• 1959 Creation of the General Problem Solver
• Development of STRIPS with MACROPs

9
Some Historical Developments

• 1959 Creation of the General Problem Solver
• 1972 Development of STRIPS with MACROPs
• 1978 First adaptive production systems developed
• 1980 Carnegie symposium on learning and cognition
• 1981 Growth of work on learning in problem
  solving
• Active research on cognitive architectures

10
Some Historical Developments

• 1959 Creation of the General Problem Solver
• 1972 Development of STRIPS with MACROPs
• 1978 First adaptive production systems developed
• 1980 Carnegie symposium on learning and cognition
• 1981 Growth of work on learning in problem
  solving
• 1983 Active research on cognitive architectures
• 1986 Growth of explanation-based learning
  movement
• 1988 Recognition of the utility problem
• 1989 Rise of experimental method, advent of UCI
  repository
• 1991 ISLE/Stanford symposium on learning and
  planning

11
Some Historical Developments

• 1959 Creation of the General Problem Solver
• 1972 Development of STRIPS with MACROPs
• 1978 First adaptive production systems developed
• 1980 Carnegie symposium on learning and cognition
• 1981 Growth of work on learning in problem
  solving
• 1983 Active research on cognitive architectures
• 1986 Growth of explanation-based learning
  movement
• 1988 Recognition of the utility problem
• 1989 Rise of experimental method, advent of UCI
  repository
• 1991 ISLE/Stanford symposium on learning and
  planning
• 1992 Influx of ideas from pattern recognition
• 1993 Excitement about reinforcement learning
• 1995 Influx of ideas from operations research
• 1998 Reduced effort on learning and reasoning

12
Some Encouraging Signs
In recent years, there have been some positive
developments

• academic courses and tutorials on learning and
  reasoning
• AI Magazine survey of work on learning in
  planning domains
• interest in model-based and relational
  reinforcement learning
• broader interest in integrated cognitive
  architectures
• DARPA workshop on rapid, embedded, and enduring
  learning
• prospects for DARPA program in learning for
  cognitive systems.

Taken together, these suggested the time had
arrived for another meeting on reasoning and
learning.
13
Some Omitted Paradigms
The meeting has some great speakers reporting on
great topics, but some may wonder why there are
no talks on

• Probabilistic learning and reasoning in Bayesian
  networks
• Model-based approaches to learning from delayed
  reward
• Learning action models for use in planning and
  execution.

Each framework can learn knowledge that supports
some form of multi-step reasoning or inference.
However, research in these paradigms focuses on
statistical issues rather than structural ones,
which we emphasize here.
14
Some Open Research Problems
Previous research in the area of learning and
reasoning has

• focused on acquisition of relatively small
  knowledge bases
• dealt with learning over relatively short periods
  of time
• emphasized mental processes over action and
  perception
• preferred logical, all-or-none frameworks over
  alternatives
• downplayed the role of hierarchical knowledge
  structures
• relied primarily on initial, handcrafted
  representations.

Each of these suggests open problems that should
be addressed in future projects.
15
Challenge Learning to Improve Reasoning
Current learning research focuses on performance
tasks that

• involve one-step decisions for classification or
  regression
• utilize simple reactive control for acting in the
  world.

But many other varieties of learning instead
involve

• the acquisition of modular knowledge elements
  that
• can be composed dynamically by multi-step
  reasoning.

We should give more attention to learning such
compositional knowledge.
knowledge
knowledge
reasoning
reasoning
16
Challenge More Rapid Learning
Current learning research focuses on asymptotic
behavior

• methods for learning classifiers from thousands
  of cases
• methods that converge on optimal controllers in
  the limit.

In contrast, humans are typically able to

• learn reasonable behavior from relatively few
  cases
• take advantage of knowledge to speed the learning
  process.

We need more work on knowledge-guided learning of
this variety.
performance
experience
17
Challenge Cumulative Learning
Current learning research focuses on isolated
induction tasks that

• take no advantage of what has been learned
  before
• provide no benefits for what is learned
  afterwards.

In contrast, much human learning involves

• incremental acquisition of knowledge over time
  that
• builds on knowledge acquired during earlier
  episodes.

We need much more research on such cumulative
learning.
initial knowledge
extended knowledge
18
Challenge Evaluating Embedded Learning
Current evaluation emphasizes static data sets
for isolated tasks that

• favor work on minor refinements of existing
  component algorithms
• encourage mindless bake offs that provide
  little understanding.

To support the evaluation of embedded learning
systems, we need

• a set of challenging environments that exercise
  learning and reasoning,
• that include performance tasks of graded
  complexity and difficulty, and
• that have real-world relevance but allow
  systematic experimental control.

battle management
in-city driving
air reconnaissance
19
An Advertisement Progress on ICARUS
We are extending ICARUS, an integrated cognitive
architecture that

• stores long-term knowledge as hierarchical skills
  and concepts
• encodes short-term elements as instances of
  long-term structures
• uses numeric value functions to select skill
  paths for execution
• modulates reactive behavior with a bias toward
  persistence
• learns value functions for concepts and durations
  of skills
• invokes means-ends analysis to handle
  unexecutable skills
• learns new hierarchical skills upon resolution of
  impasses

Come to our poster this evening to hear more
about the system.
20
Some Information

• Parking is free in Stanford lots over the
  weekend.
• There is no registration fee thanks to ONR and
  NSF.
• Participation in the meeting is by invitation
  only.
• We will provide lunch on both days to registered
  participants.
• We will provide dinner on Saturday to registered
  participants.
• Restrooms are in the hall to the right of the
  Cordura lobby.

Some Requests

• Speakers Please limit your talks to the allotted
  35 minutes.
• Audience Please ask speakers only one question
  at a time.
• Please recycle bottles, drinks, and paper in
  labeled receptacles.
• Please do not smoke in or near the CSLI buildings
  or events.

21
End of Presentation
22
Expanding our Computational Horizons
The field of machine learning has many success
stories, but

• these successes are prime examples of niche AI,
  which
• develops techniques that are increasingly
  powerful
• but that apply to an ever narrower classes of
  problems.

Instead, we need a new vision for machine
learning technology that

• supports the construction of general intelligent
  systems
• aspires to the same learning abilities as appear
  in humans.

This would produce a broader research agenda that
would take the field into unexplored regions.
niche AI
power
cognitive systems
generality
23
Challenge 1 More Varied Learning
Current learning research emphasizes tasks like
classification and reactive control, whereas
humans learn

• grammars for understanding natural language
• heuristics for reasoning and problem solving
• scripts and procedures for routine behavior
• cognitive maps for localization and navigation
• models that explain the behavior of artifacts.

We need more work on learning such varied
knowledge structures.
human learning abilities
current focus of machine learning