The SNePS Approach to Cognitive Robotics

1
The SNePS Approach to Cognitive Robotics

• Stuart C. Shapiro
• Department of Computer Science and Engineering
• and Center for Cognitive Science
• University at Buffalo
• shapiro_at_cse.buffalo.edu

2
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

3
Goal

• A computational cognitive agent that can
• Understand and communicate in English
• Discuss specific, generic, and rule-like
  information
• Reason
• Discuss acts and plans
• Sense
• Act
• Remember and report what it has sensed and done.

4
Embodied Cassie

• A computational cognitive agent
• Embodied in hardware
• or Software-Simulated
• Based on SNePS and GLAIR.

5
SNePS

• Knowledge Representation and Reasoning
• Propositions as Terms
• SNIP SNePS Inference Package
• Specialized connectives and quantifiers
• SNeBR SNePS Belief Revision
• SNeRE SNePS Rational Engine
• Interface Languages
• SNePSUL Lisp-Like
• SNePSLOG Logic-Like
• GATN for Fragments of English.

6
GLAIR Architecture
Grounded Layered Architecture with Integrated
Reasoning
Knowledge Level
NL
SNePS
Perceptuo-Motor Level
Sensory-Actuator Level
Vision
Sonar
Motion
Proprioception
7
Interaction with Cassie
(Current) Set of Beliefs SNePS
English (Statement, Question, Command)
Reasoning Clarification Dialogue Looking in World
GATN Parser
(Updated) Set of Beliefs SNePS
Actions SNeRE
(New Belief) SNePS
Answer SNIP
GATN Generator
Reasoning
English sentence expressing new belief
answering question reporting actions
8
Cassie, the BlocksWorld Robot
9
Cassie, the FEVAHR
10
FEVAHR/Cassie in the Lab
11
FEVAHRWorld Simulation
12
UXO Remediation Cassie
Corner flag
Field
Drop-off zone
UXO
NonUXO object
Battery meter
Corner flag
Corner flag
Cassie
Recharging Station
Safe zone
13
Crystal Space Environment
14
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

15
Entities, Terms, Symbols, Objects

• Cassies mental entity a person named Bill
• SNePS term B5
• Object in world

16
Intensional Representation

• Intensional entities are distinct even if
  coreferential.
• The morning star is the evening star.
• George IV wondered if Scott was the author of
  Waverly.

17
McCarthys Telephone Number Problem

• Mary's telephone number is Mike's telephone
  number.
• I understand that Mike's telephone number is
  Mary's telephone number.
• Pat knew Mike's telephone number.
• I understand that Pat knew Mike's telephone
  number.
• Pat dialed Mike's telephone number.
• I understand that Pat dialed Mike's telephone
  number.

18
Answering the Telephone Number Problem

• Did Pat dial Mary's telephone number?
• Yes, Pat dialed Mary's telephone number.
• Did Pat know Mary's telephone number?
• I don't know.

19
Representing Propositions

• Propositions must be first-class entities of
  the domain
• Represented by terms.

20
Discussing Propositions

• That Bill is sweet is Mary's favorite
  proposition.
• I understand that Mary's favorite proposition is
  that Bill is sweet.
• Mike believes Mary's favorite proposition.
• I understand that Mike believes that Bill is
  sweet.

21
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

22
Logic for NLU Commonsense Reasoning

• Either Pat is a man or Pat is a woman or Pat is
  a robot.
• I understand that Pat is a robot or Pat is a
  woman or Pat is a man.
• Pat is a woman.
• I understand that Pat is a woman.
• What is Pat?
• Pat is a woman and Pat is not a robot and Pat
  is not a man.

23
Representation in FOPL?

• Man(Pat) ? Woman(Pat) ? Robot(Pat)

24
Representation in FOPL?

• Man(Pat) ? Woman(Pat) ? Robot(Pat)
• but dont want inclusive or

25
Representation in FOPL?

• Man(Pat) ? Woman(Pat) ? Robot(Pat)
• but dont want inclusive or
• Man(Pat) Woman(Pat) Robot(Pat)

T
T
T
F
T
So dont want exclusive or either
26
andor

• andor(i, j)P1, ..., Pn
• True iff at least i, and at most j of the Pi are
  True

27
thresh

• thresh(i, j)P1, ..., Pn
• True iff either fewer than i,
• or more than j
• of the Pi are True
• Note thresh(i, j)? andor(i, j)

28
or-entailment

• P1, ..., Pn vgt Q1, ..., Qn
• True iff for all i, j Pi ? Qj

29
and-entailment

• P1, ..., Pn gt Q1, ..., Qn
• True iff for all j
• P1 Pn ? Qj

30
Numerical entailment

• P1, ..., Pn igt Q1, ..., Qn
• True iff for all j
• andor(i, n)P1, , Pn ? Qj

31
Universal Quantifier

• all(u)(R1(u),..., Rn(u) gt C1(u),...,
  Cm(u))
• Every a that satisfies
• R1(u) Rn(u)
• also satisfies
• C1(u),..., Cm(u))

32
Numerical Quantifiers

• nexists(i,j,k)(x)
• (P1(x),..., Pn(x) Q(x))
• There are k individuals that satisfy
• P1(x) ?...? Pn(x)
• and, of them, at least i and at most j also
  satisfy
• Q(x)

33
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

34
MENTAL ACTS

• Believe(proposition)
• Disbelieve(proposition)

35
Act Selection

• Do-One(act1 ... actn)
• Snif(if(condition, act),
• ...
• if(condition, act)
• else(act))

36
Act Iteration

• Do-All(act1 ... actn)
• Sniterate(if(condition, act),
• ...
• if(condition, act),
• else(act))
• Snsequence(act1, ..., actn)
• Cascade(act1, ..., actn)
• P-Do-All(act1, ..., act2)

37
Entity Iteration

• WithSome(var, suchthat,
• do, else)
• WithAll(var, suchthat,
• do, else)
• WithSome(var, suchthat,
• do, else)
• WithNew(vars, thatare, suchthat,
• do, else)

38
Proposition/Act Transformers

• Achieve(proposition)
• ActPlan(act, plan)
• GoalPlan(proposition, act)
• Precondition(act, proposition)
• Effect(act, proposition)
• WhenDo(proposition, act)
• WheneverDo(proposition, act)
• IfDo(proposition, act)

39
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

40
Conditional Plans

• If a block is on a support then a plan to
  achieve that the support is clear is to pick up
  the block and then put the block on the table.
• all(x, y)
• (Block(x), Support(y), On(x, y)
• gt GoalPlan(Clear(y),
• Snsequence(Pickup(x),
• Put(x, Table))))
• STRIPS-like representation No times

41
Use of Conditional Plan

• GoalPlan(Clear(B),
• Snsequence(Pickup(A),
• Put(A, Table)))
• Remember (cache) derived propositions.

42
Use of Conditional Plan

• GoalPlan(Clear(B),
• Snsequence(Pickup(A),
• Put(A, Table)))???
• SNeBR to the rescue!

43
A FEVAHR Acting Rule

• all(p, o) (Agent(p), Thing(o)
• gt Precondition(Follow(p, o), Near(p, o)),
• GoalPlan(Near(p, o), Goto(p, o)),
• Precondition(Goto(p, o), Lookat(p, o)),
• ActPlan(Lookat(p, o), Find(p, o)))
• Uses a temporal model.

44
Acting According to the Rule
45
Acting According to the Rule
Follow a red robot.
I found a red robot. I am looking at a red robot.
46
Acting According to the Rule
Follow a red robot.
I found a red robot. I am looking at a red robot.
I went to a red robot. I am near a red robot. I
am following a red robot.
47
A Plan for Blowing up UXOs

• all(a)(Agent(a) gt
• ActPlan(Blowup(a, UXOs),
• Act(a,
• Cascade(SearchforUxo(a),
• WithSome(obj,
• Near(a, obj),
• WithNew(ch ex,
• Charge(ch), Explosion(ex),
• Possess(a, ch),
• Cascade(Place(a, ch, obj),
• Hide(a),
• Waitfor(a, ex),
• SearchforUxo(a))),
• goto(a, SafeZone))))))

48
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

49
Representation and Use of Indexicals

• Words whose meanings are determined by occasion
  of use
• E.g. I, you, now, then, here, there
• Deictic Center ltI, YOU, NOWgt
• I SNePS term representing Cassie
• YOU person Cassie is talking with
• NOW current time.

50
Analysis of Indexicals(in input)

• First person pronouns YOU
• Second person pronouns I
• here location of YOU
• Present/Past relative to NOW.

51
Generation of Indexicals

• I First person pronouns
• YOU Second person pronouns
• NOW used to determine tense and aspect.

52
Use of Indexicals 1
Come here.
53
Use of Indexicals 2
Come here.
I came to you, Stu. I am near you.
54
Use of Indexicals 3
Who am I?
Your name is Stu and you are a person.
Who have you talked to?
I am talking to you.
Talk to Bill.
I am talking to you, Bill.
Come here.
55
Use of Indexicals 4
Come here.
I found you. I am looking at you.
56
Use of Indexicals 5
Come here.
I found you. I am looking at you.
I came to you. I am near you.
57
Use of Indexicals 6
Who am I?
Your name is Bill and you are a person.
Who are you?
I am the FEVAHR and my name is Cassie.
Who have you talked to?
I talked to Stu and I am talking to you.
58
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

59
A Personal Sense of Time

• NOW contains SNePS term representing current
  time.
• NOW moves when Cassie acts or perceives a change
  of state.

60
Representation of Time
before
after
before
after
!
!
!
event
?????????????
time
agent
act
B1
action
object
B6
I
lex
NOW
61
Movement of Time
t1
62
Performing a Punctual Act
t1
63
Performing a Durative Act
t1
64
The Pacemaker

• PML process periodically increments variable
  COUNT.
• COUNT some PML integer.
• Reset to 0 when NOW moves.
• Provides bodily feel of passing time.

65
Quantizing Time

• Cannot conceptualize fine distinctions in time
  intervals.
• So quantize, e.g. into half orders of magnitude
  (Hobbs, 2000).

66
Movement of Time with Pacemaker
q
t1
t2
KL
PML
hom
COUNT
n
NOW
0
67
The Problem of the Fleeting Now

• How can you reason about now
• if it never stands still?

68
Fleeting Now Example 1

• 121500 Is John having lunch now?
• 121502 Agent walks to Johns office.
• 121700 Agent sees John at his desk, eating.
• 121900 Agent reports yes.
• Appropriate granularity.

69
Fleeting Now Example 2

• 121500 Is John having lunch now?
• Agent knows John is at home without a phone.
• Agent contemplates driving to Johns home.
• Dont bother---inappropriate granularity.

70
The Vagueness of now

• Im now giving a talk.
• Im now teaching a course.
• Im now visiting Toronto.
• Im now living in Buffalo.
• The agent is now walking to Johns office.
• The agent is now seeing if John is eating lunch.
• Multiple nows at different granularities.

71
NOW-MTF
Maximal Temporal Frame based on NOW
NOW
Semi-lattice of times, all of which contain
NOW, any of which could be meant by
now Finite---only conceptualized times of
conceptualized states
72
Moving NOW with MTF
NOW
73
Typical Durations

• If the walk light is on now, cross the street.
• Relevant duration is typical duration of walk
  lights.
• Is John having lunch now?
• Relevant duration is typical duration of lunch.
• Use quantized typical durations when updating
  NOW-MTFs.

74
Using Appropriate Granularity
Lunch time
Lunch?
Lunch!
NOW
Yes!
75
Outline

• Introduction
• Intensional Representation Propositions
• SNePS Connectives and Quantifiers
• SNeRE Acting Constructs
• Example Plans
• Representation and Use of Indexicals
• A Personal Sense of Time
• Summary

76
Goal

• A computational cognitive agent/robot
• That can communicate in natural language.

77
Intensional Representation Propositions

• SNePS terms represent mental entities.
• May assert that two entities are coreferential.
• Relations/acts may be declared transparent.
• Propositions are first-class entities.

78
SNePS Connectives and Quantifiers

• Designed logical connectives and rules of
  inference
• More appropriate for NLU and Commonsense
  reasoning
• than in standard FOPC.

79
SNeRE Acting Constructs

• Separate, but Coordinated
• Syntax and Semantics
• For Acting and for Reasoning

80
Representation and Useof Indexicals

• Use of Deictic Center for parser to interpret
  indexicals as current referents
• And for generator to generate indexicals from
  current referents.

81
A Personal Sense of Time

• NOW is current time.
• Updated when Cassie acts
• or perceives a change of state.
• Points into MTF to support vagueness of now.

82
For More Information

• Personnel
• Manual
• Tutorial
• Bibliography
• ftpable SNePS source code
• etc.
• http//www.cse.buffalo.edu/sneps/