SymbolAnchoring in Cassie

1
Symbol-Anchoring in Cassie

• Stuart C. Shapiro and Haythem O. Ismail
• Department of Computer Science and Engineering
• and Center for Cognitive Science
• University at Buffalo
• shapiro hismail_at_cse.buffalo.edu

2
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

3
Embodied Cassie

• A computational cognitive agent
• Embodied in hardware
• or Software-Simulated
• Based on SNePS and GLAIR.

4
Cassie, the FEVAHR
5
FEVAHR/Cassie in the Lab
6
SNePS

• Knowledge Representation and Reasoning
• Intensional Representation
• Propositions as Terms
• SNIP SNePS Inference Package
• Specialized connectives and quantifiers
• SNeBR SNePS Belief Revision
• SNeRE SNePS Rational Engine
• Interface Languages
• SNePSUL Lisp-Like
• SNePSLOG Logic-Like
• GATN for Fragments of English.

7
Interaction with Cassie
(Current) Set of Beliefs SNePS
English (Statement, Question, Command)
Reasoning Clarification Dialogue Looking in World
GATN Parser
(Updated) Set of Beliefs SNePS
Actions SNeRE
(New Belief) SNePS
Answer SNIP
GATN Generator
Reasoning
English sentence expressing new belief
answering question reporting actions
8
Entities, Terms, Symbols, Objects

• Cassies mental entity a person named Stu
• SNePS term B4
• Object in world

9
GLAIR Architecture
Grounded Layered Architecture with Integrated
Reasoning
Knowledge Level
SNePS
Perceptuo-Motor Level
NL
Sensory-Actuator Level
Vision
Sonar
Motion
Proprioception
10
Alignment
KL
SNePS term
PML structure
Body (PML/SAL)
World
Object/Phenomenon
Action
11
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

12
World Objectsto Feature Vectors
World
PML/SAL
. .
13
Feature Vectorsto KL Terms
PML/SAL
KL
ProperName(B4, Stu)
. .
Alignment
14
Incomplete Feature Vectors
PML/SAL
KL
Height(B4, B12)
. .
15
Unifying PML-Descriptions
PML/SAL
KL

B20

B30
Prop
Isa

B31
Prop

B6
16
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

17
Deictic Modality Registers
KL Terms denoting
PML
I You Now Vision . . .
Cassie Addressee Current time Current state(s) of
looking at x . . .
18
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

19
Primitive Actions Alignedwith PML/SAL Functions
PML
KL
SAL
World
Find(B4)
Find(B4)

Modality Registers
20
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

21
Aligning NOW using MTF
Maximal Temporal Frame based on NOW
KL
PML
NOW
Semi-lattice of times, all of which contain
NOW, any of which could be meant by
now Finite---only conceptualized times of
conceptualized states
22
Moving NOW with MTF
KL
NOW
PML
Moves when Cassie acts, newly observes a
state, or is informed of a new state. Always
includes times of states in modality registers.
23
Providing a Feel for Time
q
t1
t2
KL
PML
hom
COUNT
n
NOW
0
24
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

25
Aligning Lexemes/NL
PML/SAL
KL
World
ProperName(B4, Stu)
npr
ctgy
Stu
Isa(B4, person)
n
ctgy
person
sing
num
Grammar
NL
26
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

27
Acting 1
28
Acting 2
Follow a red robot.
I found a red robot. I am looking at a red robot.
29
Acting 3
Follow a red robot.
I found a red robot. I am looking at a red robot.
I went to a red robot. I am near a red robot. I
am following a red robot.
30
Use of Indexicals 1
Come here.
I found you. I am looking at you.
31
Use of Indexicals 2
Come here.
I found you. I am looking at you.
I came to you, Stu. I am near you.
32
Use of Indexicals 3
Who am I?
Your name is Stu and you are a person.
Who have you talked to?
I am talking to you.
Talk to Bill.
I am talking to you, Bill.
Come here.
33
Use of Indexicals 4
Come here.
I found you. I am looking at you.
34
Use of Indexicals 5
Come here.
I found you. I am looking at you.
I came to you. I am near you.
35
Use of Indexicals 6
Who am I?
Your name is Bill and you are a person.
Who are you?
I am the FEVAHR and my name is Cassie.
Who have you talked to?
I talked to Stu and I am talking to you.
36
Outline

• Introduction
• Perceivable entities and properties
• Attentional Structures
• Actions
• Time
• Language
• Examples
• Summary

37
Summary

• KL terms denote mental entities.
• KL terms aligned to PML structures.
• PML variables/registers contain KL terms.
• PML variables/registers/structures grounded in
  world via sensors effectors.
• PML attentional structures anchor symbols by
  changing only when motivated.

38
For More Information

• Personnel
• Manual
• Tutorial
• Bibliography
• ftpable SNePS source code
• etc.
• http//www.cse.buffalo.edu/sneps/