Building a System that can Learn by Reading

1
Building a System that canLearn by Reading

• Kevin Livingston
• PhD Candidate
• Cognitive Systems Division
• EECS Department
• Northwestern University
• Presented at University of Dayton November 3,
  2006
• As part of the Computer Science Research
  Colloquium Series

2
Understanding Systems
Text Understanding System
Question Answering
Knowledge Base
Explanation
Reasoning
3
Examples of Intelligent Systems

• Digital Assistants
• Conversational Agents
• Imagine a video game character that actually
  talked to you
• Intelligent Games
• Search
• Question Answering

4
Modeling and Using Knowledge

• Frames (Minsky)
• Scripts (Shank)
• Qualitative Reasoning (Forbus)

5
Semantic Memory

• Ontology
• Doctors are people
• Bombings are events
• Semantics
• Doctors have patients
• Bombings have targets

6
Episodic Memory

• Memory Instances
• Madrid is a city
• Madrid is in Spain
• The Madrid Bombings is a terrorist attack
• The Madrid Bombings occurred on March 11, 2004
• Al Qaida is a terrorist organization

7
Modeling Reasoning

• Logic Based Systems
• Cyc (Cycorp Austin, TX)
• Fire (Qualitative Reasoning Group (QRG)
  Northwestern)
• Statistical
• Bayesian
• Markovian
• Neural Nets

8
Knowledge Bases
Text Understanding System
Question Answering
Knowledge Base
Explanation
Reasoning
9
Available Knowledge Bases

• Information Retrieval (IR) techniques
• mine information from Internet (MUC and TREC)
• Open Mind Common Sense (OMCS)
• Sentences collected from Internet contributors
• Mined for knowledge
• Knowledge Machine (KM)
• Frame based
• ResearchCyc
• Latest release 3,000,000 assertions
• Predicate Logic in CycL
• First order and some second order constructs

10
ResearchCyc

• Bombings are Attacks and Attacks are Events
• (genls Bombing AttackOnObject)
• (genls AttackOnObject Event)
• The Madrid terrorist attack was a bombing
• (isa
• TerroristAttack-September-8-2003-Madrid
• Bombing)
• Al Qaida is an Islamist Terrorist Group
• (isa AlQaida TerroristGroup-Islamist)

11
ResearchCyc (cont.)

• Location of an event
• (eventOccursAt
• TerroristAttack-September-8-2003-Madrid
• CityOfMadridSpain)
• Perpetrator of an event
• (perpetrator
• TerroristAttack-September-8-2003-Madrid
• AlQaida)

12
ResearchCyc (cont.)

• Person being killed
• (organismKilled SpaceShuttleChallengerDisaster
  ChristaMcAuliffe)
• Deaths caused by an attack
• (deathToll
• TerroristAttack-September-8-2003-Madrid
• Person
• 190)

13
Building Knowledge Bases

• Slow and Tedious
• years to grow Cyc from 1.2M to 3M assertions
• Requires Training
• measured in weeks, for GUI tools (SHAKEN)
• Expensive
• Project Halo estimates 10,000 per page! of AP
  level Chemistry content

14
Available Information

• Encyclopedias
• Newspapers
• Online sources
• Print

15
A Better Way?Teach the Computer to Read
16
What we want to Read

• Episodic Knowledge
• New people
• New events
• General Knowledge
• the heart is a pump

17
Standard Model forNatural Language Processing
Tagged Text
Syntactic Parser
POS Tagging
Dictionary
Grammar
Semantic Interpreter
Semantics
18
Time flies like an arrow.
19
Time flies like an arrow.

• Time moves quickly just like an arrow does

20
Time flies like an arrow.

• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow

21
Time flies like an arrow.

• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
• Time flies in the same way that an arrow would
  (time them)

22
Time flies like an arrow.

• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
• Time flies in the same way that an arrow would
  (time them)
• Time those flies that are like arrows

23
Time flies like an arrow.

• Time moves quickly just like an arrow does
• (You should) time flies like you would an arrow
• Time flies in the same way that an arrow would
  (time them)
• Time those flies that are like arrows
• A type of flying insect, "time-flies," enjoy
  arrows (compare Fruit flies like a banana.)

24
Picture of George Burns And Grace Allen
25
Grace, Those are beautiful flowers.
Picture of George Burns And Grace Allen
26
Grace, Those are beautiful flowers.
Picture of George Burns And Grace Allen
Where did they come from?
27
Dont you remember, George?
Picture of George Burns And Grace Allen
28
Picture of George Burns And Grace Allen
You said that if I went to visit Clara Bagley in
the hospital I should be sure to take her flowers.
So when she wasnt looking, I did.
29
Picture of George Burns And Grace Allen
30
Common Sense

• You said that if I went to visit Clara Bagley in
  the hospital I should be sure to take her
  flowers. So, when she wasn't looking, I did.
• take flowers from her
• take flowers to her

31
Picture of Elevator Operator
32
Down?
Picture of Elevator Operator
33
That Way!
Picture of Elevator Operator
34
Context in the Environment

• Down?
• What does the question mean?
• Which way is down?
• Are you going down?

35
Language Understanding

• Goals include understanding
• The meaning of the text
• How it fits into what is known
• The purpose of being told
• Requires
• Common sense knowledge
• Awareness of context
• Our model is to
• get to knowledge as quickly as possible,
• with as few intermediate steps as possible.

36
(No Transcript)
37
Reader Example

• An attack occurred in Madrid
• The bombing killed 190 people
• The bombing was perpetrated by Al-Qaida
• (eventOccursAt
• TerroristAttack-September-8-2003-Madrid
• CityOfMadridSpain)
• (perpetrator
• TerroristAttack-September-8-2003-Madrid
• AlQaida)
• (deathToll
• TerroristAttack-September-8-2003-Madrid
• Person 190)
• (isa TerroristAttack-September-8-2003-Madrid
• Bombing)

38
Lexical Processing

• An attack occurred in Madrid.
• attack
• (singular Attack-TheWord attack)
• (denotation Attack-TheWord CountNoun 0
  AttackOnObject)
• (isa ?x AttackOnObject)
• Madrid
• (placeName-Standard CityOfMadridSpain
  "Madrid")
• (isa CityOfMadridSpain City)

39
Rule Pattern

• Pattern
• (isa ?event Event)
• Occur-TheWord
• In-TheWord
• (isa ?location GeographicLocation)
• Results
• (eventOccursAt ?event ?location)

40
Pattern Matching

• Pattern
• (isa ?event Event)
• Occur-TheWord
• In-TheWord
• (isa ?location GeographicLocation)
• Input
• An
• attack (isa ?x AttackOnObject)
• occurred Occur-TheWord
• in In-TheWord
• Madrid (isa CityOfMadridSpain City)

41
Rule Completion

• An attack occurred in Madrid.
• Pattern
• (isa ?event Event)
• Occur-TheWord
• In-TheWord
• (isa ?location GeographicLocation)
• Results
• (eventOccursAt ?event ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))

42
Remindings

• An attack occurred in Madrid.
• Pattern (isa ?event Event)
• Occur-TheWord
• In-TheWord
• (isa ?location GeographicLocation)
• Results
• (eventOccursAt ?event ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))
• Remindings from Memory
• ((?event . TerrorAttack-Sept8-2003-Madrid))

43
Coreference Resolution

• An attack occurred in Madrid.

The bombing was perpetrated by Al-Qaida.
44
Coreference Resolution

• An attack occurred in Madrid
• Results (eventOccursAt
• ?event
• ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))
• Remindings from Memory
• ((?event . TerrorAttack-Sept8-2003-Madrid))

• The bombing was perpetrated by Al-Qaida
• Results (perpetrator
• ?action
• ?agent)
• Constraints
• (isa ?action Bombing)
• Bindings from Reading
• ((?agent . AlQaida))

45
Coreference Resolution

• An attack occurred in Madrid
• Results (eventOccursAt
• ?event
• ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))
• Remindings from Memory
• ((?event . TerrorAttack-Sept8-2003-Madrid))

• The bombing was perpetrated by Al-Qaida
• Results (perpetrator
• ?action
• ?agent)
• Constraints
• (isa ?action Bombing)
• Bindings from Reading
• ((?agent . AlQaida))

46
Coreference Resolution

• An attack occurred in Madrid
• Results (eventOccursAt
• ?event
• ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))
• Remindings from Memory
• ((?event . TerrorAttack-Sept8-2003-Madrid))

• The bombing was perpetrated by Al-Qaida
• Results (perpetrator
• ?action
• ?agent)
• Constraints
• (isa ?action Bombing)
• Bindings from Reading
• ((?agent . AlQaida))

47
Coreference Resolution

• An attack occurred in Madrid
• Results (eventOccursAt
• ?event
• ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))
• Remindings from Memory
• ((?event . TerrorAttack-Sept8-2003-Madrid))

• The bombing was perpetrated by Al-Qaida
• Results (perpetrator
• ?action
• ?agent)
• Constraints
• (isa ?action Bombing)
• Bindings from Reading
• ((?agent . AlQaida))

48
Coreference Resolution

• An attack occurred in Madrid
• Results (eventOccursAt
• ?event
• ?location)
• Constraints
• (isa ?event AttackOnObject)
• Bindings from Reading
• ((?location . CityOfMadridSpain))
• Remindings from Memory
• ((?event . TerrorAttack-Sept8-2003-Madrid))

• The bombing was perpetrated by Al-Qaida
• Results (perpetrator
• ?action
• ?agent)
• Constraints
• (isa ?action Bombing)
• Bindings from Reading
• ((?agent . AlQaida))

49
Coreference

• Refer to a more general or specific type
• bombing and attack
• Consistent with being the same,
• have a known shared instance in memory
• doctor and father of four

50
Tracking Ambiguity

• Words are ambiguous
• Bush
• A shrubbery?
• A president? Which one?
• Sentences and Phrases
• take her flowers
• Iraq borders Iran on the North.
• Intension
• Down?
• Where is Baghdad?

51
(No Transcript)
52
What Learning Reader can Read

• Information about existing instances
• the 2003 Madrid bombing
• Information extending existing instances
• the attack killed 190 people
• Information representing new instances
• An attack occurred in Al Anbar.

53
What Learning Reader cant Read

• Larger patterns
• Scripts A bomb went off.
• 6 people were arrested.
• Generalizations
• All countries have political leaders.
• The heart is a pump.

54
Acknowledgements
Dr. Christopher K. Riesbeck Dr. Ken Forbus Dr.
Larry Birnbaum Abhishek Sharma Dr. Jennifer
Seitzer Dr. Saverio Perugini grant
HR0011-04-1-0051
55
For more Information

• http//cs.northwestern.edu/livingston/
• http//cs.northwestern.edu/livingston/talks.html
• Kevin Livingston
• PhD Candidate
• Cognitive Systems Division
• EECS Department
• Northwestern University