Dr. Douglas B. Lenat

1
Upper Ontology Symposium
Formal Ontologies

• Dr. Douglas B. Lenat
• , 3721 Executive Center Drive,
  Suite 100, Austin, TX 78731
• Email Lenat_at_cyc.com
• Phone [phone]

2 July 2005
2
Upper Ontology Symposium
Formal Ontologies
Can Anything With That Title be Understandable or
Interesting?

• Dr. Douglas B. Lenat
• , 3721 Executive Center Drive,
  Suite 100, Austin, TX 78731
• Email Lenat_at_cyc.com
• Phone [phone]

2 July 2005
3
Upper Ontology Symposium
Formal Ontologies
Can Anything With That Title be Understandable or
Interesting?

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer
  queries, search, translate, transcribe speech,
  etc.

2 July 2005
4
Contrast With NO Formal Ontology programs are
lucky to even have the appearance of
understanding, a thin veneer of intelligence.
E.g., ELIZA.

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer
  queries, search, translate, transcribe speech,
  etc.

2 July 2005
5
ELIZA (DOCTOR)

• 1965 Joe Weizenbaum, MIT
• Simulation/Parody of reflection psychotherapy
• Patient I swear a lot.
• How do you feel about the fact that you swear a
  lot?

Carl Rogers
Patient My dogs mother died recently. Tell
me more about your mother.
6
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
7
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
8
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
9
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
10
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
11
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
12
ELIZA (DOCTOR)

• blah blah blah mother blah blah blah

Patient My dogs mother died recently. Tell
me more about your mother.
13

• ALICE (today!!)
• JUDGE119260I WILL ASK YOU SOME SIMPLE
  QUESTIONS AND GIVE SIMPLE COMMANDS
• PROGRAM9260Do you think your plan will
  succeed?
• JUDGE1192616WHAT IS THE COLOR OF A BLUE CAR
• PROGRAM92616WHAT COLOR IS a blue car

• JUDGE1192635WHERE IS SUES NOSE WHEN SUE IS
  IN HER HOUSE
• PROGRAM92635Where it belongs.
• Try searching the Web.

14
(No Transcript)
15
(No Transcript)
16
From the Users POV

• The user has a question they want answered
• The data needed to answer it is available to
  them, but not in one single, obvious, reliable
  place
• The answers follow logically (and/or
  arithmetically) from m elements in n sources
• Dont want to have to know, ahead of time, what
  sources to go to, how to access them, how to
  combine the intermediate results.
• Do want to be able to limit, ahead of time, the
  uncertainty, recency, granularity, ideology
  (and/or see such meta-level info for each answer)

Which first-run movies star a teenager born in
Texas and are showing today at a theater lt 10
minutes drive from this building?
17
MicrowaveOven is a type of Kitchen-Appliance Dishw
asher is a type of Kitchen-Appliance

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
18
Rthagide-disjaks is a type of Kitchen-Appliance Gr
acinimumples is a type of Kitchen-Appliance

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
19
Rthagide-disjaks is a type of Kitchen-Appliance Gr
acinimumples is a type of Kitchen-Appliance Rthagi
de-disjaks requires Electricity. Gracinimumples
requires Electricity and Water.

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
20
Rthagide-disjaks is a type of Kitchen-Appliance Gr
acinimumples is a type of Kitchen-Appliance Rthagi
de-disjaks requires Vorawnistz. Gracinimumples
requires Vorawnistz and Buzqa.

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
21
Rthagide-disjaks is a type of Kitchen-Appliance Gr
acinimumples is a type of Kitchen-Appliance Rthagi
de-disjaks requires Vorawnistz. Gracinimumples
requires Vorawnistz and Buzqa. Buzqa is a Liquid
and supplied through Pipes.

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
22
Rthagide-disjaks is a type of Kitchen-Appliance Gr
acinimumples is a type of Kitchen-Appliance Rthagi
de-disjaks requires Vorawnistz. Gracinimumples
requires Vorawnistz and Buzqa. Buzqa is a Thwarn
and supplied through Epluns.

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
23
Eventually, after writing millions of these
rules, the system knows as much about pipes,
liquids, water, electricity, microwave ovens,
dishwashers, etc. as you and I do. (In
Technospeak eventually there is just one
interpretation of that model.)

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
24
Example Google How could it be more powerful
if it were formal (had some understanding)?

• The basic idea
• Get the computer to understand, not just store,
  information. Then it can reason to answer your
  queries.

2 July 2005
25
How formalized knowledge helps search

• Query Show me pictures of strong and
  adventurous people
• Caption A man climbing a rock face

find information by inference (KB)
26
How formalized knowledge helps search

• Query Outdoor explosions in terrorist
  events Lebanon between 1990 and 2001
• Document 1993 pipe bombing on the patio of the
  Beirut Hilton coffee shop.

Text Document
find information by inference (KB)
27
How formalized knowledge helps search
both general and domain knowledge

• Query Threats to low-flying US airliners
  in Lebanon
• Document Hizballah buys ten SA-7s.

Text Document
find information by inference (KB)
28

• Do you mean
• vets (military veteran)
• vets (veterinary surgeon)

29
(ex-serviceman OR mili

• Do you mean
• vets (military veteran)
• vets (veterinary surgeon)

(ex-serviceman OR "military veteran") OR vet OR
veteran AND NOT (veterinarian OR "veterinary
surgeon" OR animal doctor
30
(ex-serviceman OR mili

• Do you mean
• vets (military veteran)
• vets (veterinary surgeon)

(ex-serviceman OR "military veteran") OR vet OR
veteran AND NOT (veterinarian OR "veterinary
surgeon" OR animal doctor
Web Results 1 -25
Revise
New Search
vets 25,947 matches
vets 388,109 matches
31

• Do you mean
• vets (military veteran)
• vets (veterinary surgeon)

32
veterinarian OR veteri

• Do you mean
• vets (military veteran)
• vets (veterinary surgeon)

veterinarian OR "veterinary surgeon" OR animal
doctor AND NOT (ex-serviceman OR "military
veteran" OR veteran)
33
veterinarian OR veteri

• Do you mean
• vets (military veteran)
• vets (veterinary surgeon)

veterinarian OR "veterinary surgeon" OR animal
doctor AND NOT (ex-serviceman OR "military
veteran" OR veteran)
Web Results 1 -25
Revise
New Search
vets 25,947 matches
vets 153,060 matches
34
Find and clean (consistency-check) information by
inference (KB)
35
Find and clean (consistency-check) information by
inference (KB)
36
How can our programs be intelligent, not merely
have the veneer of it?

• ANSWER By having and being able to apply, not
  just store a large corpus of knowledge,
  spanning the gamut from specific domain-dependent
  all the way up to general common sense.
• E.g., consider the task of getting a program to
  understand natural language. How would having
  lots of machine-usable knowledge help?

37
Natural Language Understanding requires having
lots of knowledge

• 1. The pen is in the box.
  The box is in the pen.
• 2. The police watched the demonstrators
• because they feared violence.
• because they advocated violence.

38
Natural Language Understanding requires having
lots of knowledge

• 3. Mary and Sue are sisters.
• Mary and Sue are mothers.

39
Natural Language Understanding requires having
lots of knowledge

• 4. Every American has a mother.
• Every American has a president.
• 5. John saw his brother skiing on TV. The fool
• ...didnt have a coat on!
• didnt recognize him!

40
Natural Language Understanding requires having
lots of knowledge

• 6. George Burns My aunt is in the hospital.
• I went to see her today, and took her flowers.
• Gracie Allen George, thats terrible!
• You should have brought her flowers!

Took Table Sanction . . .
41
What is this knowledge?

• Millions of facts, rules of thumb, etc.
• Represented as sentences in some language
• If the language is Logic, computers can do the
  deductive reasoning automatically, themselves
• The sentences are all composed of words the
  full list of words is what we call the ontology
• The sentences, expressed in logic, are formal
• Thats why we call the words (terms) and logic
  sentences (axioms) about them a formal ontology

42
Organize Terms into an Ontology
Vehicle
Surface Vehicle
Water Vehicle
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
INSTANCE
Truck
Truck809726543
43
Attach Facts/Rules/... to the Nodes

• (inherit knowledge through class hierarchy)

Vehicle
Vehicle
Surface Vehicle
Water Vehicle
Surface Vehicle
Water Vehicle
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
INSTANCE
Truck
INSTANCE
Truck
Truck809726543
Truck809726543
44
Move each rule to the best place
Vehicle
Vehicle
Driven by a trained adult human
Surface Vehicle
Water Vehicle
Surface Vehicle
Water Vehicle
Cant control its altitude
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
INSTANCE
Truck
INSTANCE
Truck
Leaves tracks
Truck809726543
Truck809726543
45
Move each rule to the best place
Vehicle
Vehicle
Driven by a trained adult human
Surface Vehicle
Water Vehicle
Surface Vehicle
Water Vehicle
Cant control its altitude
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Slow down in bad weather
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
INSTANCE
Truck
INSTANCE
Truck
Leaves tracks
Truck809726543
Truck809726543
46
Move each rule to the best place
Vehicle
Vehicle
Driven by a trained adult human
Surface Vehicle
Surface Vehicle
Water Vehicle
Water Vehicle
Cant control its altitude
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Overland Vehicle
Surface Water Vehicle
Submarine Vehicle
Slow down in bad weather
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
Wheeled Vehicle
Railed Vehicle
Tracked Vehicle
INSTANCE
Truck
INSTANCE
Truck
Leaves tracks
Truck809726543
Truck809726543
47
What do we mean represent it in logic?
(isa Socrates Man)
Socrates is a man (genls Man Mortal)
all men are
mortal (ForAll ?x
all men are mortal (implies
(isa ?x Man) (isa ?x
Mortal))) (ForAll ?x each
person has a mother whos a female person
(implies (isa ?x Person)
(ThereExists ?y (and
(isa ?y FemalePerson)
(mother ?x ?y)))))
48
What do we mean it can reason?
Simple (isa Socrates Man) (ForAll ?x
(implies (isa ?x Man) (isa ?x Mortal)))
Harder Using general and specific knowledge
Can a can can-can? Very complex An example
from our AKB (Analysts Knowledge Base)
49
What do we mean it can reason?
Simple (isa Socrates Man) (ForAll ?x
(implies (isa ?x Man) (isa ?x
Mortal))) (isa Socrates Mortal) Harder Using
general and specific knowledge Can a can
can-can? Very complex An example from our AKB
(Analysts Knowledge Base)
50
Can a can can-can?
51
Can a can can-can?
52
What do we mean it can reason?
Simple (isa Socrates Man) (ForAll ?x (implies
(isa ?x Man) (isa ?x Mortal))) (isa Socrates
Mortal) Harder Using general and specific
knowledge Can a can can-can? Very complex
An example from our AKB (Analysts Knowledge Base)
53
The Analysts Knowledge Base
CT Analyst
"What sequences of events could lead to the
destruction of Hoover Dam?"
Were there any attacks on targets of symbolic
value to Muslims since 1987 on a Christian holy
day?"


Domain Experts
Scenario
Explanation
Query
Scenario
Explanation
Query
Generation
Generation
Formulation
Generator
Generator
Formulator
Others/GOTS
Cycorp Tools For Ontology-Building, -Browsing,
-Editing, Fact/Rule Entry
Analysis and
General Knowledge
Collaboration
Components
Terrorism Knowledge
AKB
OWL
Relational DB projection of the AKB
54
Logically and Arithmetically Combining n Pieces
of Info.
)
(

• An example an analysts query posed as
• part of HPKB (1996) that Cyc answered.

Information from multiple sources Knowledge about
the domain in general Commonsense knowledge about
the real world
55
There is no one correct monolithic ontology.
E.g., Cycs 3M axioms are divided into thousands
of contexts by granularity, topic, culture,
geospatial place, time,...
There is a correct monolithic reasoning
mechanism, but it is so deadly slow that we never
call on it unless we have to
E.g., the Cyc inference engine is a community of
720 agents that attack every problem and,
recursively, every subproblem (subgoal). One
of these 720 is a general theorem prover the
others have special-purpose data
structures/algorithms to handle the most
important, most common cases, very fast.
56
Even though they are expressed in formal logic,
most axioms state usuals, not absolute truths.
Nonmonotonic (later information can show that
something you earlier believed is false after
all). So the reasoning is default
(argumentation gather up all the pro/con
arguments, and compare them).
Syria was behind the assassination of Rafik
Hariri.
Each person had a mother who was also a person.
57
Cyc A Large Formal Ontology

• Represented in
• First Order Logic
• Higher Order Logic
• Context Logic
• Micro-theories

Cyc contains 15,000 Predicates 300,000 Concept
s 3,200,000 Assertions
General Knowledge about Various Domains
Specific data, facts, and observations


58
(No Transcript)
59
Temporal Relations
37 Relations Between Temporal Things

• temporalBoundsContain
• temporalBoundsIdentical
• startsDuring
• overlapsStart
• startingPoint
• simultaneousWith
• after

• temporalBoundsIntersect
• temporallyIntersects
• startsAfterStartingOf
• endsAfterEndingOf
• startingDate
• temporallyContains
• temporallyCooriginating

60
Temporal Relations
Ariel Sharon was in Jerusalem during 2005 with
granularity calendar-week
Condoleezza Rice made a ten-day trip to
Jerusalem in February of 2005
61
Senses of Part

• parts
• intangibleParts
• subInformation
• subEvents
• physicalDecompositions
• physicalPortions

62
Senses of Part
Concepts in mereotopology X is part of Y
X overlaps Y X is connected to Y
X is is the sum the objects Y1Yn
63
23 Senses of In

• Can the inner object leave by passing between
  members of the outer group?
• Yes -- Try in-Among

64
23 Senses of In

• If the container were turned around could the
  contained object fall out?

• Does part of the inner object stick out of the
  container?
• None of it. -- Try
• in-ContCompletely
• Yes -- Try
• in-ContPartially

• Yes -- Try
• in-ContOpen

• No -- Try
• in-ContClosed

65
23 Senses of In

• Is it attached to the inside of the outer
  object?
• Yes -- Try connectedToInside

• Can it be removed, if enough force is used,
  without damaging either object?
• Yes -- Try in-Snugly or screwedIn

• Does the inner object stick into the outer
  object?
• Yes -- Try sticksInto

66
Event Types
11,000 more
67
Relations Between an Event and its Participants
Over 400 more.
68
Emotion

• Types of Emotions
• Adulation
• Abhorrence
• Relaxed-Feeling
• Gratitude
• Anticipation-Feeling
• Over 120 of these

• Predicates For Defining and Attributing Emotions
• contraryFeelings
• appropriateEmotion
• actionExpressesFeeling
• feelsTowardsObject
• feelsTowardsPersonType

69
Propositional Attitudes Relations Between Agents
and Propositions

• goals
• intends
• desires
• hopes
• expects
• beliefs

• opinions
• knows
• rememberedProp
• perceivesThat
• seesThat
• tastesThat

Most of these are modal and assertions using them
go beyond 1st-order logic
70
Devices

• Over 4000 Specializations
• of PhysicalDevice
• ClothesWasher
• NuclearAircraftCarrier
• Vocabulary for Describing Device Functions
• primaryFunction-DeviceType

• Device Specific
• Predicates
• gunCaliber
• speedOf
• Device States (40)
• DeviceOn
• CockedState

71
Building Cyc qua Engineering Task
learning by discovery
learning via natural language
1984
2004
2006
codify enter each piece of knowledge, by hand
CYC
750 person-years 21 realtime years 75 million
72
AKA by Shallow Fishing
Automated Knowledge Acquisition
(foundingDate AbuSayyaf ?X)
73
AKA by Shallow Fishing
Automated Knowledge Acquisition

• Abu Sayyaf was founded in ___
• Al Harakat Islamiya, established in ___
• ASG was established in ___

Search Strings
(foundingDate AbuSayyaf ?X)
74
AKA by Shallow Fishing
Automated Knowledge Acquisition

• Abu Sayyaf was founded in ___
• Al Harakat Islamiya, established in ___
• ASG was established in ___

Search Strings
(foundingDate AbuSayyaf ?X)
75
AKA by Shallow Fishing
Automated Knowledge Acquisition

• Abu Sayyaf was founded in ___
• Al Harakat Islamiya, established in ___
• ASG was established in ___

Search Strings
(foundingDate AbuSayyaf ?X)
Abu Sayyaf was founded in the early 1990s
?
Parse (foundingDate AbuSayyaf (EarlyPartFn
(DecadeFn 199)))
76
AKA by Shallow Fishing
(maritalStatus MohamedAtta?X)
PersonTypeByMaritalStatus
77
AKA by Shallow Fishing

• (maritalStatus MohamedAtta Single)
• (maritalStatus MohamedAtta Married)
• (maritalStatus MohamedAtta Divorced)
• (maritalStatus MohamedAtta Cohabitating-Unmarried)

Generate alternative assertions
(maritalStatus MohamedAtta?X)
PersonTypeByMaritalStatus
78
AKA by Shallow Fishing
For each one, generate a set of search strings

• (maritalStatus MohamedAtta Single)
• (maritalStatus MohamedAtta Married)
• (maritalStatus MohamedAtta Divorced)
• (maritalStatus MohamedAtta Cohabitating-Unmarried)

• Mohamed Attas fiancee
• Mohamed Attas wife
• Mohammed Attas ex-wife
• husband, Mohamed Atta,

Generate alternative assertions
(maritalStatus MohamedAtta?X)
PersonTypeByMaritalStatus
79
AKA by Shallow Fishing
For each one, generate a set of search strings

• (maritalStatus MohamedAtta Single)
• (maritalStatus MohamedAtta Married)
• (maritalStatus MohamedAtta Divorced)
• (maritalStatus MohamedAtta Cohabitating-Unmarried)

• Mohamed Attas fiancee
• Mohamed Attas wife
• Mohammed Attas ex-wife
• husband, Mohamed Atta,

Generate alternative assertions
(maritalStatus MohamedAtta?X)
PersonTypeByMaritalStatus
(maritalStatus MohamedAtta Married)
80
Harnessing Lots of Users
WWW.CYC.COM

• Identify underpopulated common sense predicates
• Use semantic constraints shallow parsing to
  identify possible fact completions
• Present multiple choice questions to novices to
  complete facts

150-400 commonsense GAFs/hour
useful distinguishing facts
Hat worn on ? Head ? Neck ? Foot ? Leg
81
(No Transcript)
82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
I.e., share a formal ontology, including a full
upper ontology, large portions of a middle
ontology, and relevant slivers of a lower
(domain-dependent) ontology.
What Needs to be Shared?

• bits/bytes/streams/network
• alphabet, special characters,
• words, morphological variants,
• syntactic meta-level markups (HTML)
• semantic meta-level markups (SGML, XML)
• content (logical representation of doc/page/...)
• context (common sense, recent utterances, and n
  dimensions of formal ontological knowledge time,
  space, level of granularity, the sources
  purpose, etc.)

87
I.e., share a formal ontology, including a full
upper ontology, large portions of a middle
ontology, and relevant slivers of a lower
(domain-dependent) ontology.
Upper Ontology Symposium
Formal Ontologies
Can Anything With That Title be Understandable or
Interesting?

• Dr. Douglas B. Lenat
• , 3721 Executive Center Drive,
  Suite 100, Austin, TX 78731
• Email Lenat_at_cyc.com
• Phone [phone]

2 July 2005