Comparing VerbNet and Cyc: Automated Methods

1
(No Transcript)
2
Comparing VerbNet and Cyc Automated Methods

• Derek Trumbo
• Institute of Cognitive Science Colloquium
• 2006.9.29

3
Itinerary

• Background
• Motivation
• VerbNet
• Cyc
• Manual Mapping
• Process
• Results
• Automatic Mapping
• The Inspector
• Results

4
Motivation

• Many NLP Software Packages
• WordNet
• FrameNet
• VerbNet
• PropBank
• TreeBank
• Cyc
• etc.

5
Motivation

• Each has its own purpose
• Useful to map two such programs
• Gain advantages that both offer
• Cyc clear candidate

6
VerbNet

• Class-based English verb lexicon
• Created in 2005
• Based off of Levins 1993 book
• Contains 4991 verbs organized into 431 classes
  and subclasses
• Classes contain frames that have syntactic and
  semantic information

7
VerbNet Structure of a Class

• mine-10.9 has
• 2 members (verbs)
• mine
• quarry
• 3 thematic roles
• Agent (animate, organization), Theme (concrete),
  Source (location)
• 2 frames
• Basic Transitive
• PP source-PP
• This class has 2 members x 2 frames 4 VF pairs

8
VerbNet Structure of a Class

• mine-10.9 has
• 2 members (verbs)
• mine
• quarry
• 3 thematic roles
• Agent (animate, organization), Theme (concrete),
  Source (location)
• 2 frames
• Basic Transitive
• PP source-PP
• This class has 2 members x 2 frames 4 VF pairs

9
VerbNet Frame Makeup

• The two frames for mine-10.9 are
• Basic Transitive
• Example The men mined the gold.
• Syntax Agent V Theme
• Semantics cause(Agent, E),
  location(start(E), Theme, ?Source), etc.
• PP source-PP
• Example The men mined the gold from the hill.
• Syntax Agent V Theme source prepos Source
• Semantics cause(Agent, E), etc.

10
VerbNet UVI

• http//verbs.colorado.edu/verb-index

11
Cyc

• General knowledge base and common sense reasoning
  engine
• Created in 1993
• Contains assertions about the world in which we
  live
• All in second-order predicate logic
• Enormous!
• http//opencyc.org http//cyc.com

12
Cyc

• What does Cyc know?
• Math
• Human Activities
• Law
• Ecology
• Software
• Astronomy
• Much more

13
Cyc
14
Cyc
15
Cyc
16
Manual Mapping (MM)

• Important for two reasons
• Have baseline for measuring success of automatic
  matching (AM) constraints
• Get hands-on experience with data

17
Verb-Frame Pairs

• Enumerate all frames in which a verb can
  participate
• A class with 10 members, 5 frames would have 50
  Verb-Frame (VF) pairs
• Each pair has the possibility of matching to one
  or more Cyc rules
• 29,245 total Verb-Frame pairs in VerbNet!!

18
Anatomy of a Match

• Take the verb mine in class mine-10.9
• Class has two frames
• Basic Transitive
• The men mined the gold.
• PP source-PP
• The men mined the gold from the hill.
• 2 total VF pairs for this verb in this class

19
Anatomy of a Match
? Match ? VN mine / f1 (Agent V Theme) The men
mined the gold. cause(Agent, E)
location(start(E), Theme, ?Source)
not(location(end(E), Theme, ?Source)) Cyc
RULE 2832 (verbSemTrans Mine-TheWord 0
TransitiveNPFrame (and (isa
ACTION Mining) (objectOfStateChange
ACTION OBJECT) (doneBy ACTION
SUBJECT)))
? Match ? VN mine / f2 (Agent V Theme source
prepos Source) The men mined the gold from the
mine. cause(Agent, E) location(start(E),
Theme, Source) not(location(end(E), Theme,
Source)) Cyc RULE 1086 (verbSemTrans
Mine-TheWord 0 (PPCompFrameFn
DitransitivePPFrameType From-TheWord)
(and (isa ACTION Mining)
(objectOfStateChange ACTION OBLIQUE-OBJECT)
(doneBy ACTION SUBJECT)
(objectRemoved ACTION OBJECT)))

• Cyc has 3 rules for the verb mine
• Thus, there are 6 VF-Cyc Naïve Matches
• A match where just the verb of the VF pair equals
  the verb in the Cyc rule
• Lemma Matching

20
Anatomy of a Match

• Results for just this one verb, mine

21
Manual Mapping Results
OKed 137
Discarded 266
(out of from 403 VF-Cyc Naïve Matches)
22
Now

• Armed with the experience from the manual
  mapping, we simply automate the process!

23
The Inspector

• Combination VerbNet viewer and Cyc matcher
• Easy to install new constraints as they occur to
  me
• Takes as input
• All VerbNet XML files
• All Cyc rules
• Cyc Manual Mapping file
• Produces
• List of matches
• Statistics measuring against MM

24
Results Situations

• The Good

25
Results Situations

• The Bad

26
The Constraints

• These can be thought of as when-to-discard rules
• n - naïve match, matches on lemma only
• p - if Cyc rule has a preposition, VN frame must
  have that preposition after the verb
• t - transitivity must match
• pt - apply both p and t constraints

27
Results Match Constraint n
28
Results Match Constraint p
A broad, powerful constraint incorrectly
discards zero matches
29
Results Match Constraint t
30
Results Match Constraint pt
31
Percent Coverage Metric

• OK-OK and Discard-Discard equally important
• Percent AM classified same as MM in these two
  categories

32
Questions?
33
(No Transcript)