An Extended Model of Natural Logic

1
An Extended Model of Natural Logic

• Bill MacCartney and Christopher D. Manning
• NLP Group
• Stanford University
• 8 January 2009

2
Natural language inference (NLI)
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Aka recognizing textual entailment (RTE)
• Does premise P justify an inference to hypothesis
  H?
• An informal, intuitive notion of inference not
  strict logic
• Emphasis on variability of linguistic expression

P Every firm polled saw costs grow more than
expected,even after adjusting for
inflation. H Every big company in the poll
reported cost increases. yes

• Necessary to goal of natural language
  understanding (NLU)
• Can also enable semantic search, question
  answering,

3
NLI a spectrum of approaches
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
Solution?
Problemhard to translate NL to FOL idioms,
anaphora, ellipsis, intensionality, tense,
aspect, vagueness, modals, indexicals,
reciprocals, propositional attitudes, scope
ambiguities, anaphoric adjectives,
non-intersective adjectives, temporal causal
relations, unselective quantifiers, adverbs of
quantification, donkey sentences, generic
determiners, comparatives, phrasal verbs,
Problemimprecise ? easily confounded by
negation, quantifiers, conditionals, factive
implicative verbs, etc.
4
What is natural logic? (? natural deduction)
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Characterizes valid patterns of inference via
  surface forms
• precise, yet sidesteps difficulties of
  translating to FOL
• A long history
• traditional logic Aristotles syllogisms,
  scholastics, Leibniz,
• modern natural logic begins with Lakoff (1970)
• van Benthem Sánchez Valencia (1986-91)
  monotonicity calculus
• Nairn et al. (2006) an account of implicatives
  factives
• We introduce a new theory of natural logic
• extends monotonicity calculus to account for
  negation exclusion
• incorporates elements of Nairn et al.s model of
  implicatives

5
16 elementary set relations
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
Assign sets ?x, y? to one of 16 relations,
depending on emptiness or non-emptiness of each
of four partitions
?y
y
?x
x
empty
non-empty
6
16 elementary set relations
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
But 9 of 16 are degenerate either x or y is
either empty or universal. I.e., they correspond
to semantically vacuous expressions, which are
rare outside logic textbooks. We therefore focus
on the remaining seven relations.
7
The set of 7 basic semantic relations
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
Relations are defined for all semantic types
tiny ? small, hover ? fly, kick ? strike,this
morning ? today, in Beijing ? in China,
everyone ? someone, all ? most ? some
8
Joining semantic relations
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
x
y
y
z
?


9
Some joins yield unions of relations!
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
10
The complete join table
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Of 49 join pairs, 32 yield relations in 17
  yield unions

Larger unions convey less information limits
power of inference
In practice, any union which contains can be
approximated by
11
Lexical semantic relations
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
x
e(x)

• ????????????? will depend on
• the lexical semantic relation generated by e
  ?(e)
• other properties of the context x in which e is
  applied

?( , )
Example suppose x is red car If e is SUB(car,
convertible), then ?(e) is ? If e is DEL(red),
then ?(e) is ? Crucially, ?(e) depends solely on
lexical items in e, independent of context x But
how are lexical semantic relations determined?
12
Lexical semantic relations SUBs
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• ?(SUB(x, y)) ?(x, y)
• For open-class terms, use lexical resource (e.g.
  WordNet)
• ? for synonyms sofa ? couch, forbid ? prohibit
• ? for hypo-/hypernyms crow ? bird, frigid ?
  cold, soar ? rise
• for antonyms and coordinate terms hot cold,
  cat dog
• ? or for proper nouns USA ? United States,
  JFK FDR
• for most other pairs hungry hippo
• Closed-class terms may require special handling
• Quantifiers all ? some, some no, no all,
  at least 4 ? at most 6
• See paper for discussion of pronouns,
  prepositions,

13
Lexical semantic relations DELs INSs
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Generic (default) case ?(DEL()) ?, ?(INS())
  ?
• Examples red car ? car, sing ? sing off-key
• Even quite long phrases car parked outside since
  last week ? car
• Applies to intersective modifiers, conjuncts,
  independent clauses,
• This heuristic underlies most approaches to RTE!
• Does P subsume H? Deletions OK insertions
  penalized.
• Special cases
• Negation didnt sleep did sleep
• Implicatives factives (e.g. refuse to, admit
  that) discussed later
• Non-intersective adjectives former spy spy,
  alleged spy spy
• Auxiliaries etc. is sleeping ? sleeps, did
  sleep ? slept

14
The impact of semantic composition
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• How are semantic relations affected by semantic
  composition?

The monotonicity calculus provides a partial
answer
If f has monotonicity
But how are other relations (, , ?) projected?
15
A typology of projectivity
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Projectivity signatures a generalization of
  monotonicity classes

In principle, 77 possible signatures, but few
actually realized
16
A typology of projectivity
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Projectivity signatures a generalization of
  monotonicity classes
• Each projectivity signature is a map
• In principle, 77 possible signatures, but few
  actually realized

See paper for projectivity of various
quantifiers, verbs
17
Projecting through multiple levels
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
Propagate semantic relation between atoms upward,
according to projectivity class of each node on
path to root
nobody can enter with a shirt ? nobody can enter
with clothes
18
Implicatives factives Nairn et al. 06
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
9 signatures, per implications (, , or o) in
positive and negative contexts
19
Implicatives factives
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
We can specify relation generated by DEL or INS
of each signature
Room for variation w.r.t. infinitives,
complementizers, passivation, etc.
Some more intuitive when negated he didnt
hesitate to ask he didnt ask
Factives not fully explained he didnt admit
that he knew he didnt know
20
Putting it all together
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• Find a sequence of edits ?e1, , en? which
  transforms p into h. Define x0 p, xn h, and
  xi ei(xi1) for i ? 1, n.
• For each atomic edit ei
• Determine the lexical semantic relation ?(ei).
• Project ?(ei) upward through the semantic
  composition tree of expression xi1 to find the
  atomic semantic relation ?(xi1, xi)
• Join atomic semantic relations across the
  sequence of edits?(p, h) ?(x0, xn) ?(x0,
  x1) ? ? ?(xi1, xi) ? ? ?(xn1, xn)

Limitations need to find appropriate edit
sequence connecting p and htendency of ?
operation toward less-informative semantic
relations lack of general mechanism for
combining multiple premises Less deductive power
than FOL. Cant handle e.g. de Morgans Laws.
21
An example
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
P The doctor didnt hesitate to recommend
Prozac. H The doctor recommended
medication. yes
?




?
?
?
?
yes
22
Different edit orders?
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
Intermediate steps may vary final result is
typically (though not necessarily) the same
23
Implementation evaluation
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion

• The NatLog system an implementation of this
  model in code
• For implementation details, see MacCartney
  Manning 2008
• Evaluation on FraCaS test suite
• 183 NLI problems, nine sections, three-way
  classification
• Accuracy 70 overall 87 on relevant sections
  (60 coverage)
• Precision 89 overall rarely predicts entailment
  wrongly
• Evaluation on RTE3 test suite
• Longer, more natural premises greater diversity
  of inference types
• NatLog alone has mediocre accuracy (59) but good
  precision
• Hybridization with broad-coverage RTE system
  yields gains of 4

24
Conclusion
Introduction Semantic Relations Joins
Lexical Relations Projectivity
Implicatives Inference Evaluation
Conclusion
Natural logic is not a universal solution for
NLI Many types of inference not amenable to
natural logic approach Our inference method faces
many limitations on deductive power More work to
be done in fleshing out our account Establishing
projectivity signatures for more quantifiers,
verbs, etc. Better incorporating
presuppositions But, our model of natural logic
fills an important niche Precise reasoning on
negation, antonymy, quantifiers, implicatives,
Sidesteps the myriad difficulties of full
semantic interpretation Practical value
demonstrated on FraCaS and RTE3 test suites

• Natural logic is not a universal solution for NLI
• Many types of inference not amenable to natural
  logic approach
• Our inference method faces many limitations on
  deductive power
• More work to be done in fleshing out our account
• Establishing projectivity signatures for more
  quantifiers, verbs, etc.
• Better incorporating presuppositions
• But, our model of natural logic fills an
  important niche
• Precise reasoning on negation, antonymy,
  quantifiers, implicatives,
• Sidesteps the myriad difficulties of full
  semantic interpretation
• Practical value demonstrated on FraCaS and RTE3
  test suites