Approximating Textual Entailment with LFG and FrameNet Frames

1
Approximating Textual Entailment with LFG and
FrameNet Frames

• Aljoscha Burchardt, Anette Frank
• Computational Linguistics Department
• Saarland University, Saarbrücken
• Second Pascal Challenge Workshop
• Venice, April 2006

2
Outline of this Talk

• Frame Semantics
• A baseline system for approximating Textual
  Entailment
• LFG syntactical analyses with
• Frame semantics
• Statistical decision entailed?
• Walk-through example from RTE 2006
• RTE 2006 results / brief conclusions

3
Frame Semantics (Fillmore 1976, Fillmore et. al.
2003)

• Lexical semantic classification of predicates and
  their argument structure
• A frame represents a prototypical situation (e.g.
  Commercial_transaction, Theft, Awareness)
• A set of roles identifies the participants or
  propositions involved
• Frames are organized in a hierarchy
• Berkeley FrameNet Project db 600 frames, 9.000
  lexical units, 135.000 annotated sentences

4
Linguistic Normalizations(Frame Commerce_buy)
5
Frame Semantics for RTE

• Focusing on lexical semantic classes and
  role-based argument structure
• Built-in normalizations help to determine
  semantic similarity at a high level of
  abstraction
• Disregarding aspects of deep semantics
  negation, modality, quantification, ...
• Open for deeper modeling on demand (e.g. our
  treatment of modality)

6
A Baseline System for Approximating Textual
Entailment

• Fine-grained LFG-based syntactic analysis
• English LFG grammar (Riezler et al. 2002)
• Wide-coverage with high-quality probabilistic
  disambiguation
• Frame Semantics
• Shallow lexical-semantic classification of
  predicate-argument structure
• Extensions WordNet senses, SUMO concepts
• Computing structural and semantic overlap of t
  and h
• Hypothesis large overlap entailment

7
A Baseline System for Approximating Textual
Entailment
Computing Semantic Overlap
Linguistic Analyses
Model training classification
Statistical Decision Entailment?
8
Linguistic Components
XLE parsing LFG f-structure
WordNet-based WSD WordNet SUMO
Fred / Detour / Rosy frames roles
F-structure w/ semantics projection
Using XLE term rewriting system (Crouch 2005)

• Rule-based extend refine sem. proj.
• NEs, Locations
• Co-reference
• Modality, etc.

9
Example from RTE 2006

• Pair 716
• Text
• In 1983, Aki Kaurismäki directed his first
  full-time feature.
• Hypothesis
• Aki Kaurismäki directed a film.

10
LFG F-Structures
11
Automatic Frame Annotation for Text (SALTO
Viewer)
Collins Parse
12
Automatic Frame Annotation for Hypothesis

• 716_h Aki Karusmäki directed a film.

13
LFG Frames for Hypothesis(FEFViewer)
Aki Kaurismäki directed a film.
14
Hypothesis-Text-Match Graphs Computing Structural
and Semantic overlap

• Match graph bundles overlapping partial graphs
  marked by match types
• Aspects of similarity
• Syntax-based (i.e. lexical and structural)
  Identical predicates (attributes) trigger node
  (edge) matches.
• Semantics-based Identical frames/concepts
  (roles) trigger node (edge) matches.
• Degrees of similarity
• Strict matching
• Weak matching conditions for non-identical
  predicates
• Structurally related e.g. via coreference
  (relative clauses, appositives, pronominals)
• Semantically related via WordNet,
  Frame-Relations

15
t In 1983, Aki Kaurismäki directed his first
full-time feature.
16
Statistical Modeling

• Feature extraction on the basis of
• Syntactic, Semantic matches (of different types)
• Matching clusters sizes
• Ratio (matched vs. hypothesis)
• (Non-)matching modality
• RTE-task, fragmentary (parse),
• Training/classification with WEKA tool
• Feature selection
• Predicate Matches
• Frame overlap
• Matching cluster size
• Model 1 Conjunctive rule (Feat. 1,2)
• Model 2 LogitBoost (Feat. 1,2,3)

17
RTE 2006 Results

• SUM (and IR) are natural tasks for Frame
  Semantics, IE and QA need more deeper modeling
  (aboutness vs. factivity)
• Error analysis
• True positives high semantic overlap
• True negatives 27 involve modality mismatches
• False examples poor modeling of dissimalrity
• Many high-frequency features measuring similarity
• Few low-frequency features measuring dissimilarity

18
Brief Conclusions

• Good approximation of semantic similarity
• Deep LFG syntactical analyses integrated with
• Shallow lexical Frame Semantics (plus other lex.
  resources)
• Match graph measuring overlap
• Need better model for semantic dissimilarity
• Too few rejections (false positives gtgt false
  negatives)
• Towards deeper modeling
• Treatment of modal contexts
• Integration of lexical inferences
• Open for collaborations

19
LFG Frames for Hypothesis (FEF)