Hybrid Text Summarization Combining External Relevance Measures with Structural Analysis

1
Hybrid Text Summarization Combining External
Relevance Measureswith Structural Analysis

• Gian Lorenzo Thione, Martin van den Berg, Livia
  Polanyi and Chris Culy
• FX Palo Alto Laboratory
• July 25, 2004

2
PALSUMM Project

• Goal High quality document summaries
• Maintain style and formatting of original text
• Coherent and readable
• Maintain information for correct semantic
  interpretation of summary
• No missing anaphors, dangling references etc.
• Corpus 300 FXPAL Technical Reports
• Many technical domains, authors, intended
  purposes
• 3-50 pages in length
• Qualitative Evaluation by external Reader Panel
• 6 point scale Readability, Clarity, Overall User
  Satisfaction
• Phase 1 Gold-standard 5 Manually Annotated docs
  (3 domains)
• Results MEAD 2.99, Symbolic 4.70 Hybrid 4.49
  
• Phase 2 10 automatically analyzed docs (3GS
  7TR 8 domains)
• Results PALSUMM Symbolic 4.12 PALSUMM Hybrid
  4.04

3
PALSUMM Approach

• Tokenize document and parse sentences to identify
  discourse segments
• Build up a representation of structure of
  document using the Unified Linguistic Discourse
  Model
• Extract text segments based on Discourse
  Structure using statistical relevance and
  symbolic context

4
U-LDM Discourse Parsing Discourse
Segments

• D-Segments are defined in formal semantic terms
• To identify minimal semantic anchors for
  discourse continuation
• Syntactic reflexes of minimum units of content or
  function
• Content Segments independently establish an
  interpretation context
• Introduce exactly one discourse referent to
  events or states
• In Davidsonian style semantics, quantification
  over an event variable.
• Function Segments encode information relating
  segments in a context
• structurally, semantically, interactionally or
  rhetorically
• D-Segments are realized syntactically
• Basic Discourse Units (BDU) minimal semantic
  anchors
• Clauses, some other verb based structures, titles
  with both content and indexical information are
  segments
• Gerunds, nominalizations, clefts do not
  independently establish an interpretation context
  and are not segments
• Operator Segments
• Discourse connectives, cue phrases, connectives
  etc

D-Segments need not be contiguous spans
5
U-LDM Discourse Parsing

• The enterprise of Discourse Parsing
• Construct a structural representation of the
  relationships among the d-segments of a text
• U-LDM Structures
• Discourse Constituent Trees
• Right edge open for discourse continuation
• U-LDM Relations
• Semantically motivated
• Recursively create contexts from simpler units

6
U-LDM Discourse Parsing Discourse Relations

• Japanese people eat noodles for lunch or a snack.
• Noodles are served in a hot soup or cold like a
  salad.
• When noodles are served in a hot soup,
• vegetables, tofu, and meat may also be in the
  soup.
• Chinese people eat noodles too.

• Subordination Node (S)
• One child (normally the left) dominates the
  other.
• Relationship is elaboration or interruptions
• Node-extension inherits content dominating child
  node
• Coordination Node (C)
• Children bear similar relationship to more
  abstract concept
• Relationship is list
• Node-extension inherits form/content common to
  all child nodes
• N-ary Nodes (N)
• Discourse constructions may be language and
  culture specific.
• Interactional, contextual, logical,
  genre-specific
• Node-extension inherits combined content of child
  nodes

7
U-LDM Discourse Parsing

• Formal rules specify how d-segments relate using
• syntactic information
• syntactic structure, grammatical functions of
  related lexemes
• Syntactic Role Promotion (e.g. OBJ..SUBJ) ?
  Subordinate
• Syntactic Construction Parallelism ? Coordinate
• semantic information
• realis status, genericity, tense, aspect, point
  of view etc.
• Switch Modality Status ? Subordinate
• lexical information
• same lexeme, synonym/antonym, hypernym, etc.
• constituents of incomplete n-ary constructions
• Question-Type d-seg Answer-Type d-seg ? N-Ary
  (Q/A)
• relation to available structural context

8
U-LDM Discourse Parsing Discourse Structure
Representation

• Sentence Level BDU-Tree
• BDU-Tree represents discourse relations among
  d-segments
• One Main-BDU and associated BDUs and preposed
  modifiers.
• main clauses or structures of sentence or
  fragment
• attachment point of S to DPT
• characteristics of Main-BDU govern attachment
  relation
• Attachment relations governed by rules of
  sentential syntax modified by rules of discourse
  structure
• Attachment point governed by information
  throughout sentence
• Text Level Discourse Parse Tree (DPT)
• DPT represents text level discourse relationships
• DPT nodes all first class citizens with fully
  computable content
• Attachment point and relation of BDU-Tree to DPT
  governed by rules of discourse structure

9
Sentence Level discourse Grammar
Text Level discourse Grammar
LiveTree
Discourse tree
Sentence Breaker
Sentence Level Discourse Parser
Text Level Discourse Parser
HTML document
LDM Engine
XML
TEXT
XML
XML
Segmentation Webservice
Ontology Webservice
Sentence Level Discourse Segmenter
Lexical Ontology Server
XLE Server
Lexical Ontology Lookup
XLE Parser
WordNet lookup
Flat Ontology Files
DAML Encoded ontology
English Grammar
10
Summarization Approach

• Create structure of text
• U-LDM Symbolic Discourse Parsing ?
• Determine Relevance Score R for all nodes
• Assign Statistical Semantic Salience Seeds
• Assign Structural Salience Seeds
• Distribute and Compound Seeds in a final
  relevance score
• Prune text by cutoff threshold on R
• Optionally apply sentence compression

11
Sentence Level discourse Grammar
Text Level discourse Grammar
HTML document
Filter
LiveTree
Discourse tree
Sentence Breaker
Sentence Level Discourse Parser
Text Level Discourse Parser
HTML document
Hybrid Summarizer
LDM Engine
XML
TEXT
XML
XML
SOAP
Segmentation Webservice
Ontology Webservice
MEAD Statistical Summarizer Webservice
Sentence Level Discourse Segmenter
Lexical Ontology Server
XLE Server
Lexical Ontology Lookup
XLE Parser
PALSUMM

WordNet lookup
Flat Ontology Files
DAML Encoded ontology
English Grammar
12
Overview

• Determine a relevance score R,
• R(n) gt R(m) guarantees
• m carries no contextual information for correct
  semantic interpretation of n
• n does not include references to referents that
  are only available in m
• R(n) gt R(m) suggests
• n is Semantically more relevant than m
• Omitting m will omit less relevant information
  than omitting n

13
Statistical Seeding

• Semantic Salience S(n)
• Approximates the semantic importance of a segment
  for inclusion in a summary
• S-Scores assigned to each D-Segment
• MEAD (Radev at al, 2003)
• Positional, centroid, Re-ranking
• Optionally skewed with query
• Cue Words (Hirschberg, 1993)
• Normalized to maxn(S(n))1

14
Structural Seeding
D1
D1
D2

• Structural Salience
• Assign subordination level to every node n
• Absolute Depth D(n)
• D(Root) 1
• Add number of subordination nodes dominating n
• Embedding Branch Weight W(n)
• Bootstrap
• W(leaf) D(leaf) and W(nonleaf) 0

D2
D3
D4
D3
15
Global Tree Scoring

• Percolate S(n), W(n) scores
• Starting from leaves
• Percolate to parent node
• Trickle to non-subordinated children
• Higher scores replace existing ones
• Each node is assigned
• S(n) Percolated Statistical Salience
• D(n) Absolute Embedding Depth
• W(n) Percolated Embedding Branch Weight
• T(n) 1 (D(n) 1)/W(n) Compound Structural
  Score
• Final Relevance Score (R)
• RS T HybReduce-R
• misprinted in paper

16
Summary Generation

• Choose a threshold R0
• Prune the tree of nodes R(n) lt R0
• Regenerate the text
• PALSUMM From leaves only
• at least one child node always has the same score
  of its parent
• Future From intermediate nodes
• Sentence Compression based on LDM N-aries
• NL Generation from LDM Non-terminal Nodes

17
One of the technical reports.
18
Qualitative Evaluation

• External panel of readers evaluated Summaries
• 6 point scale Readability, Clarity, Overall User
  Satisfaction
• Phase 1 Gold-standard 5 Technical Reports (3
  domains)
• Manually Annotated with U-LDM Discourse Structure
• Automatically Summarized at 35
• Purely Symbolic Hybrid Algorithms)
• Base Standard MEAD Summaries (35)
• Evaluation Panel of 12 readers each read 4
  summaries
• Results MEAD 2.99, Symbolic 4.70 Hybrid 4.49
• Phase 2 10 documents (3 GS 7 TR 8 domains)
• Automatically Parsed with LIDAS
• Grammars tuned on 2 GS documents
• Automatically Summarized with PALSUMM at 35
• PALSUMM Hybrid and Symbolic Algorithms
• Evaluation Panel of 20 external readers read 4
  summaries each
• Results PALSUMM Symbolic 4.12 PALSUMM Hybrid
  4.04

19
Conclusions

• Rule-Based Symbolic Discourse Analysis
• Feasible for off-line analysis
• Scalable and Extensible
• LIDAS/PALSUMM built in 2 person-years
• Discourse Grammar / Ontological Resources (4
  person-months)
• Different Languages (XLE PARGRAM 8 Languages)
• Document Types (Genre Rules)
• PALSUMM Summarization
• Acceptable Readability (4 on 6-point scale)
• Hybrid Method compared to Structural Extraction
• Often Less Readable
• Arguably More Informative

20

• The End!