A Hybrid Approach to Answering BiographicalDefinitional Questions - PowerPoint PPT Presentation

Title: A Hybrid Approach to Answering BiographicalDefinitional Questions


1
A Hybrid Approach to Answering Biographical/Defini
tional Questions

• Ana Licuanan, Scott Miller, Ralph Weischedel,
  Jinxi Xu

10 June 2003
2
System Diagram
Question
List of Responses
List of Sentences
3
Approaches

• Sentence selection
• By Information Retrieval
• By linguistic features (appositives, copulas and
  propositions)
• Using features from Information Extraction
  (co-refs and relations)
• Sentence compression
• Redundancy removal

4
Sentence Selection by IR

• Hypothesis good sentences tend to contain words
  that are common in human created definitions
• Each sentence is treated as a query and ranked
  against the training data
• The score is normalized using the geometric mean
  of the IR score to compensate for differences in
  sentence length
• Training Data
• 17,000 short biographies from www.s9.com
• 6,000 online encyclopedia biographies from
  www.wikipedia.org
• Top N sentences containing the question term is
  returned as the answer

5
Sentence Selection using Linguistic Features

• Hypothesis good sentences tend to contain
  linguistic features that are common in human
  generated answers
• Features include propositions, appositives and
  copulas extracted from parse trees
• Headword is used to represent argument value to
  reduce data sparseness
• Example Blobel, a biologist at Rockefeller
  University, won the Nobel Prize in Medicine.
• Proposition ltsubgtQTERM ltverbgtwon
  ltobjgtprize
• Appositive ltappositivegtbiologist
• Sentence with a feature f is ranked by
• P(Rf)P(R) P(fR) / P(f)
• P(fR) estimated from training data
• P(f) estimated from a background corpus

6
Sentence Selection based on Information Extraction

• Hypothesis good sentences tend to contain
  semantic relations that are common in human
  generated answers
• SERIF, a state of art Information Extraction
  Engine was used
• Co-ref was used for making inferences
• Relations were used as additional features for
  sentence selection. Types of relations include
• Spouse-of (e.g. Clinton, Hillary)
• Founder-of (e.g. Gates, Microsoft)
• Management-of (e.g. Welch, GE)
• Residence-of (e.g. John Doe, Boston)
• Citizenship-of (e.g. John Doe, American)
• Staff-of (e.g. Weischedel, BBN)

7
Redundancy Removal

• Hypothesis Variations of surface strings make
  redundancy detection difficult. The difficulty
  can be mitigated by the use of linguistic and
  semantic features
• Each response item (a sentence or phrase) is
  decomposed into a set of features
• Features appositives/copulas, propositions and
  relations
• All candidate items are ranked based on features
  (as discussed before)
• For each response item,
• Output it if it contains new features
• Skip it if it does not

8
Sentence Compression

• Motivation A good sentence may contain pieces of
  irrelevant materials to the question
• Goal extract only the pertinent parts of a
  sentence
• Strategy
• Find the smallest phrase that contains all the
  arguments of an important fact (i.e.
  proposition/appositive/copula/relation)
• Relative clauses not attached to the question
  term are trimmed from phrase
• Operations are performed on parse trees

9
Case study Who is Guenter Blobel?
10
Sentences Selected by IR

• Blobel was born in 1936 in Waltersdorf , Silesia
  , Germany , now part of Poland
• The Nobel Prize was awarded not for a single
  major discovery , but for the series of findings
  in the decades that it took to prove the
  hypothesis , Blobel said
• My work was a direct extension of the work Dr.
  Palade had started at Rockefeller , '' Blobel
  said of Palade , who went on to win a Nobel Prize
  in 1974
• In the early 1950s , Blobel escaped to the West
  through Berlin
• In 1971 , Blobel and Dr. David D. Sabatini , who
  now heads cell biology at New York University
  School of Medicine , proposed a bold idea known
  as the signal hypothesis
• Blobel , a cellular and molecular biologist ,
  discovered in the early 1970s that newly
  synthesized proteins have an intrinsic signal
  that is essential for directing them
• Blobel , who recalled witnessing the bombing of
  Dresden as a child , said he was donating most of
  the 960,000 in prize money to the Friends of
  Dresden , an independent American group that
  supports the reconstruction , restoration and
  preservation of Dresden 's artistic and
  architectural legacy
• Guenter Blobel , a cell biologist at Rockefeller
  University in New York , was awarded the Nobel
  Prize in Physiology or Medicine Monday for
  discovering how proteins get shipped to their
  proper destinations within the body after being
  manufactured by tiny molecular factories inside
  cells
• Blobel was awarded the Nobel Prize in Physiology
  or Medicine on Monday for discovering that
  proteins carry signals that act as ZIP codes ,
  helping them find their correct locations within
  the cell

 
Useful/not redundant
Useful/but redundant
Not useful
Human Judgements
11
Sentence Selected Using Linguistic Features
Dr. Gunter Blobel , a cellular and molecular
biologist at Rockefeller University in New York
City , won the 1999 Nobel Prize in medicine on
Monday for discovering that proteins carry
signals that act as ZIP codes , helping them find
their correct locations within the cell .
(APPbiologist) -LRB- Angel Franco/New York
Times Photo -RRB- -LRB- NYT10 -RRB- NEW YORK --
Oct. 11 , 1999 -- SCI - NOBEL - MEDICINE , 10-11
-- Dr. Guenter Blobel , a cellular and molecular
biologist at Rockefeller University on Monday .
(APPbiologist) -- Guenter Blobel , a cell
biologist at Rockefeller University in New York ,
was awarded the Nobel Prize in Physiology or
Medicine Monday for discovering how proteins get
shipped to their proper destinations within the
body after being manufactured by tiny molecular
factories inside cells . (APPbiologist) Reefers
SCI - NOBEL - MEDICINE -LRB- Undated -RRB- --
Dr. Guenter Blobel , a cellular and molecular
biologist at Rockefeller University in New York
City , is awarded the 1999 Nobel Prize in
Physiology of Medicine for discovering that
proteins carry signals that act as ZIP codes ,
helping them find their correct locations within
the cell . (APPbiologist) Blobel was born in
1936 in Waltersdorf , Silesia , Germany , now
part of Poland . (verbborn objTERM
inGermany) Young Gunter graduated from high
school with highest grades , although he never
studied very much , '' his oldest brother , Dr.
Hans Blobel , said in a telephone interview from
his home in Geissen , Germany . (verbgraduated,
subTERM) Blobel reasoned that proteins have to
be transported either out of the cell , or to the
different organelles . (verbreasoned,
subTERM)
12
Sentences Selected Using Relations

• Dr. Gunter Blobel , a cellular and molecular
  biologist at Rockefeller University in New York
  City , won the 1999 Nobel Prize in medicine on
  Monday for discovering that proteins carry
  signals that act as ZIP codes , helping them find
  their correct locations within the cell .
  (APPbiologist)
• -LRB- Angel Franco/New York Times Photo -RRB-
  -LRB- NYT10 -RRB- NEW YORK -- Oct. 11 , 1999 --
  SCI - NOBEL - MEDICINE , 10-11 -- Dr. Guenter
  Blobel , a cellular and molecular biologist at
  Rockefeller University on Monday .
  (APPbiologist)
• -- Guenter Blobel , a cell biologist at
  Rockefeller University in New York , was awarded
  the Nobel Prize in Physiology or Medicine Monday
  for discovering how proteins get shipped to their
  proper destinations within the body after being
  manufactured by tiny molecular factories inside
  cells . (APPbiologist)
• Reefers SCI - NOBEL - MEDICINE -LRB- Undated
  -RRB- -- Dr. Guenter Blobel , a cellular and
  molecular biologist at Rockefeller University in
  New York City , is awarded the 1999 Nobel Prize
  in Physiology of Medicine for discovering that
  proteins carry signals that act as ZIP codes ,
  helping them find their correct locations within
  the cell . (APPbiologist)
• LRB- AP -RRB- -- Dr. Guenter Blobel of The
  Rockefeller University in New York won the Nobel
  Prize for medicine today for protein research
  that shed new light on diseases including cystic
  fibrosis and early development of kidney stones .
  (rnameROLE/GENERAL-STAFF arg1TERM
  arg2University)
• Blobel was born in 1936 in Waltersdorf , Silesia
  , Germany , now part of Poland . (verbborn
  objTERM inGermany)
• Young Gunter graduated from high school with
  highest grades , although he never studied very
  much , '' his oldest brother , Dr. Hans Blobel ,
  said in a telephone interview from his home in
  Geissen , Germany . (verbgraduated, subTERM)

Relation
13
Sentence Compression

• Dr. Gunter Blobel , a cellular and molecular
  biologist at Rockefeller University in New York
  City , won the 1999 Nobel Prize in medicine on
  Monday for discovering that proteins carry
  signals that act as ZIP codes , helping them find
  their correct locations within the cell .
  (APPbiologist)
• -LRB- Angel Franco/New York Times Photo -RRB-
  -LRB- NYT10 -RRB- NEW YORK -- Oct. 11 , 1999 --
  SCI - NOBEL - MEDICINE , 10-11 -- Dr. Guenter
  Blobel , a cellular and molecular biologist at
  Rockefeller University on Monday .
  (APPbiologist)
• -- Guenter Blobel , a cell biologist at
  Rockefeller University in New York , was awarded
  the Nobel Prize in Physiology or Medicine Monday
  for discovering how proteins get shipped to their
  proper destinations within the body after being
  manufactured by tiny molecular factories inside
  cells . (APPbiologist)
• Reefers SCI - NOBEL - MEDICINE -LRB- Undated
  -RRB- -- Dr. Guenter Blobel , a cellular and
  molecular biologist at Rockefeller University in
  New York City , is awarded the 1999 Nobel Prize
  in Physiology of Medicine for discovering that
  proteins carry signals that act as ZIP codes ,
  helping them find their correct locations within
  the cell . (APPbiologist)
• LRB- AP -RRB- -- Dr. Guenter Blobel of The
  Rockefeller University in New York won the Nobel
  Prize for medicine today for protein research
  that shed new light on diseases including cystic
  fibrosis and early development of kidney stones .
  (rnameROLE/GENERAL-STAFF arg1TERM
  arg2University)
• Blobel was born in 1936 in Waltersdorf , Silesia
  , Germany , now part of Poland . (verbborn
  objTERM inGermany)
• Young Gunter graduated from high school with
  highest grades , although he never studied very
  much , '' his oldest brother , Dr. Hans Blobel ,
  said in a telephone interview from his home in
  Geissen , Germany . (verbgraduated, subTERM)

14
Redundancy Removal

• Dr. Gunter Blobel , a cellular and molecular
  biologist at Rockefeller University in New York
  City , won the 1999 Nobel Prize in medicine on
  Monday for discovering that proteins carry
  signals that act as ZIP codes , helping them find
  their correct locations within the cell .
  (APPbiologist)
• -LRB- Angel Franco/New York Times Photo -RRB-
  -LRB- NYT10 -RRB- NEW YORK -- Oct. 11 , 1999 --
  SCI - NOBEL - MEDICINE , 10-11 -- Dr. Guenter
  Blobel , a cellular and molecular biologist at
  Rockefeller University on Monday .
  (APPbiologist)
• -- Guenter Blobel , a cell biologist at
  Rockefeller University in New York , was awarded
  the Nobel Prize in Physiology or Medicine Monday
  for discovering how proteins get shipped to their
  proper destinations within the body after being
  manufactured by tiny molecular factories inside
  cells . (APPbiologist)
• Reefers SCI - NOBEL - MEDICINE -LRB- Undated
  -RRB- -- Dr. Guenter Blobel , a cellular and
  molecular biologist at Rockefeller University in
  New York City , is awarded the 1999 Nobel Prize
  in Physiology of Medicine for discovering that
  proteins carry signals that act as ZIP codes ,
  helping them find their correct locations within
  the cell . (APPbiologist)
• LRB- AP -RRB- -- Dr. Guenter Blobel of The
  Rockefeller University in New York won the Nobel
  Prize for medicine today for protein research
  that shed new light on diseases including cystic
  fibrosis and early development of kidney stones .
  (rnameROLE/GENERAL-STAFF arg1TERM
  arg2University)
• Blobel was born in 1936 in Waltersdorf , Silesia
  , Germany , now part of Poland . (verbborn
  objTERM inGermany)
• Young Gunter graduated from high school with
  highest grades , although he never studied very
  much , '' his oldest brother , Dr. Hans Blobel ,
  said in a telephone interview from his home in
  Geissen , Germany . (verbgraduated, subTERM)

15
System Output

• A cellular and molecular biologist at Rockefeller
  University in New York City
• Dr. Guenter Blobel of The Rockefeller University
  in New York
• Blobel was born in 1936 in Waltersdorf , Silesia
  , Germany , now part of Poland
• Young Gunter graduated from high school with
  highest grades , although he never studied very
  much , ''

16
Evaluation

• Test bed 30 biographical questions with human
  created answers
• BLEU metric was used
• For each question, system output the top N
  response items that are less or equal to the size
  of the manual answer
• Answer brevity, which should be rewarded for
  bio/def QA, is penalized by BLEU

17
Summary

• Approach to biography generation combines
• Information retrieval
• Linguistic analysis
• Information extraction
• Redundancy detection
• Sentence compression
• Automatic evaluation by BLEU metric
• Much work remains