Information Retrieval using Word Senses: Root Sense Tagging Approach

1
Information Retrieval using Word SensesRoot
Sense Tagging Approach

• Hae-Chang Rim
• NLP Lab., Dept. of CSE, Korea University

2
Contents

• Introduction
• Related Works
• Overview of Sense Oriented IR system
• Root Sense Tagging Approach
• Co-occurrence Data Construction
• MI-based root sense tagging
• Sense encoded Indexing and Retrieval
• Experimental Results
• Conclusions

3
Introduction (1/3)

• Lexical Ambiguity
• A general problem in natural language processing
• Can be resolved by using context of the given
  target word
• In Information Retrieval
• Query and document contain many ambiguous words
• Resolving lexical ambiguity SEEMS to be useful to
  improve performance

Q paying interest on debt
The tax rate is closely related to interest on
loans from the federal government
If you have a particular interest, our program
can be tailored towards your needs
D1
D2
4
Introduction (2/3)

• Nevertheless,
• Most of the previous IR experiments have shown
  that it is very difficult to improve retrieval
  performance by using word senses
• Why?
• Characteristics of IR disambiguation is
  unnecessary
• Collocation can reduce the effect of ambiguity in
  a querys individual words
• Most query words are unambiguous or used with the
  majority sense
• Disambiguation errors
• 2030 disambiguation errors deteriorate the
  retrieval performance

5
Introduction (3/3)

• Our Hypotheses
• Coarse-grained disambiguation gt Fine-grained
  disambiguation
• Fine-grained disambiguation can result in serious
  disambiguation errors
• Even humans can not determine fine-grained senses
  defined in a dictionary
• Consistent disambiguation gt Accurate
  disambiguation
• Exactly the same disambiguation errors in query
  and document will be OK!
• Same senses should be always assigned to the
  words in collocation or multi-word expressions
• Flexible disambiguation gt Strict disambiguation
• Matching failure from strict disambiguation is
  highly risky
• Disambiguation results should be used as safely
  as possible

6
Related Works (1/2)

• (Schütze1995)
• The most successful work
• Reporting 14 improvement in retrieval
  performance
• Coarse-grained, flexible WSD
• Merging Sense-based Term-based raking results
• Context clustering using SVD
• (Stoke2003)
• Experiments on Large-Scale Web collection
• Fine-grained, strict WSD
• Supervised learning with sense-tagged SEMCOR
  corpus

7
Related Works (2/2)

• We try to ..
• Perform Coarse-grained and flexible WSD like
  (Schütze1995),
• but ..
• NOT require expensive collection processing such
  as SVD
• NOT keep both term-based index and sense-based
  index
• Use well-constructed external resource like
  (Stoke2003),
• but ..
• NOT use all the fine-grained senses
• NOT perform supervised learning with expensive
  sense-tagged corpus

8
System Overview
Non-ambiguous words
WordNet
Co-occurrence data
Co-occurrence data construction
Text Collection
MI-based Root sense tagging
Index
Sense-encoded Indexing
Sense-oriented Ranking
Result
9
Root Sense Tagging Approach1. Co-occurrence
Data Construction (1/4)
Non-ambiguous words
WordNet
Co-occurrence data
Co-occurrence data construction
Query
Text Collection
MI-based Root Sense Tagging
MI-based Root sense tagging
Indexing
Index
Sense-encoded Indexing
Sense-oriented Ranking
Result
Sense-encoded Indexing
Ranking
10
Root Sense Tagging Approach1. Co-occurrence
Data Construction (2/4)

• Non-ambiguous Words
• all nouns and compound nouns having a unique root
  sense in WordNet
• Root sense one of 25 Beginner senses of noun
  hierarchy in WordNet
• Used as training examples for MI-based root sense
  tagger
• We can calculate mutual information between root
  senses and words using co-occurrence information
  between non-ambiguous words and their neighboring
  words

11
Root Sense Tagging Approach1. Co-occurrence
Data Construction (3/4)

• Co-occurrence Data Construction
• For each document in the collection,
• Assign a root sense to each non-ambiguous noun in
  the document.
• Assign a root sense to each second noun of
  non-ambiguous compound nouns in the document.
• Even if any noun tagged in step 2 occurs alone in
  other position, assign the same root sense in
  step 2. ( one-sense-per-discourse assumption )
• For each sense-assigned noun in the document,
  extract all (context word, sense) pairs within a
  predefined window.
• Used for calculating mutual information between a
  sense and a word

12
Root Sense Tagging Approach1. Co-occurrence
Data Construction (4/4)

• Co-occurrence Data Construction (contd)
• Mutual Information between a sense and a word

13
Root Sense Tagging Approach2. MI-based Root
Sense Tagging (1/5)
Non-ambiguous units
WordNet
Root Sense Tagging
Co-occurrence data
Co-occurrence data construction
Co-occurrence Data Construction
Text Collection
MI-based Root Sense Tagging
MI-based Root sense tagging
Indexing
Index
Sense-encoded Indexing
Result
Sense-encoded Indexing
Sense-oriented Ranking
Ranking
14
Root Sense Tagging Approach2. MI-based Root
Sense Tagging (2/5)

• Root Sense Tagging
• select the most related context word c(w) to
  ambiguous word w among the context words by
  mutual information

The tax rate is closely related to interest on
loans from the federal government
If you have a particular interest, our program
can be tailored towards your needs
D1
D2
Root sense Possession
Root sense Cognition
D1
D2
15
Root Sense Tagging Approach2. MI-based Root
Sense Tagging (3/5)

• find the highest MI-valued candidate root sense
  s(w) with the selected context word c(w) in the
  first step

If you have a particular interest, our program
can be tailored towards your needs
The tax rate is closely related to interest on
loans from the federal government
D1
D2
16
Root Sense Tagging Approach2. MI-based Root
Sense Tagging (4/5)

• Root Sense Tagging More Examples

e.g.) interest
... and bay has been designated site of special
scientific interest ...
... nations with very different interests could
never reach consensus ...
COGNITION
... many hours of discussion between various
interests represented on it ...
... government, which has not paid interest on
its bank debt since ...
POSSESSION
... interest margin is 55 basis points over Libor
...
... related to changes in poll tax and interest
rates ...
e.g.) system
... makes other automation control components and
systems, ...
ARTIFACT
... TCS stands for traction-control system ...
... system is changing slowly but be suspicious
...
COGNITION
... It made sense to change system slightly so
that ...
... if studys plan for opening up auction system
is adopted ...
BODY
... enable Poland to maintain the open import
system that we now enjoy ...
17
Root Sense Tagging Approach2. MI-based Root
Sense Tagging (5/5)

• Characteristics
• Using automatically constructed co-occurrence
  data
• doesnt require any expensive sense-tagged corpus
• Simple coarse-grained disambiguation
• simplified version of statistical WSD method
• Consistent disambiguation is performed
• by using only the single context word
• for the words in multiword expressions or
  collocations
• interest rate
• star wars
• make a promise

18
Root Sense Tagging Approach3. Sense encoded
Indexing and Retrieval (1/4)
Non-ambiguous units
WordNet
Co-occurrence data
Co-occurrence data construction
Co-occurrence Data Construction
Text Collection
MI-based Root Sense Tagging
MI-based Root sense tagging
Indexing
Index
Sense-encoded Indexing
Sense-oriented Ranking
Result
Sense-encoded Indexing
Ranking
19
Root Sense Tagging Approach3. Sense encoded
Indexing and Retrieval (2/4)

• Using Sense Field
• A new data field to encode senses to the term
  posting element in the inverted index
• Consists of 25 bits for root senses and 1 bit for
  unknown word

Document ID
Term Frequency
Sense Field
..
Term Posting Element in Inverted Index
20
Root Sense Tagging Approach3. Sense encoded
Indexing and Retrieval (3/4)

• Two problems in using Sense Field
• Several different root senses may be assigned to
  the same word within a document according to
  their different contexts
• Sometimes, a noun word and a verb word have a
  same stem, where noun word is assigned by a sense
  but verb word is not.
• Sense Merging
• We simply merge all the sense fields for each
  stem by the bitwise-OR operation, and take the
  merged sense field for its final sense field

21
Root Sense Tagging Approach3. Sense encoded
Indexing and Retrieval (4/4)

• Sense Merging Example
• By allowing multiple-sense assignment,
• reducing risk from inaccurate strict
  disambiguation
• avoiding exponential increase of the index size

22
Experimental Results (1/9)

• Data
• Query
• TREC 7(351-400), 8(401-450) title and description
  queries
• Collection
• Financial Times collection in TREC CD vol. 4
  (210,157 documents)
• LA Times collection in TREC CD vol. 5 ( 127,742
  documents)

23
Experimental Results (2/9)

• Evaluation Methodology
• Sense-Oriented Term Reweighting
• Multiply original term weight by sense weight
• Sense Weight
• 1a if lexically matched term between a query
  and a document has a common sense bit
• 1- a if there is no common sense bit
• where,

24
Experimental Results (3/9)

• Evaluation Methodology (contd)
• Three original term weights are considered as
  baselines

Exclude the effects from term frequency Exclude
the effects from length normalization
Hold the effects from term frequency Exclude the
effects from length normalization
constrain the effects from term frequency Exclude
the effects from length normalization
For each Wi ,
25
Experimental Results (4/9)

• Performances 11pt average precisions

• More improvements in long query experiments
• Terms in long queries have similar contexts with
  those in documents
• Root sense tagging successfully specialize the
  terms using senses

• Less improvements in W2 and W3
• proper length normalization technique as well as
  term weighting
• method considering word senses should be
  developed

26
Experimental Results (5/9)

• Performances precisions at 10 documents

• More improvements in W1 experiments than 11pt
  avg. prec.
• certainly contributes to pulling up more relevant
  documents to the
• top ranks

• Less improvements in W2 and W3 experiments than
  11pt avg. prec.
• overgrown term weights by sense-oriented term
  weighting
• constraining term frequencies by log-heuristics
  is not a solution

27
Experimental Results (6/9)

• Performances improved vs. deteriorated

• Better performance is achieved for the more than
  half of the
• queries in all experiments
• Performances of some queries are deteriorated
• More elaborate sense tagging method is required

28
Experimental Results (7/9)

• What if ..
• Our root sense tagging approach meets
• Relevance Feedback
• State-of-the-art Ranking Function such as BM25

?
29
Experimental Results (8/9)

• Performances 11pt average precisions with RF
• Select 5 terms from the top 10 documents
• Sense Voting is performed to encode senses for
  added terms

• Further improvements are achieved in all
  experiments
• Good feedback terms are selected
• Senses encoded in feedback terms contribute to
  improvement

30
Experimental Results (9/9)
Initial Search

• Performances of BM25 with senses

Feedback Search
No sense weighting if a0
31
Conclusions (1/2)

• Implemented three hypotheses
• Coarse-grained disambiguation
• Utilize 25 unique beginner senses in WordNet
• Enable us to construct co-occurrence information
  without expensive sense-tagged corpus
• Consistent disambiguation
• Consider only the single most informative
  neighboring word as evidence of determining the
  sense of the target word
• Maximize disambiguation consistency at the cost
  of accuracy
• Flexible disambiguation
• Merge multiple senses into a sense field to avoid
  the risks from disambiguation errors
• Do not seriously increase system-overhead by
  sense-field based indexing and sense-weight
  oriented ranking

32
Conclusions (2/2)

• Experimental Results show that..
• Our root sense tagging approach retrieves
  relevant documents more accurately
• Especially, IDF term weighting method (W1
  sense) substantially improves performance
• It also works well with relevance feedback and
  state-of-the-art ranking function such as BM25

33
Future Works

• We will try to
• Assign senses to the verbs
• Develop a more elaborate term weighting method
  considering word senses and length normalization
  technique
• Devise a more advanced root sense tagging method
• will consider syntactic features such as POS
  patterns

34
Thank You !!!