Motivation and Outline

1
Motivation and Outline

• Background
• Definitions, etc.
• The Problem
• 100,000 pages
• The Solution
• Ranking docs
• Vector space
• Extensions
• Relevance feedback,
• clustering,
• query expansion, etc.

2
Motivation

• IR representation, storage, organization of, and
  access to information items
• Focus is on the user information need
• User information need
• Find all docs containing information on college
  tennis teams which (1) are maintained by a USA
  university and (2) participate in the NCAA
  tournament.
• Emphasis is on the retrieval of information (not
  data)

3
Motivation

• Data retrieval
• which docs contain a set of keywords?
• Well defined semantics
• a single erroneous object implies failure!
• Information retrieval
• information about a subject or topic
• semantics is frequently loose
• small errors are tolerated
• IR system
• interpret contents of information items
• generate a ranking which reflects relevance
• notion of relevance is most important

4
Motivation

• IR at the center of the stage
• IR in the last 20 years
• classification and categorization
• systems and languages
• user interfaces and visualization
• Still, area was seen as of narrow interest
• Advent of the Web changed this perception once
  and for all
• universal repository of knowledge
• free (low cost) universal access
• no central editorial board
• many problems though IR seen as key to finding
  the solutions!

5
Basic Concepts

• The User Task
• Retrieval
• information or data
• purposeful
• Browsing
• glancing around
• F1 cars, Le Mans, France, tourism

6
The Retrieval Process
7
Measuring Performance
tn

• Precision
• Proportion of selected items that are correct
• Recall
• Proportion of target items that were selected
• Precision-Recall curve
• Shows tradeoff

fp
tp
fn
System returned these
Actual relevant docs
Precision
Recall
8
Precision/Recall Curves

• 11-point recall-precision curve
• Example Suppose for a given query, 10 documents
  are relevant. Suppose when all documents are
  ranked in descending similarities, we have
• d1 d2 d3 d4 d5 d6 d7 d8 d9 d10 d11 d12 d13
  d14 d15 d16 d17 d18 d19 d20 d21 d22 d23 d24 d25
  d26 d27 d28 d29 d30 d31

precision
recall
1.0
.1
.3
9
Precision Recall Curves

• When evaluating the retrieval effectiveness of a
  text retrieval system or method, a large number
  of queries are used and their average 11-point
  recall-precision curve is plotted.
• Methods 1 and 2 are better than method 3.
• Method 1 is better than method 2 for high recalls.

precision
Method 1 Method 2 Method 3
recall
10
Query Models

• IR systems usually adopt index terms to process
  queries
• Index term
• a keyword or group of selected words
• any word (more general)
• Stemming might be used
• connect connecting, connection, connections
• An inverted file is built for the chosen index
  terms

11
Introduction
Docs
Index Terms
doc
match
Ranking
Information Need
query
12
Introduction

• Matching at index term level is quite imprecise
• No surprise that users get frequently unsatisfied
• Since most users have no training in query
  formation, problem is even worst
• Frequent dissatisfaction of Web users
• Issue of deciding relevance is critical for IR
  systems ranking

13
Introduction

• A ranking is an ordering of the documents
  retrieved that (hopefully) reflects the relevance
  of the documents to the user query
• A ranking is based on fundamental premisses
  regarding the notion of relevance, such as
• common sets of index terms
• sharing of weighted terms
• likelihood of relevance
• Each set of premisses leads to a distinct IR model

14
IR Models
U s e r T a s k
Retrieval Adhoc Filtering
Browsing
15
IR Models

• The IR model, the logical view of the docs, and
  the retrieval task are distinct aspects of the
  system

16
Retrieval Ad Hoc x Filtering

• Ad hoc retrieval

Q1
Q2
Collection Fixed Size
Q3
Q4
Q5
17
Retrieval Ad Hoc x Filtering

• Filtering

Docs Filtered for User 2
User 2 Profile
User 1 Profile
Docs for User 1
Documents Stream
18
Classic IR Models - Basic Concepts

• Each document represented by a set of
  representative keywords or index terms
• An index term is a document word useful for
  remembering the document main themes
• Usually, index terms are nouns because nouns have
  meaning by themselves
• However, search engines assume that all words are
  index terms (full text representation)

19
Generating keywords

• Logical view of the documents

Accents spacing
Noun groups
Manual indexing
stopwords
stemming
Docs
structure

• Stop-word elimination
• Noun phrase detection
• Stemming
• Generating index terms
• Improving quality of terms.
• Synonyms, co-occurence detection, latent semantic
  indexing..

20
Classic IR Models - Basic Concepts

• Not all terms are equally useful for representing
  the document contents less frequent terms allow
  identifying a narrower set of documents
• The importance of the index terms is represented
  by weights associated to them
• Let
• ki be an index term
• dj be a document
• wij is a weight associated with (ki,dj)
• The weight wij quantifies the importance of the
  index term for describing the document contents

21
Classic IR Models - Basic Concepts

• Ki is an index term
• dj is a document
• t is the total number of docs
• K (k1, k2, , kt) is the set of all index
  terms
• wij gt 0 is a weight associated with (ki,dj)
• wij 0 indicates that term does not belong to
  doc
• vec(dj) (w1j, w2j, , wtj) is a weighted
  vector associated with the document dj
• gi(vec(dj)) wij is a function which returns
  the weight associated with pair (ki,dj)

22
The Boolean Model

• Simple model based on set theory
• Queries specified as boolean expressions
• precise semantics
• neat formalism
• q ka ? (kb ? ?kc)
• Terms are either present or absent. Thus,
  wij ? 0,1
• Consider
• q ka ? (kb ? ?kc)
• vec(qdnf) (1,1,1) ? (1,1,0) ? (1,0,0)
• vec(qcc) (1,1,0) is a conjunctive component

23
The Boolean Model

• q ka ? (kb ? ?kc)
• sim(q,dj) 1 if ? vec(qcc)
  (vec(qcc) ? vec(qdnf)) ? (?ki,
  gi(vec(dj)) gi(vec(qcc))) 0 otherwise

24
Drawbacks of the Boolean Model

• Retrieval based on binary decision criteria with
  no notion of partial matching
• No ranking of the documents is provided (absence
  of a grading scale)
• Information need has to be translated into a
  Boolean expression which most users find awkward
• The Boolean queries formulated by the users are
  most often too simplistic
• As a consequence, the Boolean model frequently
  returns either too few or too many documents in
  response to a user query

25
The Vector Model

• Use of binary weights is too limiting
• Non-binary weights provide consideration for
  partial matches
• These term weights are used to compute a degree
  of similarity between a query and each document
• Ranked set of documents provides for better
  matching

26
The Vector Model

• Define
• wij gt 0 whenever ki ? dj
• wiq gt 0 associated with the pair (ki,q)
• vec(dj) (w1j, w2j, ..., wtj) vec(q)
  (w1q, w2q, ..., wtq)
• To each term ki is associated a unitary vector
  vec(i)
• The unitary vectors vec(i) and vec(j) are
  assumed to be orthonormal (i.e., index terms are
  assumed to occur independently within the
  documents)
• The t unitary vectors vec(i) form an orthonormal
  basis for a t-dimensional space
• In this space, queries and documents are
  represented as weighted vectors

27
Document Vectors

• Documents are represented as bags of words
• Represented as vectors when used computationally
• A vector is like an array of floating point
• Has direction and magnitude
• Each vector holds a place for every term in the
  collection
• Therefore, most vectors are sparse

28
Vector Space Example

• a System and human system engineering testing of
  EPS
• b A survey of user opinion of computer system
  response time
• c The EPS user interface management system
• d Human machine interface for ABC computer
  applications
• e Relation of user perceived response time to
  error measurement
• f The generation of random, binary, ordered
  trees
• g The intersection graph of paths in trees
• h Graph minors IV Widths of trees and
  well-quasi-ordering
• i Graph minors A survey

29
Document VectorsOne location for each word.

• nova galaxy heat hwood film role diet fur
• 10 5 3
• 5 10
• 10 8 7
• 9 10 5
• 10 10
• 9 10
• 5 7 9
• 6 10 2 8
• 7 5 1 3

A B C D E F G H I
Nova occurs 10 times in text A Galaxy occurs
5 times in text A Heat occurs 3 times in text
A (Blank means 0 occurrences.)
30
Document VectorsOne location for each word.

• nova galaxy heat hwood film role diet fur
• 10 5 3
• 5 10
• 10 8 7
• 9 10 5
• 10 10
• 9 10
• 5 7 9
• 6 10 2 8
• 7 5 1 3

A B C D E F G H I
Hollywood occurs 7 times in text I Film
occurs 5 times in text I Diet occurs 1 time in
text I Fur occurs 3 times in text I
31
Document Vectors
Document ids

• nova galaxy heat hwood film role diet fur
• 10 5 3
• 5 10
• 10 8 7
• 9 10 5
• 10 10
• 9 10
• 5 7 9
• 6 10 2 8
• 7 5 1 3

A B C D E F G H I
32
We Can Plot the Vectors
Star
Doc about movie stars
Doc about astronomy
Doc about mammal behavior
Diet
33
Documents in 3D Space
34
Similarity Function (1)

• The similarity or closeness of a document d (
  w1, , wi, , wn ) with respect to a query q (
  q1, , qi, , qn ) is computed using a similarity
  function.
• Many similarity functions exist.
• Dot product function
• sim(q, d) dot(q, d) q1 ? w1 qn ?
  wn
• Example Suppose d (0.2, 0, 0.3, 1) and
• q (0.75, 0.75, 0, 1), then
  
• sim(q, d) 0.15 0 0 1 1.15

35
Similarity Function (2)

• Observations of the dot product function.
• Documents having more terms in common with a
  query tend to have higher similarities with the
  query.
• For terms that appear in both q and d, those with
  higher weights contribute more to sim(q, d) than
  those with lower weights.
• It favors long documents over short documents.
• The computed similarities have no clear upper
  bound.

36
A normalized similarity metric
j
dj
?
q
i

• Sim(q,dj) cos(?) vec(dj) ?
  vec(q) / dj q ? wij wiq /
  dj q
• Since wij gt 0 and wiq gt 0, 0 lt
  sim(q,dj) lt1
• A document is retrieved even if it matches the
  query terms only partially

37
Vector Space Example cont.
interface
user
c
b
system
a
38
Answering a Query UsingVector Space

• Represent query as vector
• Compute distances to all documents
• Rank according to distance
• Example
• computer system

39
The Vector Model

• Sim(q,dj) ? wij wiq / dj q
• How to compute the weights wij and wiq ?
• Simple keyword frequencies tend to favor common
  words
• E.g. Query The Computer Tomography
• A good weight must take into account two effects
• quantification of intra-document contents
  (similarity)
• tf factor, the term frequency within a document
• quantification of inter-documents separation
  (dissi-milarity)
• idf factor, the inverse document frequency
• wij tf(i,j) idf(i)

40
The Vector Model

• Let,
• N be the total number of docs in the collection
• ni be the number of docs which contain ki
• freq(i,j) raw frequency of ki within dj
• A normalized tf factor is given by
• f(i,j) freq(i,j) / max(freq(l,j))
• where the maximum is computed over all terms
  which occur within the document dj
• The idf factor is computed as
• idf(i) log (N/ni)
• the log is used to make the values of tf and
  idf comparable. It can also be interpreted as
  the amount of information associated with the
  term ki.

41
The Vector Model

• The best term-weighting schemes use weights which
  are give by
• wij f(i,j) log(N/ni)
• the strategy is called a tf-idf weighting
  scheme
• For the query term weights, a suggestion is
• wiq (0.5 0.5 freq(i,q) /
  max(freq(l,q)) log(N/ni)
• The vector model with tf-idf weights is a good
  ranking strategy with general collections
• The vector model is usually as good as the known
  ranking alternatives. It is also simple and fast
  to compute.

42
The Vector Model

• Advantages
• term-weighting improves quality of the answer set
• partial matching allows retrieval of docs that
  approximate the query conditions
• cosine ranking formula sorts documents according
  to degree of similarity to the query
• Disadvantages
• assumes independence of index terms (??) not
  clear that this is bad though

43
The Vector Model Example I
44
The Vector Model Example II
45
The Vector Model Example III