IST 497E Information Retrieval and Organization

1
IST 497E Information Retrieval and Organization
Meta Search
2
Overview

• What is a Meta Search Engine
• Features
• Differences
• Architecture
• Some Meta Search Engines
• Conclusions and Future ?

3
(No Transcript)
4
Overview
5
What Is a Meta Search Engine ?

• Dictionary meaning for Meta
• more comprehensive transcending.- webster.com
• Simple Explanation
• A Meta Search engine allows you to search
  multiple search engines at once, returning more
  comprehensive and relevant results, fast.
  MetaCrawler modified

6
Traveling Further

• A little bit of History
• Started with Harvest (1995) Ref Information
  Discovery and Access System C. Mic Bowman et
  al.
• Was developed for gathering information from
  repositories, building topic-specific content
  indexes, web-caching,flexible searching.
• In many ways current Meta Search engines have
  similar aims.

7
Why Are Meta Search Engines Useful ?

• Meta Search improves the Search Quality in many
  ways
• Comprehensive,
• Efficient,
• One query queries all one-click paradigm,

8
Why Meta Search ?

• Individual Search engines dont cover all the web
  by themselves,
• Individual Search Engines are prone to spamming
  people trying to raise their ranking profile. In
  a non-legitimate manner or to promote commerce,
• Difficulty in deciding and obtaining results with
  combined searches on different search engines,

9
Why Meta Search ?

• Data Fusion multiple formats supported,
• In Case of niche search engines provides the big
  picture,
• Takes less effort.

10
Overview
11
Features.

• Unifies the Search Interface and provides a
  consistent user interface,
• Standardizes the query structure,
• May make use of an independent ranking method for
  the results rank-merging,
• May have an independent ranking system for each
  search engine/database it searches,
• Meta Search is not a search for Meta Data.

12
Overview
13
Differences Search Vs. Meta-search

• Doesnt generally have a Database by itself,
• Does not searchcrawl the web.
• A Meta-Search Engine in terms of search engine.
• Essentially is a hub of search engines/databases
  accessible by a common interface providing the
  user with results which may/may not be ranked
  independently of the original search
  engine/source ranking.

14
Overview
Architecture/Internals
A block representation,
What do those blocks do ?,
Queries,
Ranking.
15
Architecture
Feedback
Knowledge Personalize
Dispatcher
Query
User Interface
User
S E 1
S E 2
S E 3
Web
Display
16
What Do Those Blocks Do ?

• User Interface
• Normally resemble search engine interfaces with
  options for
• Types of search Media
• Search Engines to Use
• Dispatcher
• Generates actual queries to the search engines by
  using the user query
• May involve choosing/expanding search engines to
  use

17
What Do Those Blocks Do ?

• Display.
• Generates Results page from the replies received,
  May involve ranking,parsing,clustering of the
  search results or just plain stitching.
• Personalization/Knowledge.
• May contain either or both. Personalization may
  involve weighting of search results/query/engine
  for each user.

18
Queries

• STARTS protocol .
• A simple protocol that text search engines should
  follow to facilitate searching and indexing
  multiple collections of text documents.
• Choosing best source for a query,
• Evaluating a query at those sources,
• Merging the query results from them.
• Inquirus Expand Queries.
• Ex (What does Satellite stand for Satellite
  stands for).

19
Independent Ranking

• Stitch together the results Dogpile,
• Selection of a particular Search Engine based on
  a query a meta index SavySearch-Resource
  Balancing,
• Rank merging based on Search Engine rating
  MetaCrawler,
• Context analysis for search results with respect
  to the query Inquirus.

20
Fusion of Other Media

• Inquirus.
• Motivation for a meta-search Images
• Queries from the user can be modified, since
  individual search engines are good at different
  types of queries the results are very good.
• How ?
• See Reference Skipped because of topic overlap.
• Can it work for others ?
• Example FTP,Music Probably yes P2P searches.

21
Other Work

• Other work Media Fusion.
• MetaSEEk.
• Visual Search Engine for images,
• Makes use of query by example.
• Ixquick.
• MP3,Images,News.all come from different search
  interfaces may not exactly be fusion.
• Dogpile.
• Multimedia,Images,News,Files.

22
(No Transcript)
23
Overview
Some Meta Search Engines
On the web,
Why are they not so popular ?,
Some Reviews.
24
Meta search Engines on the Web

• MetaCrawler,
• Ixquick,
• Inquirus,
• SavySearch now cnet search ?,
• Dogpile,
• Sherlockhound,
• Vivisimo.

25
Why Are They Not So Popular ?

• Ads !
• Some Meta Search engines pick up ads as part of
  search results from the participating search
  engines.
• Example Dogpile.
• Similar reasons as general search engines (ads
  clutter search results).
• Example MetaCrawler,Ixquick.
• Paid Placement Search Engines included,
• Relation with the search engines they depend on.
  load,interaction.

26
Why Are They Not So Popular ?

• Results as good as the worst search engine in the
  group.
• Combining results using meta-index and rankings
  can lead to incorrectly ranked results from a
  search engine reducing the relevance of correct
  results.
• If your highly ranked search engine returns a
  badly ranked result, then your results are also
  badly ranked.

27
Some Reviews

• Search Engine Watch May 2001

28
Overview
29
Conclusions and Future

• Vivisimo reported 43 increase in traffic
  wired.com Aug. 14, 2001,
• Apple Sherlock has been popular,
• Multiple Media Search,
• P2P Searches ?,
• Growth or Niche Search Engines.

30
References

• The MetaCrawler Architecture for Resource
  Aggregation on the web(1997) Erik Selberg
  Oren Etzioni.
• Context and Page Analysis for Improved Web
  Search(1998) Steve Lawrence C. Lee Giles.
• Experiences with Selecting Search Engines Using
  Metasearch(1997) Daniel Dreilinger Adele E.
  Howe.

31
Time to Wake Up
32
(No Transcript)
33
Web References

• Search Engine Watch Article Meta Search Or Meta
  Ads?,
• Wired Article Searching For Googles Successor,
• MetaSEEk Writeup,
• Text and Image Meta Search on the Web Citeseer
  Paper Reference.