Search Engines and Directories Networking Concepts

1
Search Engines and DirectoriesNetworking Concepts

• D.A. Clements

2
Search Engines
3
What is a Search Engine?

• A client/server application
• A document retrieval system
• Use regularly updated indexes to operate quickly
  and efficiently
• Designed to help find information stored
• On a computer system, such as on the World Wide
  Web
• Inside a corporate or proprietary network
• In a personal computer
• Different selection and relevance criteria can
  apply in different environments, or for different
  uses
• Allows one to ask for content meeting specific
  criteria
• Typically those containing a given word or phrase
• Retrieves a list of items that match those
  criteria

4
What else?

• Some also mine data available in
• Newsgroups
• Large databases
• Open directories like DMOZ.org
• What about the text of books
• Web directories maintained by human editors
• Search engines operate algorithmically
• Many website search engines are actually front
  ends to search engines of others

5
History

• Archie First search tool for the Internet
• Archive" without the "v", not the character from
  'Archie' comic book
• Created (1990) by Alan Emtage, a student at
  McGill University, Montreal
• Downloaded the directory listings of all files
  located on public anonymous FTP sites
• Creating a searchable database of filenames - not
  file contents
• Gopher indexed plain text documents
• Created (1991) by Mark McCahill at the University
  of Minnesota
• Named after the school's mascot
• Most of the Gopher sites became websites after
  the creation of the WWW
• Veronica searched the files stored in Gopher
  index systems
• Very Easy Rodent-Oriented Net-wide Index to
  Computerized Archives
• Provided a keyword search of most Gopher menu
  titles
• Jughead searched the files stored in Gopher
  index systems
• Jonzy's Universal Gopher Hierarchy Excavation And
  Display
• Tool for obtaining menu information from various
  Gopher servers
• Wandex first Web search engine
• Used index collected by the World Wide Web
  Wanderer, a web crawler developed by Matthew Gray
  at MIT in 1993

6
Google

• 2001 rose to prominence
• Currently the most popular search engine
• Success based on the concept of link popularity
  and PageRank
• PageRank The number of websites and webpages
  that link to a page
• Possible to order its results by how many
  websites link to each found page
• PageRank is based on citation analysis developed
  (1950s) by Eugene Garfield at the University of
  Pennsylvania
• Minimalist user interface was very popular with
  users
• Utilize more than 150 criteria to determine
  relevancy

7
Others

• Yahoo! Search
• Founders David Filo and Jerry Yang, Ph.D.
  candidates at Stanford University
• Started in a campus trailer (February 1994) to
  keep track of their personal interests on the
  Internet
• 2002, Yahoo! acquired Inktomi
• 2003, Yahoo! acquired Overture, which owned
  AlltheWeb and AltaVista
• 2004, launched its own search engine
• Microsofts Windows Live Search
• Most recent major search engine is
• Powered by its own web crawler (called msnbot)
• 2006, Microsoft migrated to the new search
  platform
• Ask.com
• February 2006, rebranded Ask Jeeves
• Maps (with walking directions and dynamic address
  generation)
• "Smart Answers" were added
• Algorithmic engine using relevance ranking
  originally developed for Teoma
• Features generally unavailable elsewhere to help
  narrow, expand, and select related names
• Page previews
• "Zoom"

8
Other Search Indexes
9
Advanced Search Link
10
About Google Link
11
Google Help
12
3rd Party Resources
Google Pocket Guide (Paperback) by Tara
Calishain, Rael Dornfest, D J Adams Paperback
140 pagesPublisher O'Reilly Media 1 edition
(June 30, 2003)Language EnglishISBN 0596005504
13
Search Directories

• Different from Search Engines.

14
Search Engines vs. Directories

• Search Engines

• Search Directories

• Automatedno human intervention
• Paid advertisers buy position at top of results
  lists

• Indexed by humans
• Examples
• Yahoo

15
Search engine comparisons

• Graphical and hybrid

16
Kartoovisual meta search engine
17
Kartoo
Click images tab to see thumbnails pulled from
Yahoo!
18
Quinturawith tag clouds

• Tag clouds change results

19
Quintura

• Tag clouds change results

20
End papers

• Computers are quick but they dont think.
• D.A. Clements

21
End papers

• The Internet has no such organizationfiles are
  made available at random locations. To search
  through this chaos, we need smart tools, programs
  that find resources for us.
• Clifford Stoll, Silicon Snake Oil, 1995