Search Engines

1
Search Engines Marketing Your Web Site

• Dr. Soe CIS Dept.
• (updated January 2005)

2
Agenda

• How do Search Engines work?
• How do you get your web site listed?
• Search Engine exercise

3
High Search Engine Rankings

• "We can guarantee you a top 10 ranking"
• What's it worth?
• How can they do it?
• Ixquick search on telecommuting productivity

4
Locate Information on Internet

• Search Engines Directories
• Meta Search Engines
• On-Line Indexes
• Pages with information on specific topics
• White Yellow pages
• Usenet News
• On-line newspapers, magazines, radio and TV
  channels

5
Types of Search Engines

• Spiders, webcrawlers, robots
• automatic indexing of Key- other-Words
• Google, AltaVista, Northern Light, FAST Search
• Web subject directories
• built by humans who review web pages
• Yahoo!, Open Directory
• Hybrids spiders humans (trend)
• Excite, Go/InfoSeek, Lycos

6
How Web Crawlers Work

• Automated index building
• Crawlers or spiders (indexing robot programs) go
  to web sites
• Examine pages extract indexing information
• may simply locate words
• may identify key words, phrases, links
• Store data in search engines database with URL
  for page

7
Search Engines Deliver Indexes

• User requests information via query engine
• Engine searches database
• Delivers list of web resources
• Creates results web page based on search
• Listed in order based on mathematical formulas
• Calculate values for pages based on search words,
  and possibly on "popularity" of site

8
Problems with Automated Index Building

• No standards
• HTML Documents are not structured so that robots
  can extract routine information
• Except for tags keywords, description,
  publication date, author, etc.
• Usually indexes text, not graphics, movies, etc.
• Search turns up inappropriate documents
• Paid advertising/buying good positions
• Google indexes 8 billion web pages

9
Web Directory Built by Human Indexing

• Analyzes sites purpose
• Classifies sites by broad subject area
• Hierarchical classification schemes
• YAHOO! - has many people reviewing web site
  submissions
• Doesn't have to accept submissions
• Delay (6 weeks) unless pay for priority service

10
Meta Search Engines

• Don't have their own databases or indexing
• Instead, combine results from other search
  engines
• Examples
• Dogpile, MetaCrawler
• Ixquick (top 10 listings)

11

• Overture / pay for placement
• being used by
• Yahoo!
• MSN
• InfoSpace
• CNN

12
Specialized Search Engines

• directory of
• different languages and countries
• "special" search engines (Dutch site)
• Google search on specialized "search engines"

13
Search Engines Ranked by of People that Use Them

• Google 29.5
• Yahoo 28.9
• MSN 27.6
• AOL 18.4
• Ask Jeeves 9.9
• Source Nielsen/Net Ratings for Search Engine
  Watch.com Jan. 2003

14
Search Engines Ranked by Pages Indexed (billions)

• Google 8.1 (current)
• AllTheWeb 3.2 (09/02/03)
• Inktomi 3.0
• Teoma 1.5
• Altavista 1.0
• Inktomi is used by some of the major sites
  just taken over by Yahoo!/Overture

15
Publicity for Your Web Site

• Directories
• Directories (e.g., Yahoo!) require careful
  selection of search categories keywords
• Search for your keywords on Yahoo! to find
  appropriate categories
• Yahoo! asks for a 25-word description of content
• make it really good to impress human indexers

16
Publicity for Your Web Site

• Meta Tags (not all engines use)
• research, telecommuting research, telecommute,
  telecommutes, telecommuter, telecommuters,
  telework"
• research and papers on telecommuting,
  telecommuting productivity, telecommuting
  economic analyses, telecommuting strategies"

17
Publicity for Your Web Site

• Spiders give heavy weight to titles, headers,
  content near top of page
• Keywords in the (more than once?)
• Key words in and other headers
• Key words in other text near top of page
• How Search Engines Rank Web Pages
• Other sites linking to your site could help a lot

18
Publicize Your Web Site

• Content
• Use keywords frequently, but don't repeat same
  word more than once in a row
• OK pizza pizza
• not good pizza pizza pizza pizza pizza pizza
• Use variations of keywords (plurals)
• Use keywords in alternate text for images
• Put keywords in link text

19
Publicity for Your Web Site

• Search Engine Spamming
• Doesnt work very well anymore
• Examples
• Repeat hidden keywords
• same color as background, or
• Keywords not related to site content

20
Submitting Your Web Site to Search Engines

• Register individually with top sites
• Yahoo! , AOL, MSN others, Open Directory
  Project (goes into Google, etc.)
• Could try site submission web sites
• Submit-it to hundreds of search engines
• Change content, resubmit every month?
• don't resubmit more than once a week?

21
Evaluating Information Quality

• Source of site
• Educational institution (e.g., MIT)
• Professional organization (e.g., IEEE)
• Government agency (e.g., NASA)
• Ratings by independent evaluators
• Corroborating evidence multiple, reliable sources

22
Citing Web Information

• Whenever you use someone elses ideas, you have
  to cite them
• Format for a research paper
• American Psychological Association (APA)
• Beckleheimer, J. (1994) How do you cite URL's
  in a bibliography? WWW document. URL
  http//www.nrlssc.navy.mil/meta/bibliography.html
  
• Graphics if owner gives permission, follow
  their directions for giving credit

23
Search Engine Exercise

• Search for your keywords on any automated search
  engine (not Yahoo)
• For top 2-4 web sites, look for your keywords in
• , , , ,
  , etc. (use View, Source)
• words in page, esp. near top
• Report any patterns you see

24
Site Submit Exercise

• Identify a site to submit
• Find sites in Google related to Cal Poly
• Go through the process of submitting to a search
  engine or other submittal site
• Take notes, report back on experiences
• How long it took to submit the site
• Information required
• Etc.