Basics of Search Engine Optimization

1
Basics of Search Engine Optimization

• Bob Keating
• robert_keating_at_sra.com
• Web Manager University
• March 19, 2007

2
Course Summary

• Overview of Search Engine Optimization
• How Search Engines Work
• What Search Engines Like
• Basics of Search Friendly Design
• Site Maintenance
• Resources

3
Course Goals

• Understand how web search engines handle your
  content
  
• Design, write and publish search engine friendly
  pages
  
• Maximize visibility of your pages for search
  engines and users
  
• Attract attention of search engines when
  publishing content

4
SEO vs. SEM

• What is Search Engine Optimization (SEO)?
• A set of web design and development methodologies
  aimed at improving the ranking of web pages in
  search engine results
  
• What is Search Engine Marketing (SEM)?
• Methodologies for submitting and positioning a
  website within proper search engines for maximum
  exposure.

5
Why is SEO Important?
6
People Use Search Engines

• Search is the second most popular activity after
  e-mail
  
• 60 million Americans using search engines during
  a typical day, an increase of 55 from June 04
  to Sept 05
  
• Hundreds of millions of searches each day
• Industry watchers estimate that Google gets 59
  million unique visitors producing over 250
  million search queries each day
  

7
The Golden Triangle

• Gord Hotchkiss Eye Tracking Study -
  http//www.enquiro.com/eyetrackingreport.asp
  
• Shows scan patterns on search results pages, and
  the interplay among page elements
  
• Key Findings
• Difference in eye patterns above the fold than
  below the fold
  
• Increase in visibility clicks for top links
  than side links
  
• Importance of rank for both organic and paid
  searches
  

8
Program Performance

• Web metrics are used to report on program
  performance to senior management
  
• Maximize web traffic via the features citizens
  use most
  
• 54 of Web users are looking for a government
  website you want to rank high for your agencys
  keywords

9
Government Policy

• OMB Memorandum, Dec. 2005
• When disseminating information to the
  public-at-large, publish your information
  directly to the Internet. This procedure exposes
  information to freely available and other search
  functions and adequately organizes and
  categorizes your information.

10
OMB Memorandum

• Major Assumptions
• Publishing information to the Internet means
  search engines will find it
  
• Publishing information to the Internet means the
  information is accessible
  
• Webmasters will take any additional steps to
  ensure information is found and adequately
  organized by search engines
  
• Webmasters understand how search engines work

11
How Search Engines Work
12
Search Crawling Overview
Request URL from your web server, which gives the
page to the Search engine
Follow a link to URL
Spider starts at major ISPs and Directories
Web Page
Search Index
Crawler reads page makes a list of words and
phrases
User search index Most are completely updated 4-6
wks
Search Engine applies ranking algorithms,
factoring in text, link and popularity components
13
Major Search Engines and Directories

• Search Engine Producers
• Google
• Yahoo!
• Live Search (MSN)
• Ask.com
• Gigablast
• WiseNut

• Web Directories
• Open Directory Project (DMOZ)
• Yahoo! Directory

14
Bruce Clays Search Engine Chart
http//www.bruceclay.com/searchenginechart.pdf
15
What About Web Directories?

• No longer featured prominently on search engines
• Directory data used to enhance search results
• Directories used to seed crawlers and
  metasearch engines
  
• Open Directory Project (DMOZ) data is syndicated
  to hundreds of websites, including the Google
  Directory

16
Metasearch Engines
17
comScore Media Metrix Search Ratings
Change Since 2005 Google 7.2 Yahoo - 1.
7 MSN - 2.7 AOL - 4.0 Ask -
0.7

Source Search Engine Watch, Aug. 21, 2006,
http//searchenginewatch.com/showPage.html?page21
56431

18
What Do Search Engines Like?
19
Basic Page Components

• Text (Keyword) Component
• Words and phrases that match what your target
  audience types into search engines
  
• Link Component
• Site navigation and URL structure that search
  crawlers can easily follow
  
• Popularity Component
• Are sites link to you?

20
Keyword Component

• Choose one or two keywords or phases to optimize
  for each page
  
• Do not over use - avoid keyword stuffing
• Over use of keywords can result in being
  penalized or ignored
  
• Incorporate other complimentary words and
  phrases
  
• Check Keyword Density
• http//www.webjectives.com/keyword.htm
• http//www.keyworddensity.com/

21
Keyword Prominence

• Search engines place weight on terms according
  to where they are used
  
• Place Keywords in
• Title tags
• Headings and emphasized text
• Visible body text
• Description meta tags
• Alt text in images
• Title and body tags are most important
• Keyword in the URL is helpful, but not a
  significant factor in ranking

22
Metadata Myth
Quote Metadata improves search relevancy
False. Except for TITLE and META description
tags, Web search engines ignore other metadata.
However, Metadata matters to enterprise search.
Quote Using standard metadata is a best practi
ce False. Only best practice if the metadata is
used by your agency for specific applications
(e.g. enterprise search or content syndication).
It is not a best practice among professional web
designers.
23
Metadata Myth

• Quote If more agencies were diligent about
  adding metadata, Google would pay attention
  
• False. Industry is focused on developing
  algorithms that determine relevancy based on
  content rather than what an author or metadata
  creator says about the content
• Key points
• Metadata is not the key to high rankings
• Use metadata if it is important to your agencys
  mission
  
• Concentrating on keyword usage in other areas
  will have a greater impact on relevancy
  
• Good practice to include META description,
  keywords, and a descriptive tag.

24
Link Component

• Pages will not rank well if your site does not
  have a navigation scheme
  
• Navigation scheme must please users and search
  engines
  
• Create a site map, but also plan how pages link
  to each other
  
• Avoid dangling pages

25
Creating Links

• Keywords in links tell crawlers about the pages
  to which you are linking
  
• Keywords in links influence relevancy of the page
  to which you are linking
  
• Avoid click here links, instead create links
  like
  
• Bad Click here for more info on famous
  admirals.
  
• Good Visit our naval history site for more info
  on famous admirals.

26
Popularity Component

• Based on the number pages that link to you
• The more popular pages that link to you, the
  higher your popularity
  
• All search engines have different popularity
  algorithms
  
• Googles algorithm is called Page Rank
• Every page on the Web is given a calculation of
  it is popularity based on inbound links

27
Popularity Factors

• Number and Popularity of Inbound Links
• Get listed Yahoo!, DMOZ
• Network with other agency and industry sites
• Make your site a link magnet
• Anchor Text
• Others use of keywords in a link to your site
• Popularity is assigned per page, not for the
  entire site
  
• Popularity is not inherited
• Need to deliberately link internal pages to pass
  on PR

28
Outbound Links

• Popularity is not determined by the sites you
  link to
  
• Outbound links affect the popularity of the sites
  youre linking to
  
• Internal links and inbound links have the most
  impact on your popularity
  
• Outbound links help identify you with a hub

29
Robots Exclusion

• Meta-Tag Robots Exclusion
• nofollow
  
• Robots.txt File
• Place in servers root directory
• Two elements User-agent, Disallow
• Example
• User-agent
• Disallow /cgi-bin/
• Disallow /scripts/
• Disallow /images/

30
Robots Exclusion

• Not all search engines pay attention to
  robots.txt instructions
  
• MSN and Yahoo! obey robots exclusion more often
  than Google
  
• USA.gov Users never exclude msnbot you wont
  have site search
  
• Blocking bots is contrary to OMBs guidance

31
Case Study NOAA Fisheries

• Term Fisheries
• 1 on USA.gov
• 2 on MSN
• 2 on Yahoo
• 3 on Google

• Term Fish
• Doesnt show up on the first page or subsequent
  pages

32
(No Transcript)
33
Case Study NOAA Fisheries

• Text Component page is well-optimized for
  fisheries but not fish
  
• Link Component Fly-out navigation may be a
  problem for some crawlers. Add search friendly
  navigation (e.g., site map)
  
• Popularity Component Lots of inbound links using
  fisheries (not fish) in anchor text

34
Basics of Search Friendly Design
35
Basic Concepts

• Page Content using content and text that target
  your audiences and attract search engines and
  links from other sites
  
• Navigation giving users and crawlers easy access
  to content
  
• Design Considerations make sure bells and
  whistles dont undermine SEM efforts
  
• Page Rank link popularity

36
Managing Page Content
37
Optimizing Page Content

• How you write, design, and optimize a page
  depends on the type of page
  
• Page content should appear focused, and tagged
  deliberately to appeal to search engines and
  users
  
• Title tag, headings, contextual links,
  cross-links
  
• Body text should be visible, i.e., should not
  have to do any action to view main page text
  

38
Primary vs. Secondary Text

• Primary Text
• Title tag
• Body text
• Text near the top of the page
• Text in and around links (e.g. anchor text)

• Secondary Text
• Alt text
• Description tag
• Domain name and URL elements

39
Keywords in Text

• Use language of your audience
• Use keywords consistently, but do not overuse
• Use keyword selection tools
• Yahoo!/Overture http//inventory.overture.com/
• Google https//adwords.google.com/select/KeywordS
  andbox
  
• Keywords in links tell crawlers about pages, and
  influence relevancy of the pages your linking to
  
• No click here links

40
Content Controversy Which is Best?

• Write your own
• Pros Original, unique content
• Cons Time consuming, bureaucratic process
• Use someone elses
• Pros Easy way to bulk up content
• Cons Still need to worry about how to add value
  in order to attract links

41
Syndicated Content

• Tends to update frequently
• Crawlers visit frequently-updated pages more
  often
  
• Combine syndicated content to create a unique
  resource
  
• Syndicate your own content (e.g. RSS)
• Increase site visibility and attracts traffic
• Another opportunity for inbound links

42
Site Navigation
43
Problem Navigation Schemes

• Poor HTML coding
• Image maps
• Frames
• JavaScript
• Dynamic Pages
• Flash

44
Site Navigation Scheme

• Text Links
• Very search engine friendly
• Use for primary or secondary navigation
• Problems with Text Links
• Can negatively skew keyword density
• Crawlers tend to read text links first

45
Site Navigation Scheme

• Navigation buttons
• Okay as long as you include alt text
• Avoid JavaScript, unless you can provide
  navigation crawlers can follow
  
• Recommendation use alt text and text navigation
  at the bottom of the page allows you put
  keywords in multiple places

46
Site Navigation Scheme

• Image Maps
• Crawlers ignore links inside image maps
• Use text links or navigation buttons elsewhere
• Pull Down Menus
• Generally not crawler friendly because they need
  JavaScript or a CGI program
  
• Always provide two forms of navigation one for
  your users, and one for your crawlers

47
Help Crawlers Navigate

• Create a Site Map
• Subscribe to Google Site Maps
• A crawler enabling tool to assist the Google
  crawler
  
• Analyzes information about your sites
  architecture to improve crawling

48
Design Considerations
49
Design Considerations

• Bells and whistles (Flash, JavaScript, animation)
  enhance user experience but can hurt search
  visibility
  
• Implement features carefully in order to keep
  search engine ranking
  
• Ensure design team understands SEO concepts

50
Use External JavaScript and CSS

• Using JavaScript on site navigation can greatly
  decrease site crawlability
  
• Crawlers do not follow links embedded inside
  JavaScript code or they limit the types of
  embedded links crawled
  
• JavaScript in can decrease page load time
• Long download times may indicate the site is
  spam, and crawlers could ignore

51
Use External JavaScript and CSS

• External files decrease page load time for
  visitors
  
• External files decrease download time for
  crawlers
  
• Remember to disallow crawling scripts in the
  robots.txt
  
• External scripts are easier to re-use

52
Frames

• Do not design in frames
• Crawlers have trouble getting from the frameset
  page to the actual web page
  
• Frameset does not provide crawler with keyword
  rich text and links
  
• Each page is indexed separately, so pages that
  only make sense as a frameset will be indexed
  individually in search engines
  
• tag is ignored due to spam

53
Frames Workarounds

• Add Navigation
• Give all pages unique title and description tags
• Put navigation links on your pages
• Add Java Script in your tag
• if (top self) self.location.href
  index.html
  
• // ?
• This will force the browser to always load the
  frameset
  
• However, browser will always load the home page,
  not the indexed page
  
• Back button will be disabled

54
Flash

• Few search engines crawl links embedded inside a
  Flash navigation scheme
  
• Flash sites contain little text
• If you include Flash
• Include a Skip link so both the user and
  crawler can go to the real homepage
  
• Include title and description meta tags

55
Dynamic Pages

• Database-driven, created on the fly by asp, cfm,
  php, jsp or cgi scripts
  
• Dynamic sites are comprised of templates, but
  usually without original content
  
• When a page is viewed, the template loads the
  content from the database
  
• Parameters are added to the URL, which tells the
  template to load specific content

56
Dynamic Pages

• Example http//smithsonianstore.com/catalog/produ
  ct.jsp?productId14273parentCategoryId3151categ
  oryId3152
  
• URLs such as this are difficult for search
  engines to index because they do not know the
  parameters that define a unique page
  
• The more parameters, the less likely pages will
  be indexed
  
• A database may continually feed data, crashing
  your server and scaring off the crawler

57
Search Friendly Dynamic Pages

• Create static HTML pages
• Modify URLs so they dont look like dynamic
  pages, fewer parameters
  
• Use URL re-write trick using mod_rewrite

58
Session IDs

• The kiss of death if left unmanaged
• Same content is delivered to the crawler but as
  unique URLs
  
• Crawlers will ignore web pages with session IDs
• Omit session IDs if the requestor is a crawler
  but no cloaking!
  

59
Optimizing PDFs

• Make sure PDFs contain actual text, not images of
  text
  
• Same rules for use of keywords and phrases apply
• Put the most important text in the title,
  headlines
  
• Minimize document size ( 100K)
• Create optimized HTML pages for PDFs

60
Managing Page Rank
61
Understanding Page Rank

• All search engines assign a value to your site
  based on inbound links
  
• Google calls this relevancy factor Page Rank
  (PR) - synonymous with popularity ranking
  
• An inbound link is a vote for your page
• An outbound link is a vote for the page youre
  linking to

62
Understanding Page Rank

• A page is assigned PR as soon as its indexed
• Make sure all your pages have at least one link
  back into your site
  
• The page receiving the most inbound links gets
  the highest page rank
  
• You can pass PR around to pages on your site

63
Can You Leak Page Rank?

• Controversial topic among SEOs
• Supporting View PR leaks in the sense that a
  pages outbound links will decrease that pages
  available PR for redistribution throughout the
  site
• Opposing View Search engines analyze inbound and
  outbound links to determine your authority as
  hub. No site is an island
  

64
Outbound Link Strategy

• Do not create pages with mostly outbound links
• Link to quality, related sites helps to
  establish you as an authority or hub
  
• If a page contains several outgoing links, also
  include links to other pages on your site
  
• Dont be afraid to link to sites with low page
  rank, but quality content

65
Getting Links to Your Site

• Links from Yahoo! and DMOZ
• Impact on popularity may vary
• Helps to get noticed by crawlers
• Establish reciprocal link arrangements with
  agencies covering similar topics
  
• Reach out to state and local agencies
• Syndicate your content
• Reach out to professional communities of interest

66
Links that Arent Links

• Link has been created in a manner that search
  engines cant understand
  
• Link actually points elsewhere, and not directly
  to your site
  
• Example
• http//www.site.gov/external/external.jsp?urlhttp
  //www.noaa.gov/
  
• Link passes through a JSP program that logs the
  click and re-directs the browser to noaa.gov
  
• Search engines read this as an internal link to
  Site.gov, not an outbound link to NOAA
  
• Links from these sites will not help boost your
  search ranking

67
Site Maintenance
68
Re-Designing Your Site

• Plan BEFORE you embark on a re-design
• Will the site architecture change?
• Static to Dynamic?
• How is the content changing?
• Add SEO review as an activity to your project
  plan
  
• Make sure your contractor has SEO skills
• Poor planning and execution can kill your search
  rankings
  
• Do not wait until after you re-designed!

69
Changes to Site Architecture

• Try to keep the same filenames and directory
  structure when redesigning
  
• Follow MSNs What to do when your site moves
  http//search.msn.com/docs/siteowner.aspx?tSEARCH
  _WEBMASTER_REF_Redirectcode.htm
  
• Recommended Set up HTTP 301 redirects that point
  to the new site or pages
  
• Not Recommended Add a meta-refresh tag to your
  page header. This wont remove your original
  page from the MSN index, and thus your sites
  search engine.

70
Removing Content

• Avoid using default 404 pages
• Custom 404 pages are more user-friendly
• Submit all 404 URLs to the search engines using
  their Add URL form quickest way to get the 404
  pages out of the index
  
• Remember to change the HTTP status code on custom
  error pages from 200 to 404
  

71
Resources and Recommendations
72
Next Steps

• Visit WebContent.gov Guide to Search Engines
• http//www.usa.gov/webcontent/technology/search/se
  arch.shtml
  
• Conduct an SEO best practices review
• Conduct keyword analysis and determine your rank
• Set success measures
• Re-submit your site
• Use Webmaster Tools offered by the search
  engines, (e.g. Google Site Maps, etc.)

Make SEO a routine in your web content management
program
73
Recommended Reading

• Battelle, John. The Search How Google and Its
  Rivals Rewrote the Rules of Business and
  Transformed our Culture. New York. Penguin,
  2005.
• Kent, Peter. Search Engine Optimization for
  Dummies. Hoboken, NJ. Wiley, 2004.
  
• Moran, Mike, Hunt, Bill. Search Engine Marketing,
  Inc. Upper Saddle River, NJ. IBM Press, 2006.
  
• Thurow, Shari. Search Engine Visibility.
  Indianapolis, IN New Riders, 2003.
  

74
SEO Forums Events

• Industry Forums
• Webmaster World http//www.webmasterworld.com/
• Search Engine Watch http//www.searchenginewatch.
  com/
  
• Digital Point Forums http//forums.digitalpoint.c
  om/
  
• High Rankings http//www.highrankings.com/forum/
  
• Industry Events
• Search Engine Strategies http//www.searchengines
  trategies.com/
  

75
Bob KeatingSRA Touchstone Consulting
Grouprobert_keating_at_sra.com(202) 449-7132
Contact Information