Content or Graphics An Empirical Analysis of Criteria for AwardWinning Websites

1
Content or Graphics? An Empirical Analysis of
Criteria for Award-Winning Websites

Rashmi Sinha, Marti Hearst Melody
Ivory University of California, Berkeley Maya
Draisin International Academy of Digital Arts and
Sciences
For more information http//sims.berkeley.edu/si
nha/Webby2000
2
What are the qualities of award-winning websites

• Webby Awards dataset is an untapped resource
• Large pool of sites are rated on multiple
  criteria
• Analysis of the dataset can inform us about the
  qualities that distinguish good from bad websites

3
Ultimate Goal Tools to Help Non-Professional
Designers

• Examples
• A grammar checker to assess guideline
  conformance
• Imperfect
• Only suggestions not dogma
• Automatic comparison to highly usable pages/sites
• Automatic template suggestions

4
The Current Goal

• What are the qualities of award-winning websites?
• Do good websites in different content categories
  differ other in terms of importance of different
  criteria?
• What is more important for website quality
  Content or Visual Design

5
Webby Awards 2000

• Fifth Year in operation
• Webby Awards 2000
• 3000 websites from 27 content categories rated on
  multiple criteria

6
Webby Awards 2000

• 6 criteria
• Overall Site Experience
• Five specific Criteria
• Content
• Structure and Navigation
• Visual Design
• Interactivity
• Functionality
• Scale 1-10 (highest)

7
Judging Process

• Open Submission Process
• Anyone can submit a site
• 3 rounds of Judging
• Site Review Stage (from 3000 to 400 sites)
• Site Nominating Stage (from 400 to 135 sites)
• Final Judging (from 135 to 27)

8
Webby Judges

• Review Stage Judges
• Internet professionals.
• Familiarity with content category
• Nominating Judges
• New media journalists, editors, web developers,
  and other Internet professionals
• Possess comprehensive knowledge of sites in their
  content category
• Peoples Voice Ratings
• People vote for their favorite sites of the final
  135

9
Review Stage Analysis

• 6 criteria
• Content, Structure Navigation, Visual Design,
  Functionality Interactivity
• Overall experience
• 3000 sites
• 3 judges rated each site

10
The whole range of sites Good to Bad
Mean 6.01 SD 1.59
11
What criteria contribute to overall rating?
12
What factors predict ratings

• The best predictor of the overall score is
  content. Also the best unique predictor.
• The worst unique predictor is visual design

13
Review Stage Interrelationship of Criteria
14
Nominating Stage Analysis

• 6 criteria
• Content, Structure Navigation, Visual Design,
  Functionality Interactivity
• Overall experience
• 414 sites
• 3 judges rated each site

15
Mostly Higher Rated Sites
Mean 7.6 SD 1.66
16
What criteria contribute to overall rating?
17
Example Webby Content CategoriesArt, Commerce
Radio
18
Art
19
Commerce Sites
20
Radio Sites
21
Conclusions

• Content is by far the best predictor of overall
  site experience.
• Visual Design does not have much predictive power

22

• More information

http//sims.berkeley.edu/sinha/Webby2000