The Wubbulous World of Web Assessment

1
The Wubbulous World of Web Assessment

• Assessing the success of your website

2
What will be covered in part one (theory)

• General principles of web assessment
• Planning an assessment project
• Audiences, mission, and vision
• Types of assessment, strengths and weaknesses of
  each

3
What will be covered in part two (hands-on)

• Usability testing
• Heuristics testing
• Card sort

4
What we wont cover

• Specifics of how to run tests other than the
  three in the hands-on section
• Technical details of changes made to websites
• General redesign process

5
Our objectives

• To widen your understanding of the types of
  assessment tools available to you
• To instill an appreciation for the importance of
  process and planning in assessment
• To give you knowledge and experience that will
  help you use three specific assessment tools
• To learn from you and share our experiences
• To have fun while we learn!

6
My many colored days, and how I made them an
assessment project

• General overview of web assessment and planning

7
Assessment projects require

• Project management skills (the more so the bigger
  they get)
• Skill and familiarity with the technique(s) being
  used
• Staff time and money
• Clear audience identification
• Precise understanding of what you want or need to
  know
• An articulated mission and vision for the site

8
Clear identification of audience

• Even academic library websites serve many
  different user groups. If site is large, may be
  better of focusing on a few defined groups of
  users.

9
Examples of Well-defined user groups

• Users with visual disabilities
• First-semester undergraduates
• Faculty and students in the department of science
  

10
Examples of not well-thought out user groups

• Undergraduates
• Everyone within 500 miles of us
• Huh? Wuzzat?

11
What is the end goal of this assessment?

• To develop new products and services
• To improve the ease of use of the site
• To determine if more marketing is needed about
  the site

12
Put the two together

• Our goal is to improve quality and ease of use of
  the site for first semester undergraduates
• Our goal is to develop three new services that
  would benefit patrons who speak Spanish as their
  primary language.

13
Do your goals fit in with

• The mission statement of the site?
• The audiences identified for the site?
• The vision for the site?
• Remember to involve stakeholders in the
  formulation of goals.

14
Add to packet samples of mission, vision, and
goals
15
Once you have a goal for your project

• Decide what assessment tools are most appropriate
  to find out what you want to know
• Decide how many different tests you can run with
  the resources available
• Draw up a scheduleand stick to it!
• If this is the first time, start small and narrow!

16
Remember

• Pilot test all tests under conditions as closely
  matching those of the actual test as possible
  (build this into your budget and schedule)
• Set aside a place to store all data long-term and
  assign responsibility for its upkeep
• Involve stakeholders-schedule opportunities for
  input before testing begins

17
Remember

• Utilize more than one kind of test
• Preserve what you learned about the
  testinglessons learned
• criticism is hard to takedont resist results
  because they are not what you expected

18
Packet sample test schedule

• Done in CMP (critical path method)
• Each box is task
• Boxes in sequence-tasks must be completed in that
  order
• Boxes in parallel-tasks going on simultaneously
• Document your assumptions!

19
Integrate assessment projects into process of
change
20
One Test, Two Test, Red Test, Blue Test

• The Types of Assessment Techniques

21
The Big Picture
Awareness, satisfaction, new services
Quality of the product, ease of use
Actual use
22
Oh, the Places Youll Go! (But Only if You Can
Find Them)

• Useability tests

23
What does this Kind of Testing Tell you?

• How effective is my site or service at doing what
  we built it to do?
• How easy is it for people to use the service or
  site to do what we intended it to do?

24
What Does Usability Testing Not Tell You?

• Whether users know you have a site
• Whether users know that your site is a place to
  go to perform the tasks in the first place, or
  even want to
• What specific solutions to the problems
  identified might be

25
User testing

• Recruit people from target audience, provide with
  a list of tasks to perform using the site, then
  observe how they perform the tasks.

26
Pros/cons

• Excellent way of seeing what actually happens
  when users try to use site
• Easy to run

• Behavior of a few users may not be indicative of
  general behavior
• Hard to decide on questions

27
Horton Hears an Heuristic

• Heuristics Testing

28
What are Heuristics?

• Heuristics are principles of design, originally
  developed and applied to software interfaces. If
  properly applied, they can help you catch many
  basic usability problems before they become
  problems.

29
Sample Web Heuristics

• No page should scroll more than three screens
• Writing should be short, and to the point.
• Every page should identify itself as a __________
  library page
• Navigation features should remain consistent
  throughout the site

30
How does it work?

• Draw up list of heuristics
• Get a group of people to go through the site and
  check pages against list
• Note and rate problems in severity
• Put problems in priority order
• Recommend changes

31
Pros/cons

• Easy and cheap
• Depending on the rules you use, can be very
  effective

• Does not involve real users

32
Anthropological observation

• Follow a person trying to achieve a task from
  start to finish, write down what you observe, or,
  watch a person perform regular tasks in own
  environment.

33
Pros/Cons

• Gives you context-how do related services work
  together

• Very hard to do this kind of fieldwork
• Very time consuming
• Difficult to arrange

34
I am the Lorax, I speak for the Ease!

• Marketing tests and techniques for getting user
  feedback

35
Surveys/Questionnaires

• What can they tell you?
• Attitudes
• Preferences, what customers want
• Beliefs
• What people think is true
• Behavior
• What people have done, are doing, and might do in
  the future
• Demographics
• What people are

36
The major components of any survey project

• Design of the instrument
• Determining population and selecting a sample
• Administering the survey
• Analysis of results

37
Surveys and Questionnaires

• Major issues with survey design
• What are you trying to find out?
• What will you do with the information?
• Wording of questions
• Wording of answers
• Appearance and layout
• Length
• Cover letter

38
Surveys and Questionnaires

• Major issues with population and sample
• Who is your audience
• What is the population
• Size of the sample
• Selection of the sample

39
Surveys and Questionnaires

• Major issues with analysis
• What is best way to present this data?
• How do we analyze the data?
• How do we put the data to use?

40
Surveys are good at/not good at

• Determining how widespread a belief or level of
  satisfaction is
• Gathering demographic data

• Getting at underlying reasons or motivations

41
Focus Groups and Interviews

• Can help you find out about
• Peoples needs and wants
• A range of ideas or feelings on a topic
• Uncover reasons for opinions or motivations
• Constructing good surveys, or clarify data
  gathered from surveys
• Pilot test ideas, services, or products

42
Issues with focus groups

• Need a skilled and detached moderator
• Transcription-pay for it!
• Not necessarily representative of the entire
  population

43
Good for/not good for

• Collecting ideas
• Exploring motivations and reasons why

• Educating users
• Understanding how widespread a belief or attitude
  is