INBS 540: Week 8

1
INBS 540 Week 8

• Usability Testing
• Task Design
• Testing Procedures

2
(No Transcript)
3
Usability Testing (Quick review)

• from Websites That Work Designing with Your
  Eyes Open a tutorial at CHI2001, by Jared
  Spool, Matthew Klee, Lori Landesman, User
  Interface Engineering (uie.com)
• Basic definition of usability
• Can users accomplish their goals?
• Shades of gray how effectively can users
  accomplish their goals?
• Obviously we need to know
• Companys goals
• Who are the users?
• What are the users goals?
• Do different users have different goals?

4
User Goals

• We used to think users had 5 distinct goals
• Find specific information
• Browse (aka surfing, or opportunistic surfing)
• Community
• Commerce
• Entertainment

5
User Goals Missions

• We now understand that users see the web as a
  tool that helps them accomplish missions that
  comprise larger goals
• User goals what users are trying to accomplish
• long term, big picture, not necessarily
  web-related
• Eg., get my printer working so I can run my home
  office
• User mission steps users take to achieve their
  goals
• Specific tasks
• Eg., find and download a driver for my inkjet
  printer

6
Understanding Users Goals

• A good User Experience or User Scenario document
  will help you encourage your team to think about,
  and design for,
• Your users goals, and why they will come to your
  site
• Their missions or specific tasks they will need
  to accomplish on your site

7
The Process

• Test current site (if there is one)
• Use the data to tell you what to focus on
• Mock up a design
• Test the mock up
• Use data to
• Fix the mockup
• Change parts of the site you wont test

8
Test Use, not Opinions

• Watch users behavior more than what they say
  they like theres a big difference
• Testing opinions is a focus group
• Focus groups can be misleading
• Remember New Coke? Focus groups loved it
• Minivans? Focus groups said no way, bad idea

9
Next Usability Topic

• Designing Running Usability Tests
• Discount User Testing on a budget

10
(No Transcript)
11
Designing Running Usability Tests

• Types of Tests
• The Testing Process
• Materials Logistics
• How often How many people?
• Designing Tests
• Analyzing Results
• User Testing on a Budget

12
Types of Tests
13
Testing Process Overview
14
What do you need to test a web site?

• Computer with Internet access
• Person to monitor the test participant
• where they went,
• what they clicked,
• how long did it take
• A task, or list of tasks for the test participant
  to perform
• Video camera (optimum 2 cameras) to capture
• Participants face
• Computer screen
• Non-disclosure agreement
• http//www.secretsites.com/legal/set_legal.html
• Cash incentive for participants (or some other
  incentive)
• Post-test Survey form

15
Why videotape the session?

• Documentation for other members of the
  development team
• If something goes wrong, the test participant
  will have some reaction on face and on screen
• Debriefing (end of session)
• Tester monitor review session (run through
  video)
• Stop video at interesting points to discuss
• Reasons participant performed the task the way
  they did
• What participant would like to happen

16
Layout of the room
17
PC is connected to notebook. Notebook displays
whatever happens on the participants PC
18
Video camera set to record notebook screen and
participant at PC (ideal setup would have 2
cameras)
19
The TV shows the video camera recording tester
test participant review this at the end of the
session
20
(No Transcript)
21
Orientation Dos

• Introduce yourself, explain why you are here
• Explain your role as a silent observer
• Explain you are not testing and there are no
  wrong answers
• Explain participant can think aloud if desired
• Keep session relaxed, use humor when appropriate
• Stay impartial

22
Orientation Don'ts

• Don't tell them what the tasks are in advance, or
  how many
• Don't say "this is so easy" - no expectations is
  better
• Don't rescue participants when struggling
• Know when to 'give up' and list as failed task
• Don't answer questions, but respond impartially

23
Collect Basic Data

• Could they complete the task?
• Did they need help?
• How much time did it take? (track time through
  testing)
• Stumbling blocks (problems/obstacles)
• Overall observations, commentary
• Sample user grid

24
Post Test Survey

• Prepare survey on paper for user to fill in
• Ask basic questions about how user felt overall
  about site graphics, logic, copy, navigation,
  etc.
• Rate questions on a scale of 110
• Gather data about overall effectiveness of site

25
Compile and Summarize Data

• Gather data as you go
• Transfer handwritten notes to computer
• Write reports while test is still fresh in your
  mind
• Summarize data in grid format showing results,
  problem areas and comments
• Identify problem areas
• Identify global or broad-scope issues
• Identify specific task-oriented issues

26
(No Transcript)
27
Iterative Testing

• Multiple tests with a few participants reveals
  the most usability problems for the lowest budget

Lotta problems here!
Still some issues
Hey this is pretty good!
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
28
Why You Only Need to Test With 5 Users

• Some people think that usability is very costly
  and complex and that user tests should be
  reserved for the rare web design project with a
  huge budget and a lavish time schedule. Not true.
  Elaborate usability tests are a waste of
  resources. The best results come from testing no
  more than 5 users and running as many small tests
  as you can afford.
• the number of usability problems found in a
  usability test with n users is
• N(1-(1-L)n)
• N is the total number of usability problems in
  the design
• L is the proportion of usability problems
  discovered while testing a single user. The
  typical value of L is 31, averaged across a
  large number of projects we studied. Plotting the
  curve for L31 gives the following result

29
N(1-(1-L)n) where L31
30
What the Curve means

• The most striking truth of the curve is that zero
  users give zero insights.
• As soon as you collect data from a single test
  user, your insights shoot up and you have already
  learned almost a third of all there is to know
  about the usability of the design. The difference
  between zero and even a little bit of data is
  astounding.
• When you test the second user, you will discover
  that this person does some of the same things as
  the first user, so there is some overlap in what
  you learn. People are definitely different, so
  there will also be something new that the second
  user does that you did not observe with the first
  user. So the second user adds some amount of new
  insight, but not nearly as much as the first user
  did.
• The third user will do many things that you
  already observed with the first user or with the
  second user and even some things that you have
  already seen twice. Plus, of course, the third
  user will generate a small amount of new data,
  even if not as much as the first and the second
  user did.
• As you add more and more users, you learn less
  and less because you will keep seeing the same
  things again and again. There is no real need to
  keep observing the same thing multiple times, and
  you will be very motivated to go back to the
  drawing board and redesign the site to eliminate
  the usability problems.
• After the fifth user, you are wasting your time
  by observing the same findings repeatedly but not
  learning much new.

31
When To Test More Users

• You need to test additional users when a website
  has several highly distinct groups of users. The
  formula only holds for comparable users who will
  be using the site in fairly similar ways. If, for
  example, you have a site that will be used by
  both children and parents, then the two groups of
  users will have sufficiently different behavior
  that it becomes necessary to test with people
  from both groups. The same would be true for a
  system aimed at connecting purchasing agents with
  sales staff.
• Even when the groups of users are very different,
  there will still be great similarities between
  the observations from the two groups. All the
  users are human, after all. Also, many of the
  usability problems are related to the fundamental
  way people interact with the Web and the
  influence from other sites on user behavior.
• In testing multiple groups of disparate users,
  you don't need to include as many members of each
  group as you would in a single test of a single
  group of users. The overlap between observations
  will ensure a better outcome from testing a
  smaller number of people in each group. I
  recommend
• 3-4 users from each category if testing two
  groups of users
• 3 users from each category if testing three or
  more groups of users (you always want at least 3
  users to ensure that you have covered the
  diversity of behavior within the group)

32
Testing Paper Prototypes

• This test (for Sun.com) was conducted by showing
  the page to users and asking them to first
  comment on their general impression of the page
  and then to point to any element on the page that
  they thought they could click on and tell us what
  they expected would happen. This simple method
  provided us with early feedback indicating the
  importance of a prominent placement of the month
  name (since we wanted users to know that the page
  would change monthly) and the need to make the
  "What's Happening" bar look very clickable.

Question Whats the masking tape for?
33
Testing Paper Prototypes

• Three versions of the homepage (Sun.com)
  Revised for each test

A
B
C
34
Designing Tasks for Testing

• Most Usability testing for the web centers on how
  well site visitors can accomplish specific tasks
• These tasks come from the business goals of the
  site
• Examples
• Register for a newsletter
• Find XYZ product and purchase it
• Post a message on a bulletin board
• Many web tasks involve finding some information
• These tasks are called scavenger-hunt tasks
• Other tasks involve doing something besides
  finding..
• These are called verb-based tasks

35
Verb-based tasks

• Good for
• Testing functionality, logical design
• Show how different users approach a web
  application differently
• Pitfalls
• May not be realistic
• (is this something people really do?)
• May not fit into user goals missions
• You need to design tasks that DO fit

36
Scavenger-hunt tasks

• Scavenger-hunt tasks are the most common type of
  user testing for information-rich web sites
• Challenge users to find specific information
• Test users understanding of information
• Test whether the site design information scent
  are effective

37
Whats the difference?

• Whats the difference between a verb-based task
  and a scavenger-hunt task?
• When would you use one or the other?

38
Goals for Task Design

• Well-designed tasks will make the testing process
  more valuable in terms of showing how the site
  design can better accomplish business objectives
• Try to devise tasks that
• Test whether the site meets its business goals
• Determine whether users can accomplish what they
  want to do

39
Things to Look for

• You may see some of these factors affecting your
  sites usability
• List length
• Link words, Groups of links
• Scent blockers
• Levels of information
• Page length
• Page density white space
• Page layout grids

40
Exercise

• Brainstorm at least 4 tasks for your site
• Any combination of verb- or scavenger-hunt tasks
• Make up a brief scenario to motivate each task
• Avoid using giveaway words
• You will use these tasks in your Usability test
  plan

41
Simplified User Testing

• The three main rules for simplified user testing
  are
• Get representative users
• Ask them to perform representative tasks with the
  design
• Shut up and let the users do the talking
• The third rule is surprisingly difficult, while
  rule 2 requires some experience to execute well.
  Still, the main obstacle to quick and frequent
  user testing is the difficulty of finding warm
  bodies that satisfy rule 1. Most companies have
  no procedures for getting five customers to show
  up at specified times next Wednesday, and yet
  that's what is required for a successful
  usability study.