UKOLN is supported by:

1
Usability testing for the WWW Emma Tonkin UKOLN
UKOLN is supported by
www.bath.ac.uk
2
Introduction

• UKOLN, the University of Bath
• Why this session?

3
Why do projects fail?

• Project Impaired Factors of the Responses
• 1. Incomplete Requirements 13.1
• 2. Lack of User Involvement 12.4
• 3. Lack of Resources 10.6
• 4. Unrealistic Expectations 9.9
• 5. Lack of Executive Support 9.3
• 6. Changing Requirements Specifications 8.7
• 7. Lack of Planning 8.1
• 8. Didn't Need It Any Longer 7.5
• 9. Lack of IT Management 6.2
• 10. Technology Illiteracy 4.3
• 11. Other 9.9

4
Introducing usability

• Definition the measure of a products potential
  to accomplish the goals of a user
• How easy a user interface is to understand and
  use
• Ability of a system to be used easily?
  Efficiently? Quickly?
• The people who use the project can accomplish
  their tasks quickly and easily

5
Assumptions

• There are several dimensions to usability
• Focus on users
• People use products to be productive
• Users are busy people trying to accomplish tasks
  quickly
• Users decide when a product is easy to use
• (Adapted from Redish Dumas, A Practical Guide
  to User Testing)?

6
However

• Are users always busy? Does this imply that
  usability is only present in the workplace?!
• Effectiveness vs. efficiency vs. satisfaction
• Do users know when a product is ready?
• Do all users agree about usability?
• Is usability actually measurable?
• Is there one statistic that usability?

7
Elements of usability

• Nielsen refers to five elements or components of
  usability
• Learnability
• Efficiency
• Memorability
• Errors
• Satisfaction
• Usability Engineering, 1993, p.26
• These may not be of equal importance in all cases.

8
In other words

• Usability depends on context
• What does the user want to do?
• Who is the user?
• Related to
• Internationalisation cultural, social issues
• Task analysis working out what the user wants to
  do (what the goal is) and how he/she would expect
  to accomplish it

9
Science vs craft

• Formal approaches
• Research-driven
• hard science
• Laboratory-based
• Informal approaches
• Naturalistic, qualitative observations
• Informal setting

10
User model vs user testing

• Either we apply our understanding of the way
  users act, and test the interface that way
• Or we simply observe users...

11
(No Transcript)
12
A note about rule-based testing/validation

• Should be vs is model vs reality
• Great handwriting does not guarantee a
  compellingly readable result

13
Scenario-based user testing

• Based around tasks
• Simple scenarios (hypothetical
  stories/abstract-level test cases)
• For a company web page, locating and using
  contact details
• Registration and login to a wiki
• Process provide a task and ask the user to
  complete it
• It is important to test the right tasks!

14
Cognitive walkthrough

• Works something like this

Task Climb mountain and find the highest peak
15
Required for CW

• A description of the interface
• A task scenario
• Assumptions What knowledge does the user already
  have?
• Functionality What actions will accomplish the
  task with this interface?

16
Method

• Look at each step that is required to accomplish
  the task
• Will the user try this step?
• Will the user notice that this action (control,
  button, switch) is available?
• Will the user associate this action with the
  effect that they are hoping for?
• If this action is performed, does it appear that
  progress is being made?
• Can you 'tell a success story' for each step? If
  not, there is a usability problem.

17
Recording your test

• Create a diary format
• Trying to achieve whatever
• Looking for something that does whatever
• Found a button marked foo
• But clicking on foo took me to unrelated-looking
  screen blah
• Like the mountain-climbing line, you can go back
  and try another trajectory document this in a
  similar way.

18
Developing appropriate task scenarios

• Probably the hardest thing about any usability
  testing
• On the one hand, you are not required to support
  very improbable scenarios.
• On the other hand, developing and supporting
  probable scenarios is key to a user-centred
  development process.

19
Trying out a CW

• Who's got a mobile phone?
• In groups
• Work out a couple of tasks.
• Working from the perspective of a user with an
  appropriate level of knowledge (you will have to
  define what that means!), try the tasks. Document
  the result.

20
User testing (with real users!)?

• The popular example is heuristic evaluation.
• Heuristics are rules of thumb.
• Heuristic evaluation requires about six people
  and a large amount of coffee.
• Provide them with a list of the ten (or twelve,
  or...) heuristics, and ask them to examine each
  page ('screen') for problems, according to the
  heuristics.

21
Ten heuristics

• Visibility of system status Does the system
  give timely appropriate feedback?
• Match between system and the real world Is it
  speaking the users language?
• User control and freedom How hard is it to undo
  unwanted actions?
• Consistency and standards Does it follow
  conventions and expectations?
• Error prevention Are potential errors
  recognised before becoming a problem?
• Recognition rather than recall Does the system
  rely on the users memory?
• Aesthetic minimalist design Are dialogs
  cluttered with information?
• Help users recognise, diagnose recover from
  errors Are error messages useful?
• Help and documentation Is there online help? Is
  it useful?

22
Evaluating the results

• Again, a diary form can be helpful 'Screen 1
  violates heuristic 10 because...'
• Merge these notes.
• List by frequency order to see most obvious bugs
• List by heuristic to see severity for your
  purposes

23
Applying the results
Low-hanging fruit?
Bug fixes
Strange interaction flow
Misnamed element
Confusing colours
Feature requests
It would be much easier if this textbox
autocompletedthe system remembered my email
preferences
Major objections
I dont like type of applicationI prefer
totally different type of application
(Oh)?
24
Testing layouts via greeked text

• Wasn't going to talk about this, but it's turned
  out to be useful
• Early stage of web site design often involves
  developing layouts/templates
• Because no real content exists yet, these may be
  hard to test using the above methods
• However, a layout should communicate something
  about page function. Does it?

25
Preparing a template

• Get greeked text from the Lorem Ipsum generator
• http//lorem-ipsum.perbang.dk/
• Place it into template. Do not leave a single
  readable word!
• Make yourself a list of elements that should be
  visible on the page
• Find/bribe about six test subjects

26
Example list

• Main page content
• Page title
• Person responsible for page
• Navigation elements
• Last updated date
• Logo
• How to get back to the front page?
• News items

27
Testing process

• One user at a time
• On each layout 'greeked', ask the user to
  identify each element or group of elements. If
  they can't find it, invite them to mark where
  they think it ought to be.
• Asking the user to 'think aloud' can be helpful
• Also, ask the user to give a mark (out of ten, or
  from -3 to 3, or whatever...) on 'subjective
  appeal'
• Note Randomising order reduces systematic error

28
Coming up with a preamble

• This is a strange thing to ask someone to do. Do
  not be surprised if you get some funny looks.
• Come up with a short, reassuring introduction to
  the test. Useful items to include
• Introducing the software (purpose, not detail)?
• Your participation will help us...
• Remember, we are testing the software, not your
  performance...
• Please think out loud...
• This style of test helps us to...

29
Examining the results

• Build a table
• As the layout is improved, the number of
  misidentified elements should reduce

30
Creating scenarios

• Must be
• Motivating
• Credible
• Complex
• Provide easy-to-evaluate results
• An Introduction to Scenario Testing, Cem Kaner,
  Florida Tech, June 2003
• Can be gleaned from documented requirements?

31
The test process

• A facilitator with detailed knowledge about the
  site/software is chosen to oversee the test
• They must take care not to influence the users
  behaviour!
• The tester (user) is briefed about the
  site/software
• They then go through each scenario
• Think-aloud method describing and explaining
  actions
• Talk-aloud method describing without
  explanation (considered more accurate)?
• The facilitator keeps notes and prompts the user
  where necessary
• Alternatively/additionally, the session can be
  videoed