Pragmatic Usability Evaluation

1
Pragmatic Usability Evaluation

• Lynne Hall

2
Assignments

• NO Assignment 2
• Assessed work will be
• Assignment A (50 )
• Portfolio (50 )
• Portfolio to be handed in on 10th of January 2000
  between 12.00 and 16.00 at EST Faculty Office

3
heuristic evaluation

• heuristic - used of problem solving
  techniques that proceed by trial and error
  (related to Greek eureka)
• Longman Concise English Dictionary, 1985
• a method of usability evaluation where an
  analyst finds usability problems by checking the
  user interface against a set of supplied
  heuristics or principles
• Lavery, Cockton and Atkinson, 1996

4
who should do heuristic evaluation?

• use more than one evaluator
• ideally should not be the designer
• ideally should be a usability specialist
• technical authors also useful
• each carries out independent inspection, then
  aggregate findings
• evaluators may need help
• unless walk-up-and-use application
• could provide typical usage scenario

5
effectiveness of increasing number of evaluators
(after Nielsen, 1993)
6
evaluator performance

• Nielsen 1992
• same interface evaluated by 3 groups
• novice (knowledge about computers only)
• usability experts
• usability and specialist domain experts

7
how to do heuristic evaluation

• go through interface, compare against recognised
  usability principles
• first pass flow of interaction and general scope
• subsequently focus on specific elements
• typically 1 - 2 hours in total
• output list of usability problems
  cross-referenced to usability guidelines

8
usability metrics

• subjective satisfaction
• efficiency

• learnability
• memorability
• errors

9
but how to set the target level...

• skill and intuition...
• better than last version
• better than the competition
• client targets
• set a range of levels
• unacceptable
• minimum
• target
• ideal

10
a usability profile
11
usability testing - planning

• draw up a test plan
• see separate handout Checklist for usability
  test plans
• informal use by the test team
• formal use for QA procedures
• consider whether to use video/audio recording

12
usability testing - users

• test users should represent target users
• remember sales staff as a special user group
• may need to give basic training
• getting hold of users
• internal users should be easy
• customers from user groups may help
• paid volunteers students, classified ads..
• take account of older users if relevant

13
pay-off ratio for user testing (after Nielsen,
1993)
14
usability testing - designing test tasks

• representative provide reasonable coverage
• do-able but not trivial
• consider relating to a larger scenario
• provide a written task description
• present in increasing level of difficulty
• decide whether to use verbal protocols

15
relative effectiveness

• Karat el (1992)
• expert individual group walkthroughs
• used guidelines and tasks
• usability testing
• users identified and described problems
• testing identified most problems
• including some severe ones missed by experts

16
effectiveness continued

• walkthroughs useful when resources linited, or
  for early design
• team walkthroughs better than individuals
• techniques are complementary
• cost effectiveness similar
• also formal experimental trials

17
usability testing - procedure

• preparation
• remember to switch off screen-savers, email, etc.
• introduction
• testing
• debriefing
• questionnaires if used
• also ask about the testing process
• write up quickly

18
usability metrics

• subjective satisfaction
• rating scales
• physiological measures
• efficiency
• times for experts to complete specified task(s)
• frequency of non-productive actions
• ratio of used to unused commands

• learnability
• time to reach specified level of proficiency
• e.g. complete a specified, representative task
• note that learning is a continuum
• memorability
• test users on commands after trial session
• errors
• number of errors in completing specified task