CS 544 Discount Usability Engineering

1
CS 544 Discount Usability Engineering

• Heuristic Evaluation

Acknowledgement Some of the material in these
lectures is based on material prepared for
similar courses by Saul Greenberg (University of
Calgary), Ravin Balakrishnan (University of
Toronto), James Landay (University of California
at Berkeley), monica schraefel (University of
Toronto), and Colin Ware (University of New
Hampshire). Used with the permission of the
respective original authors.
2
Usability Inspection

• is the generic name for a set of methods based on
  having evaluators inspect or examine
  usability-related aspects of a user interface
• usability inspectors can be usability
  specialists, software development consultants
  with special expertise, end users with
  task/content knowledge, or other professionals
• 4 ways to evaluate an interface
• automatically (compute usability measures using
  evaluation software)
• empirically (usability testing with real users)
• formally (using models to calculate usability
  measures)
• informally (rule of thumb and general knowledge
  of evaluators)
• usability inspection is an informal evaluation

3
Discount Usability Engineering

• Developed by Nielson, based on using four
  techniques
• User and task observation
• Scenarios
• Simplified thinking aloud
• Heuristic evaluation

4
Interface Design and Usability Engineering

• Articulate
• who users are
• their key tasks

Brainstorm designs
Refined designs
Completed designs
Goals
Task centered system design Participatory
design User-centered design
Graphical screen design Interface
guidelines Style guides
Psychology of everyday things User
involvement Representation metaphors
Participatory interaction Task scenario
walk-through
Evaluate
Usability testing Heuristic evaluation
Field testing
Methods
high fidelity prototyping methods
low fidelity prototyping methods
User and task descriptions
Products
Throw-away paper prototypes
Testable prototypes
Alpha/beta systems or complete specification
5
Discount usability engineering

• Cheap
• no special labs or equipment needed
• the more careful you are, the better it gets
• Fast
• on order of 1 day to apply
• standard usability testing may take a week
• Easy to use
• can be taught in 2-4 hours

6
Heuristic Evaluation

• Developed by Jakob Nielsen
• Principles used to evaluate a system for
  usability problems
• Becoming very popular
• user involvement not required
• catches many design flaws
• Small set (3-5) of evaluators examine UI
• independently check for compliance with usability
  principles (heuristics)
• different evaluators will find different problems
• evaluators only communicate afterwards
• findings are then aggregated
• Can perform on working UI or on sketches

7
Why multiple evaluators?

• Every evaluator doesnt find every problem
• Good evaluators find both easy hard ones

8
Heuristic Evaluation Process

• Evaluators go through UI several times
• inspect various dialogue elements
• compare with list of usability principles
• consider other principles/results that come to
  mind
• Usability principles
• Nielsens heuristics
• supplementary list of category-specific
  heuristics
• competitive analysis user testing of existing
  products
• Use violations to redesign/fix problems

9
Neilsons Heuristics (original)

• H1-1 Simple natural dialog
• H1-2 Speak the users language
• H1-3 Minimize users memory load
• H1-4 Consistency
• H1-5 Feedback
• H1-6 Clearly marked exits
• H1-7 Shortcuts
• H1-8 Precise constructive error messages
• H1-9 Prevent errors
• H1-10 Help and documentation

10
H2-1 Visibility of system status

• keep users informed about what is going on
• example pay attention to response time
• 0.1 sec no special indicators needed, why?
• 1.0 sec user tends to lose track of data
• 10 sec max. duration if user to stay focused on
  action
• for longer delays, use percent-done progress bars

11
H2-1 Visibility of system status

• keep users informed about what is going on
• Appropriate visible feedback

What mode am I in now?
What did I select?
How is the system interpreting my actions?
12
H2-2 Match between system real world

• speak the users language
• follow real world conventions
• (bad) example Mac desktop
• Dragging disk to trash
• should delete it, not eject it!

13
H2-2 Match between system real world

• speak the users language
• follow real world conventions
• e.g. withdrawing money from a bank machine

14
H2-3 User control freedom

• exits for mistaken choices, undo, redo
• dont force down fixed paths

How do I get out of this?
15
H2-3 User control freedom

• exits for mistaken choices, undo, redo
• dont force down fixed paths
• Strategies
• Cancel button (for dialogs waiting for user
  input)
• Universal Undo (can get back to previous state)
• Interrupt (especially for lengthy operations)
• Quit (for leaving the program at any time)
• Defaults (for restoring a property sheet)

16
H2-4 Consistency standards

• Consistency of effects
• same words, commands, actions will always have
  the same effect in equivalent situations
• predictability
• Consistency of language and graphics
• same info/controls in same location on all
  screens/dialog boxes
• forms follow boiler plate
• same visual appearance across the system (e.g.
  widgets)
• e.g. different scroll bars in a single window
  system!
• Consistency of input
• consistent syntax across complete system

17
H2-4 Consistency standards
These are labels with a raised appearance. Is it
any surprise that people try and click on them?
18
H2-5 Error prevention

• try to make errors impossible
• modern widgets only legal commands selected,
  or legal data entered
• Provide reasonableness checks on input data
• on entering order for office supplies
• 5000 pencils is an unusually large order. Do you
  really want to order that many?

19
H2-6 Recognition rather than recall

• Computers good at remembering things, people
  arent!
• Promote recognition over recall
• menus, icons, choice dialog boxes vs command
  lines, field formats
• relies on visibility of objects to the user (but
  less is more!)

20
H2-7 Flexibility and efficiency of use

• Experienced users should be able to perform
  frequently used operations quickly
• Strategies
• keyboard and mouse accelerators
• abbreviations
• command completion
• menu shortcuts
• function keys
• double clicking vs menu selection
• type-ahead (entering input before the system is
  ready for it)
• navigation jumps
• e.g., going to window/location directly, and
  avoiding intermediate nodes
• history systems
• WWW 60 of pages are revisits

21
H2-7 Flexibility and efficiency of use
Keyboard accelerators for menus
Customizable toolbars andpalettes for frequent
actions
Split menu, with recently used fonts on top
Double-click raises toolbar dialog box
Double-click raises object-specific menu
Scrolling controls for page-sized increments
22
H2-8 Aesthetic and minimalist design

• No irrelevant information in dialogues
• Bad example

23
H2-9 Help users recognize, diagnose, and recover
from errors

• error messages in plain language
• precisely indicate the problem
• constructively suggest a solution

24
Errors we make

• Mistakes
• arise from conscious deliberations that lead to
  an error instead of the correct solution
• Slips
• unconscious behaviour that gets misdirected en
  route to satisfying goal
• e.g. drive to store, end up in the office
• shows up frequently in skilled behaviour
• usually due to inattention
• often arises from similarities of actions

25
Types of slips

• Capture error
• frequently done activity takes charge instead of
  one intended
• occurs when common and rarer actions have same
  initial sequence
• change clothes for dinner and find oneself in bed
  (William James, 1890)
• confirm saving of a file when you dont want to
  delete it

I cant believe I pressed Yes...
26
Types of slips

• Description error
• intended action has much in common with others
  that are possible
• usually occurs when right and wrong objects
  physically near each other
• pour juice into bowl instead of glass
• go jogging, come home, throw sweaty shirt in
  toilet instead of laundry basket
• move file to trash instead of to folder
• Loss of activation
• forgetting what the goal is while undergoing the
  sequence of actions
• start going to room and forget why you are going
  there
• navigating menus/dialogs and cant remember what
  you are looking for
• but continue action to remember (or go back to
  beginning)!
• Mode errors
• people do actions in one mode thinking they are
  in another
• refer to file thats in a different directory
• look for commands / menu options that are not
  relevant

27
Designing for slips

• General rules
• Prevent slips before they occur
• Detect and correct slips when they do occur
• User correction through feedback and undo
• Examples
• mode errors
• have as few modes as possible (preferably none)
• make modes highly visible
• capture errors
• instead of confirmation, make actions undoable
• allows reconsideration of action by user
• e.g. Mac trash can can be opened and deleted
  file taken back out
• loss of activation
• if system knows goal, make it explicit
• if not, allow person to see path taken
• description errors
• in icon-based interfaces, make sure icons are not
  too similar,
• check for reasonable input, etc.

28
Generic system responses for errors

• General idea Forcing functions
• prevent / mitigate continuation of wrongful
  action
• Gag
• deals with errors by preventing the user from
  continuing
• eg cannot get past login screen until correct
  password entered
• Warn
• warn people that an unusual situation is
  occurring
• when overused, becomes an irritant
• e.g.,
• audible bell
• alert box
• Do nothing
• illegal action just doesnt do anything
• user must infer what happened
• enter letter into a numeric-only field (key
  clicks ignored)
• put a file icon on top of another file icon
  (returns it to original position)

29
Generic system responses for errors

• Self-correct
• system guesses legal action and does it instead
• but leads to a problem of trust
• spelling corrector
• Lets talk about it
• system initiates dialog with user to come up with
  solution to the problem
• compile error brings up offending line in source
  code
• Teach me
• system asks user what the action was supposed to
  have meant
• action then becomes a legal one

30
H2-10 Help and documentation

• Help is not a replacement for bad design!
• Simple systems
• walk up and use minimal instructions
• Most other systems
• feature rich
• some users will want to become experts rather
  than casual users
• intermediate users need reminding, plus a
  learning path
• Many users do not read manuals
• prefer to spend their time pursuing their task
• Usually used when users are in some kind of
  panic, need immediate help
• indicates need for online documentation, good
  search/lookup tools
• online help can be specific to current context
• paper manuals unavailable in many businesses!
• e.g. single copy locked away in system
  administrators office
• Sometimes used for quick reference
• syntax of actions, possibilities...
• list of shortcuts ...

31
Types of help

• Tutorial and/or getting started manuals
• short guides that people are likely to read when
  first obtaining their systems
• encourages exploration and getting to know the
  system
• tries to get conceptual material across and
  essential syntax
• on-line tours, exercises, and demos
• demonstrates very basic principles through
  working examples
• Reference manuals
• used mostly for detailed lookup by experts
• rarely introduces concepts
• thematically arranged
• on-line hypertext
• search / find
• table of contents
• index
• cross-index

32
Types of help (cont.)

• Reminders
• short reference cards
• expert user who just wants to check facts
• novice who wants to get overview of systems
  capabilities
• keyboard templates
• shortcuts/syntactic meanings of keys recognition
  vs. recall capabilities
• tooltips
• text over graphical items indicates their meaning
  or purpose

33
Types of help (cont.)

• Context-sensitive help
• system provides help on the interface component
  the user is currently working with
• Macintosh balloon help
• Microsoft Whats this help
• brief help explaining whatever the user is
  pointing at on the screen
• Wizards
• walks user through typical tasks
• but dangerous if user gets stuck
• Remember the MS paperclip?

34
How to perform evaluation

• At least two passes for each evaluator
• first to get feel for flow and scope of system
• second to focus on specific elements
• If system is walk-up-and-use or evaluators are
  domain experts, no assistance needed
• otherwise might supply evaluators with scenarios
• Each evaluator produces list of problems
• explain why with reference to heuristic or other
  information
• be specific and list each problem separately

35
How to perform evaluation (cont.)

• Why separate listings for each violation?
• risk of repeating problematic aspect
• may not be possible to fix all problems
• Where problems may be found
• single location in UI
• two or more locations that need to be compared
• problem with overall structure of UI
• something that is missing
• hard w/ paper prototypes so work extra hard on
  those
• note sometimes features are implied by design
  docs and just havent been implemented relax
  on those

36
Severity ratings

• Used to allocate resources to fix problems
• Estimates of need for more usability efforts
• Combination of
• frequency
• impact
• persistence (one time or repeating)
• Should be calculated after all evals. are in
• Should be done independently by all judges

37
Severity ratings (cont).

• 0 - dont agree that this is a usability problem
• 1 - cosmetic problem
• 2 - minor usability problem
• 3 - major usability problem important to fix
• 4 - usability catastrophe imperative to fix

38
Summary

• Advantages
• the minimalist approach
• a few general guidelines can correct for the
  majority of usability problems
• easily remembered, easily applied with modest
  effort
• discount usability engineering
• cheap and fast way to inspect a system
• can be done by usability experts, double experts,
  and end users
• Problems
• principles are more or less at the motherhood
  level
• cant be treated as a simple checklist
• subtleties involved in their use
• doesnt necessarily predict user/customers
  overall satisfaction
• may not have the same credibility as user test
  data (one solution to this is to include the
  design team and developers in the usability
  evaluation, as in a pluralistic walkthrough)

39
Summary

• Research result
• between 4-5 evaluators appear to be sufficient in
  most cases to identify 80 of total usability
  problems
• user testing and usability inspection have a
  large degree of non-overlap in the usability
  problems they find (i.e., it pays to use both
  methods)
• Cost-benefit
• usability engineering activities are often
  difficult to justify and carry out in a timely
  way, but many activities can be done quickly and
  cheaply, and produce useful results
• usability inspection turns less on what is
  correct than on what can be done within
  development constraints
• ultimate trade-off may be between doing no
  usability assessment and doing some kind

40
You now know

• 10 Heuristics for evaluating designs
• H2-1 Visibility of system status
• H2-2 Match between system real world
• H2-3 User control freedom
• H2-4 Consistency standards
• H2-5 Error prevention
• H2-6 Recognition rather than recall
• H2-7 Flexibility and efficiency of use
• H2-8 Aesthetic and minimalist design
• H2-9 Help users recognize, diagnose, and recover
  from errors
• H2-10 Help and documentation
• Principles can be used to systematically inspect
  the interface for usability problems

41
Readings and References

• BGBG 80-86 (Evaluating Systems and Their User
  Interfaces)
• BGBG 170-181 (Usability Inspection Methods)
• Optional
• Neilsen, J. (1993). Usability Engineering. Read
  Chapter 5 Usability Heuristics, 115 163.