Cognitive Walkthrough Evaluation

1
Cognitive Walkthrough Evaluation

• Discount formative evaluation technique for
  learnability

2
Agenda

• Questions
• Intro to discount usability
• Cognitive walkthrough
• HW

3
Discount usability techniques

• Less demanding on resources
• Inexpensive
• Requires few participants
• Can be done relatively quickly
• Does not require fully functional prototypes

4
Discount usability techniques

• Good candidates for project evaluation plan
• cognitive walkthrough
• heuristic evaluation
• think aloud

5
Cognitive walkthrough some basic facts

• A discount usability technique
• Uses a small number of HCI experts
• Focus is on learning through exploration
• Check for potential usability problems
• Derived from software engineering code
  walkthrough
• From Polson, Lewis, et al. at UC Boulder

6
Cognitive walkthrough

• Requires prototype or fairly detailed description
• Requires a description of the user task to be
  analyzed
• Requires a complete, written list of actions
  necessary to complete the task
• Requires an indication of who the users are and
  their assumed knowledge

7
Procedure

• Define required inputs
• Walk through action sequences for task
• Record critical information
• Obtain believability story

8
Inputs (what the study designer does)

• Select interaction task
• Define interaction action sequence
• Identify users
• What knowledge experience
• Prototype
• Must know how interface looks for each step

9
Selecting the task

• Task should be a one that would be common or
  typical for a potential user
• Should be representative of what users would want
  to do with the system

10
Defining the interaction sequence

• Tasks should be broken down until any further
  division yields obvious subtasks
• E.g. Type run at the prompt
• Not Type r at the prompt, Type u at the
  prompt

11
Identify users knowledge and experience

• Educate the HCI experts on the domain knowledge
  and characteristics of the user
• Give them a perspective from which to evaluate
  the system
• Base this information on what you have learned
  about your users!

12
Prototype

• Need not be functional but
• Must be at a level of detail where any action
  necessary to complete the task is defined

13
Doing the walkthrough(what the HCI expert does)

• Address each step of task sequence in turn
• Formulate a believability story
• answer 4 questions

14
Believability story

• Will the user be trying to produce whatever
  effect the action has?
• 2. Will the user be able to notice that the
  correct action is available?
• 3. Once the user finds the correct action at the
  interface, will she know that it is the right one
  for the effect she is trying to produce?
• 4. After the action is taken, will the user
  understand the feedback given?

15
Believability story

• Also includes justification for each answer based
  on the interface, knowledge of HCI, and
  understanding of users

16
Question 1

• Will the user be trying to produce whatever
  effect the action has?

17
Common supporting evidence

• It is part of their original task
• They have experience using the system
• The system tells them to do it

18
No supporting evidence?

• Construct a failure story.
• Why would the user not be trying to do this?

19
Question 2

• Will the user be able to notice that the correct
  action is available?

20
Common supporting evidence

• Known through experience
• Visible device, such as a button
• Visible representation of an action, such as a
  menu entry)

21
Failure story

• Why would the user not notice that the action is
  available?

22
Question 3

• Once the user finds the correct action at the
  interface, will she know that it is the right one
  for the effect she is trying to produce?

23
Common supporting evidence

• Based on past experience with similar
  interactions
• The interface provides a prompt or label that
  connects the action to what she is trying to do.
• All other actions look wrong.
• If not, why not?

24
Question 4

• After the action is taken, will the user
  understand the feedback given?

25
Common supporting evidence

• Past experience with similar interactions
• Recognizing a connection between a system
  response and what she was trying to do.
• If not, why not?

26
Example

• Is Delta flight 827 from Atlanta to Jackson on
  time today?
• Using a web browser, go to the Delta website by
  typing www.delta.com in the URL field
• Type 827 in the search field labeled Flight
  number in the Flight Information section at
  the bottom of the web page
• Leave default radio button checked for Today in
  Flight Information section
• Click go button in Flight Information section
• Read the status field for the Atlanta?Jackson
  flight to see if it is on time

27
(No Transcript)