Cognitive Walkthrough

1
Cognitive Walkthrough

• Discount formative evaluation technique for
  learnability

2
Agenda

• Questions
• Feedback on Part 3
• Questionnaire design
• Intro to discount usability
• Cognitive walkthrough

3
Evaluation techniques

• Predictive modeling
• Questionnaire
• Empirical user studies
• Heuristic evaluation
• Cognitive walkthrough
• Think aloud (protocol analysis)
• Field deployment user studies
• Interviews

4
Quick review of questionnaires

• Why are questionnaires difficult?

5
Questionnaires

• Objective must be well-defined and specific
• Question clarity
• Leading questions
• Phrasing
• Prestige bias

6
Discount usability techniques

• Less demanding on resources
• Inexpensive
• Requires few participants
• Can be done relatively quickly
• Does not require fully functional prototypes

7
Questionnaires

• Are they a discount usability technique?

8
Questionnaires

• Require many participants
• Analyzing data can be time consuming

9
Discount usability techniques

• Good candidates for project evaluation plan
• cognitive walkthrough
• heuristic evaluation
• think aloud

10
Cognitive Walkthrough basic facts

• A discount usability technique
• Uses a small number of HCI experts
• Focus is on learning through exploration
• Check for potential usability problems
• Derived from software engineering code
  walkthrough

11
Cognitive Walkthrough

• Requires fairly detailed description of prototype
• Requires a description of the user task to be
  analyzed
• Requires a complete, written list of actions
  necessary to complete the task
• Requires an indication of who the users are and
  their assumed knowledge

12
CW Procedure

• Define required inputs
• Walk through action sequences for task
• Record critical information
• Obtain believability story

13
Inputs (what the study designer does)

• Select interaction task
• Define interaction action sequence
• Identify users
• what knowledge experience
• Prototype
• Must know how interface looks for each step

14
Selecting the task

• Task should be a one that would be common or
  typical for a potential user
• Should be representative of what users would want
  to do with the system

15
Defining the interaction sequence

• Tasks should be broken down until any further
  division yields obvious subtasks
• Eg Type run at the prompt
• Not Type r at the prompt, Type u at the
  prompt

16
Identify users knowledge and experience

• Educate the HCI experts on the domain knowledge
  and characteristics of the user
• Give them a perspective from which to evaluate
  the system
• Base this information on what you have learned
  about your users!

17
Prototype

• Need not be functional but
• Must be at a level of detail where any action
  necessary to complete the task is defined

18
Doing the walkthrough(what the HCI expert does)

• Address each step of task sequence in turn
• Formulate a believability story
• answer 4 questions

19
Believability story

• 1. Will the user be trying to produce whatever
  effect the action has?
• 2. Will the user be able to notice that the
  correct action is available?
• 3. Once the user finds the correct action at the
  interface, will she know that it is the right one
  for the effect she is trying to produce?
• 4. After the action is taken, will the user
  understand the feedback given?

20
Believability Story

• Also includes justification for each answer based
  on the interface, knowledge of HCI, and
  understanding of users

21
Question 1

• Will the user be trying to produce whatever
  effect the action has?

22
Common supporting evidence

• It is part of their original task.
• They have experience using the system.
• The system tells them to do it.

23
No supporting evidence?

• Construct a failure story.
• Why would the user not be trying to do this?

24
Question 2

• Will the user be able to notice that the correct
  action is available?

25
Common supporting evidence

• Known through experience
• Visible device, such as a button
• Visible representation of an action, such as a
  menu entry)

26
Failure story

• Why would the user not notice that the action is
  available?

27
Question 3

• Once the user finds the correct action at the
  interface, will she know that it is the right one
  for the effect she is trying to produce?

28
Common supporting evidence

• Based on past experience with similar
  interactions
• The interface provides a prompt or label that
  connects the action to what she is trying to do.
• All other actions look wrong.
• If not, why not?

29
Question 4

• After the action is taken, will the user
  understand the feedback given?

30
Common supporting evidence

• Past experience with similar interactions
• Recognizing a connection between a system
  response and what she was trying to do.
• If not, why not?

31
Example

• Is Delta flight 367 from Atlanta to Jackson on
  time today?
• 1. using a web browser, go to the Delta website
  by typing www.delta.com in the URL field
• 2. type 367 in the search field labeled
  Flight number in the Flight Information
  section at the bottom of the web page
• 3. Leave default radio button checked for
  Today in Flight Information section
• 4. Click go button in Flight Information
  section
• 5. Read the status field for the
  Atlanta-gtJackson flight to see if it is on time

32
Midterm grades

• 110-118 7
• 100-109 4
• 90-99 7
• 80-89 7
• 70-79 5
• Under 70 1

33
If you got under 80

• I will post an optional extra credit homework
  assignment worth either 25 points or the
  difference between 80 and your score.
• More info on that to come.

34
Upcoming

• Monday Heuristic Evaluation
• DFAB 9.3