Think Aloud Evaluation

1
Think Aloud Evaluation

• and other observational techniques

2
Agenda

• Questions
• Observational techniques
• Think aloud
• Cooperative evaluation

3
Observational techniques

• Watching users as they perform
• Summative or formative
• depends on the purpose of exercise
• Qualitative or quantitative
• depends on recording and analysis

4
Recording

• Paper and pencil
• Audio/video
• Computer logging
• User journals

5
Pros and Cons of recording

• Richness of record
• Time to transcribe analyze

6
Support for recording

• Effective coding schemes
• Automated tools

7
Post-task walkthroughs

• Discussion with subject after observation
• Added richness and interpretations
• Warning post hoc interpretation

8
Specific techniques

• Think aloud
• encourage verbalization during interaction
• Cooperative evaluation
• designer/user discussion

9
Objective

• Formative
• need operational prototype
• Qualitative
• process or how-to information
• compare users (verbalized) mental model to
  designers intended model

10
Example Think aloud

• Lets find out about the world series
• From www.nytimes.com
• Find the start time for tonights game

11
Example What was learned

• What was the users model?
• What was the designers intended model?
• Any discrepancies?

12
Cooperative evaluation
13
Upcoming

• Fri Empirical Evaluation
• Mon Special Topics
• Wed GVU Demo Showcase
• Fri Part 3 Poster Session