The Logbook Probe: Facilitating auto-thick description for evaluation

1
The Logbook Probe Facilitating auto-thick
description for evaluation

• Workshop on Innovative Approaches for Evaluating
  Affective Systems
• Swedish Institute for Computer Science, Kista,
  Sweden
• 9 January 2006
• Joseph Jofish Kaye
• Culturally Embedded Computing
• Cornell Information Science
• jofish_at_cornell.edu

2
Unpacking an overpacked title

• The Logbook Probe Facilitating auto-thick
  description for evaluation

3
Starting points

• Gaver et. al Cultural Probes
• Geertz Thick description
• Dourish, Wright McCarthy, etc experience-based
  HCI
• Iterative design Design, Build, Evaluate..

4
Thick Description

• Winking isnt just something your eye does
• Its culturally embedded
• Its only the thick description of the context
  and culture that lets us understand the role of
  the wink in sharing a conspiracy, or even
  parodying another sharing conspiracy.
• Geertz, C. (1973) Interpretation of Cultures Ch.
  1

5
Cultural Probes

• Gives context around a situation
• Originally for inspiring the design part of the
  design ? build ? evaluate ? iterative design
  cycle
• Repurposed here for inspiring the evaluation part
  of the cycle
• Gaver et. al. Cultural Probes, interactions 6(1)
  1999

6
Experience-based HCI

• Excellent theoretical foundations
• Dourish (2001) Where the Action Is
• McCarthy Wright (2004) Technology as Experience
• etc.
• Significant work on designs in CHI, DIS, etc
• Lack of work on evaluation for this domain as
  compared to traditional HCI (GOMS!)

7
Object for evaluation VIO

• VIO (Virtual Intimate Object)
• Software device represented as a small red circle
  in the taskbar of Windows screen
• When circle is clicked, partners circle glows
  bright red, then fades over time.
• http//io.infosci.cornell.edu

• Kaye, Levitt, Nevins, Golden Schmidt.
  Communicating Intimacy One Bit at a Time.
  Proceedings of CHI 2005, ACM Press.

8
Two studies to date

• I 10 couples, 5 using VIO, 5 using hardware
  version. 1 week, paper logbook probes.
• II 80 initially, pre- and post- surveys, two
  weeks of Internet based logbook probes, four
  groups
• VIO
• VIO daily survey
• daily survey
• only pre- and post surveys
• Currently being analyzed.

9
Principles

• To get the subjects to give us a rich description
  of their experience
• To encourage reflection by the subjects (and
  researchers) to get more insight into the
  situation
• To defamiliarize subjects (and researchers) with
  their assumptions to get rich, novel descriptions

10
Three topics

• Questions/tasks about the technology being
  evaluated.
• Questions/tasks about the situation.
• Questions/tasks about instrument being used to
  gather information

11
And three kinds of questions(for organization,
not canonical)

• Context Where? When? Take a photo of How
  long? How often?
• Metaphor What TV show? What song? What colour?
  What metaphor?
• Valence Whats the best? Whats the worse?
  Whats your favorite? What should we change?
  What shouldnt we change? Draw the best, the
  worst.

12
Design chart
  Context Metaphor Valence
Technology (VIO)      
Situation (Long distance relationship)      
Instrument (Probe / Survey)      
13
Example questions

• What did you hate the most about using the VIO?
• Draw a picture of your ideal intimate object
• What colour is your relationship? Why?
• What TV show most represents your relationship?
• What song most represents your relationship?
• What metaphor would you use to explain VIO to
  people?

14
A final note getting beyond usability

• Evaluating in context requires working in context
• VIO becomes ready-at-hand, not present-at-hand,
  and so evaluation must be in context
• which makes demonstrating this, in this
  context, at this workshop, hard!
• Must have significant, continuous use, otherwise
  most of what you get is usability issues see
  Mobile Probes paper
• Hulkko et. al. Mobile Probes Proc. NordCHI 04

15
Thank you

• VIO software http//io.infosci.cornell.edu
• Many thanks to Kia Hook, Jarmo Laaksolahti,
  Kirsten Boehner, Petra Sundstrom, Phoebe Sengers,
  Bill Gaver and the Culturally Embedded Computing
  Group
• Dots for use on Swedish names ..