Journal and Pager Studies

1
Journal and Pager Studies

• Experience sampling techniques

2
Agenda

• Questions
• IRB Update
• Overview of sampling techniques
• Pager studies
• Diary studies

3
IRB proposals

• If asked who is going to sign off on the
  proposal, use the name Elizabeth Mynatt

4
Overview

• Can be formative or summative
• Depends on types of questions
• Format of study
• Uses of results
• Qualitative or quantitative methods
• Pager/beeper sampling
• Journal/diary studies

5
Goals

• Understand experience in natural settings
• Better understanding of experience over time
• Remove constraints of more structured methods

6
Types of beeper sampling methods

• Taken from psychology
• Thought sampling
• Experience sampling method (ESM)
• Descriptive experience sampling

7
Benefits of beeper sampling
8
Thought Sampling

• Uses random paging
• Series of questionnaires to determine mental
  content
• 24 4-point scales for features of thought
• (eg. Detail, color, auditoriness, interior
  monologue)

9
Experience Sampling Method

• More general than thought-sampling
• Does not necessarily measure thought variables
• Focuses on context of experience
• Where are you? With whom? What are you doing?
• Often uses Likert scales or multiple choice to
  assess experience

10
Descriptive Experience Sampling

• Qualitative (no Likert scales!)
• Encourages study participants to develop their
  own descriptive language
• Less constraint on user responses

11
Limitations

• Certain populations unwilling to carry pager
• How much time between page and response?
• Potential for post-hoc rationalization
• Potentially inconvenient
• Random or timed paging may miss important
  experiences

12
Pager study design factors

• How often to page
• How much time between page and response
• Random or regular paging
• Question content and format

13
Benefits

• Asks users to report what they were doing before
  the page
• Users less likely to modify behavior
• Can collect data at any time no observer needed
• Can help get at tacit knowledge

14
Diary/Journal studies

• Qualitative method
• Participant creates logs of activities
• Generally gives control of data collection to
  participant
• Also good for collecting data over time

15
Diary study design factors

• Structure of questions
• Record all activities or questions about specific
  activities?
• Prescribed time to log, or left up to
  participant?
• Once a day? Every time you use a system?
• Means of recording
• Writing, web entry, audio, photographs?

16
Benefits of diary studies

• Non-intrusive, non-interrupting
• Participants log at their convenience
• Potential for detailed, interpreted answers
• Not random
• participants will report most significant
  information

17
Limitations

• Relies on participants to remember to log
• Amount of information will vary with
  participants motivation
• Much time may elapse between experience and
  reporting
• Potential for post-hoc rationalization

18
Upcoming

• Monday Ubicomp
• Weiser article on web be prepared to discuss