Instant Messaging and Interruption: Influence of Task Type on Performance

1
Instant Messaging and Interruption Influence of
Task Type on Performance

• Mary Czerwinski
• Ed Cutrell
• Eric Horvitz
• Microsoft Research

2
What Is Attention?

• There are many definitions of attention
• A function which selectively improves processing
  for one item, location, or task at the expense of
  others
• Different modalities

3
Some Motivation

• Miyata Norman (1986)
• Predicted interruptions after important actions
  or between task execution and evaluation would be
  less harmful when multitasking

4
Background 1

• Notifications are most distracting when they bear
  surface resemblance to the UI of the task at hand
• Gillie Broadbent (1989) Kreifelt McCarthy
  (1981) Rhodes, Benoit Payne (2000)
• Auditory notifications can be more distracting
  than visual notifications
• Mollenhauer, et al. (1994)

5
Background 2

• People habituate to notifications over time, and
  training can help
• Hess Detweiler (1994) Altmann Gray (2000)
• Interruptions can be useful!
• OConaill Frohlich (1995)

6
McFarlane (1999)

• Examined 4 methods for instant messages
• immediate (requiring immediate user response)
• negotiated (user chooses when to attend)
• mediated (an intelligent agent might determine
  when best to interrupt)
• scheduled (interruptions come at prearranged time
  intervals) interruption methods
• Negotiated resulted in good performance
• users may postpone attending to interrupting
  messages in these cases
• Immediate is fast but users are less efficient
  overall

7
Previous Work 1

• Degree of disruption depends on what task a user
  is doing and when the notification arrives
• Relevant notifications are less disruptive than
  irrelevant
• Czerwinski, Cutrell Horvitz (2000)

8
Previous Work 2

• Notifs during Web search task phases
• Planningdeciding what search terms to use
• Executionentering the search terms
• Evaluationsearch the list looking for target
• Disruption worst during
• Execution (chunking)
• Evaluation (? ? ?)

9
(No Transcript)
10
Day Trading StudyTask Type

• The time to respond to the notification depended
  on the ongoing task.
• Word and Excel task users more reluctant to
  disengage.

11
Day Trading StudyInterrupt Position

• Time to respond to the notification was least
  when it arrived at the beginning of the task
• Easiest phase at which to disengage in this study

12
Day Trading StudyTask Type

• The time to respond to the notification depended
  on the ongoing task.
• Word and Excel task users more reluctant to
  disengage.

13
Messaging on the Web

• Some trials interrupted with MSNs Messenger
  service (v. 2.0)
• Relevant messages (design category)
• Irrelevant (factoid about target site)
• Interruptions occurred during one of the three
  phases

14
(No Transcript)
15
(No Transcript)
16
Web StudyTime to Switch

• Time to switch to the notification is slowest
  when the execution phase is messaged (while a
  user is typing). This replicates what we saw in
  Day Trading study.
• Execution phase slowest to disengage.

17
Evaluation Stage Most Harmed

• The evaluation stage was reliably more disrupted
  than otherswhen notification came in the
  evaluation stage
• Not slower to disengage, so slower to process
  message and re-instate task context

18
Web StudyEffect of Relevance
When notifications are irrelevant, total time on
task and time to resume task after interruption
are both slower
19
Why is List Scanning So Susceptible to
Interruption?

• 2 hypotheses
• 1) Visual reorienting is hard to do People
  just lose their place in the list Where was
  I?
• 2) Problem with conceptual reacquisition
  Delay from accessing memory of goal What was
  I doing?

20
How do we find out?

• Sent IM messages to participants while they were
  scanning lists of book titles
• Two kinds of targets
• Verbatim title (easy visual scan)
• Gist of title (difficult semantic-based search)
• Can we help?
• Used a visual marker to save place in list

21
Method

• 12 experienced Microsoft Office 2000 users, aged
  25-54, participated in this study. 6 had some
  experience with MSNs Messenger.
• 64 sets of 80 book titles obtained from MS
  library. Each set was an Excel spreadsheet.
  Targets were distinctive within a set of 80
  titles.

22
List Search

• Target always visible at top
• Navigation with Cursor Up/Down or Page Up/Down
  keys
• Cursorgtmarker
• Search target either verbatim title or gist of
  title

23
Procedure

• In half of all trials, participants search task
  was interrupted with an instant message asking
  them a simple math problem.
• Half of all trials had gist targets and half
  had title targets
• Navigation was blocked, with half of the
  participants using Cursor Up/Down (Marked) first
  and the other half using Page Up/Down (Unmarked).

24
Results

• Only report time data accuracy was quite good
• Used log response times to normalize common
  skewing variability of RT data

25
Overall task times

• IMs slow down task times
• Searches for Gists are slower than for Titles
• No difference in navigation

26
Task Times Minus IM Time

• Same pattern as overall task timeseffects not
  due to device switching time
• Marker only helped title search a
  littlenavigational confound

27
Summary

• IM is disruptive
• More disruptive for fast, stimulus-driven search
  than for slower, semantic-based search tasks
• Marker didnt seem to help, but was confounded by
  navigation style
• Reran study w/o confound, same result

28
Memory Effects Notification

• Reran book title search study correcting for
  navigation confounds
• Removed title from top of page and added a
  Remind Me button to list
• Users could use button any time
• Recorded where and when users needed to be
  reminded of search target

29
Methods

• 16 Ss (9 female)
• Intermediate to advanced PC users
• All but 1 had tried IM before
• 2 (marker or not) x 2 (IM or not) within subjects
  design
• 64 search trials

30
Results
31
Attention-Based Principles of Notifications 1

• Unless you are absolutely sure the user wants to
  know what youre telling them at that moment, be
  careful of very salient notifications (this is
  from previous work)
• Autoarchive in Outlook
• Frequent audio alerts from Messenger
• Users trust is fragile. Once they perceive a
  system is unreliable, it is very hard to win them
  back (from ongoing work)
• Be cautious repeating information it might be
  outdated or irritating

32
Attention-Based Principles of Notifications 2

• Make notifications situation-aware.
• Look for breakpoints and pauses in users
  interactions. Weve identified a few Open or
  Save as dialog boxes probably good places to
  interrupt typing, selecting, and other direct
  interactions probably bad
• When possible, use smart monitoring.
• Monitor the user (what are they doing?)
• Content of interruption
• Obvious privacy issues, etc.

33
Adaptive Systems and Interaction Group at MSR

• Complementary work on modeling and decision
  making for alerts going on in our group
• An information-theoretic perspective with
  supportive infrastructure.
• Work tends to rely on theories that consider
  direct preference assessments about outcomes.
• User studies will hopefully minimize the needs
  for preference elicitation
• Results of the work will be useful to the
  cost-benefit modeling, decision making work.

34
Future Work

• User models of distractibility
• Better cost/benefit user models of the value of
  delaying information
• Better UI for notifying and reminding user of
  what they were doing before the notification
• Field studies with teens
• Longitudinal studies of our beta Mobile Manager
  software w/cell phones