Poster

1
The Effect of Miscommunication Rate on User
Response Preferences Hua Ai, University of
Pittsburgh Thomas Harris, Carolyn Penstein Rosé,
Carnegie Mellon University
Show Tell
2 Speech Modality
1 Motivation Smart Home

• Prompt users to switch to direct manipulation
  when continuous errors occur
• Direct manipulation serves as an interpretation
  of users speech input to train the speech
  recognizer
• Direct manipulation makes the user feel in
  control, thus increases user satisfaction rate

Voice-control Air conditioner
Smart Kitchen
Security System
Remote Water Monitoring
However, the undesirable error rate and the high
cost of error resolution degrade system
performance and frustrate users
However, How likely will the user follow the
system prompts to resort direct manipulation?
Personalized Room Modes
Garage Door Sensors
3 Wizard-of-Oz Experiments
4 Conclusions
System prompts Miscommunication
condition askRepeat Could you repeat
that? showMe Could you show me what do you
mean? Normal condition ok OK, Ill do that
for you. Done Im done!
Miscommunication Rate Low error rate (30) Mid
error rate (50) High error rate (70)

• While user does not always follow the systems
  advice, it does have an effect on their choice
  regardless of misrecognition rate
• Significant effect of system response to
  miscommunication on the likelihood of user direct
  manipulation (Plt.05) and Repeat (Plt.05)
• Overall, users were significantly more likely to
  continue with speech interaction when the system
  ask the user to repeat (p lt .01)
• While we would predict that users are more likely
  to resort to direct manipulation as
  miscommunication rate increased, our surprising
  finding was that users were most likely to resort
  to direct manipulation where communication
  success was least predictable. we found a
  significant effect of High versus Low Certainty
  on the likelihood of Direct Manipulation (p lt
  .005) and likelihood of continuing with speech (p
  lt .01)

Stereo
User actions Direct Manipulation Repeat Rephrase

• Experimental Design
• 3X2 within subjects Factorial Design
• explore the separate and joint effects of 3
  miscommunication rate and 2 system response to
  miscommunications (askRepeat and showMe) on user
  behavior
• Experimental setups
• 9 subjects, 4 females and 5 males, recruited on
  CMU campus.
• Each subject is required to complete 10 tasks
  with a stereo while staying alive in an eyes
  hands busy video game.
• Sample task turn the stereo on, play a CD, skip
  over a song, etc.
• Subjects are told that the stereo is an
  intelligent stereo that can accept commands
  either via direct manipulation or through speech
  commands. No vocabulary list is given.
• Subjects are randomly assigned to the 3
  miscommunication rate conditions, 3 people per
  group.
• A wizard plays the pre-recorded system prompts to
  interact with the subjects.
• The wizard decides the faked miscommunication
  points according to a pre-defined random
  sequence. Then either askRepeat or showMe prompts
  are given.
• The subject can response by Direct Manipulation
  or speech (Repeat or Rephrase).