A Speech-Operated Voicemail System: Reactions from the Field

1
A Speech-Operated Voicemail System Reactions
from the Field

• Divya Ramachandran
• October 29, 2005

2
Technology for rural communication

• Intermittent connectivity makes asynchronous
  communication a more feasible solution
• Educational and language barriers could be
  transcended through speech interfaces
• Limiting functionality can reduce costs

3
Roadmap

• Description of ComBadge, a voice mail device from
  Mitsubishi Electric Research Labs
• Usability studies conducted in uncontrolled
  settings in Pudupalayam, Tamil Nadu, India
• Feedback and observations
• Consequent implications for future deployment and
  design

4
ComBadge from MERL

• Mobile two-way voice messaging device
• Operated entirely through a set of spoken
  commands
• Address book matches speaker dependent nametags
  to a MAC address

5
ComBadge Design

• Intel Xscale StrongARM 206 MHz processor
• 64 MB SDRAM, 64 MB Flash
• Integrated GSM/GPRS modem
• Compact Flash slot for 802.11b LAN
• Built-in microphone and speaker
• Runs Linux 2.4.19

6
ComBadge Features

• Send and receive messages
• Stores messages on device and forwards when
  within range
• Saves all old messages on server
• Phonebook management
• Map nametag to GSM/GPRS SIM ID address, 802.11b
  MAC address, Ethernet MAC address
• Utility function commands
• Check status of battery, network connection
• Configure network
• Check status of messages (unsent, old, new)

7
Speech Recognition

• Fully implemented interface in English using SDX
  recognizer from SpeechWorks
• No SDX trainer is available for creating new
  models
• 3 hours of recorded Tamil audio samples from
  TIER used as training data
• Used CMU Sphinx trainer to build Tamil acoustic
  models with near perfect accuracy running on 1.4
  GHz laptop
• Performance decreased to 16x real time on the
  ComBadge because of floating point operations
• Solutions exist, but were not feasible in time
  frame
• Used Wizard-of-Oz method of faking recognition
  for purposes of this study

8
Pudupalayam

• Primarily agricultural village in Erode district
  of Tamil Nadu
• Approximately 50 of residents are daily wage
  field workers
• Others include land owners, managers, brokers or
  micro-entrepreneurs and workers running
  businesses to meet local needs

9
Motivation

• Does this technology have benefits that cannot be
  matched by existing systems?
• Can this barebones instantiation of a voice mail
  system allow us to
• Get feedback about usability in an uncontrolled
  setting?
• Find new directions for further deployment and
  design strategies?

10
System Design

User 1
ComBadge 1
Ad-hoc Server (laptop)
User 2
ComBadge 2
Faked recognition
11
Task Diagram

New Message Play new
Play again
Play new Reply
Reply
Play new Reply
Play new Play again
Researcher
User
12
Recruitment of Subjects

• Word of mouth
• Friends of the host family
• Users would tell their friends and send more
  users
• Passersby
• Total of 64 users

13
Reactions Implications

• Popularity of cell phones
• Training via existing social networks
• Uncovering needs through scenarios of use
• Issues with speech as mode of input output

14
Popularity of Cell Phones

• 33/64 users tried to draw comparisons to cell
  phones, only 15/33 had actually used them
• Asked for features such as ring tones, birthday
  alarms, conference calls, sending pictures,
  screen, time display
• Only worth buying if its cheaper than a cell
  phone (50-60 of the cost)
• Advantages of ComBadge over cell phones
• No need to dial numbers
• Simplicity I dont use 90 of the features of
  cell phones
• Disadvantages
• No live conversation or receipt acknowledgement
• Sharing is difficult because of individualized
  nametags

15
Training via Existing Social Networks

• Illiterate or uneducated users always brought
  along someone more educated
• Women were relaxed when trained by other women
• Children showed willingness to teach uneducated
  parents
• Participants and observers helped each other use
  the device
• Relatively tech-savvy people provided useful
  information

16
Scenarios of Use

• Carpenter needs to coordinate instructions with
  workers
• Family wants to send messages to son who is
  abroad
• Milkman wants to inform customers that he will be
  late
• Residents want to invite friends outside the
  village to frequent religious functions

17
Speech as mode of output

• System status feedback prompts in Tamil were
  effective
• Users spoke in the same style as the prompt and
  believed that was the only way the ComBadge
  would understand.
• Constant ambient noise often caused users to miss
  the prompts completely

18
Speech as mode of input

• Users had difficulty remembering exact commands
• Dialect differences caused variation in
  naturalness of commands
• Users often added clarification phrases in
  response to a What did you say? prompt (i.e. I
  said, send a message.)
• Users had trouble understanding the difference
  between recognize record modes and consequently
  felt restricted when they recorded messages
• Of course, fast accurate recognition is a
  priority

19
Implications for Deployment

• Understanding the strong social ties among people
  can potentially lead to more effective ways of
  reaching out to the larger population
• By training educated family members, even school
  children, we could train the rest of the family
• Should make use of the clear willingness of
  trainers and trainees
• Longer term in-context deployment could test
  feasibility of projected scenarios of use
• Communication needs are all across the map, no
  real reason to focus on one application

20
Implications for Design

• Growing infrastructure market for cell phones
  makes them a potential platform
• Some minimal function cell phones exist
• Multipurpose computational platform
• Possible low-cost cell phone solution
• Large colored buttons for basic functions send
  message and listen to messages
• Some dialogue for choosing contacts key word
  search
• Speech-based feedback

21
Conclusion

• Uncovered existing practices that can be aided by
  an asynchronous communication technology
• The spread of cell phones promises advancements
  both in terms of communication and computational
  needs
• Important to focus on limiting functionality for
  reaching out to novice technology users