Towards a Humanoriented system for Personal Information Management

1
Towards a Human-oriented system forPersonal
Information Management

• DPhil Project Audit
• Nuno Jorge Gonçalves de Magalhães Ribeiro
• Supervisor Ian Benest
• Assessor Alistair Edwards
• Department of Computer Science
• University of York
• March 2001

2
Motivation (1/2)

• Speech employed in multimedia interfaces for PIM
  is potentially useful
• when information is visually hidden (e.g.
  information lower in hierarchy)
• Interactive Guide (interactive multimedia
  presentations)
• when information is visually disruptive (e.g.
  description of a diagram)
• Reportage (summarising reports including
  multimedia presentations)
• when various kinds of reminding and alerting
  information must be reported in an integrated and
  co-ordinated way w/semantics
• Monitor reports notifications (unpredictable
  events)
• reminders (event alarms with semantics)

3
Motivation (2/2)

• Open issues to investigate
• When does speech enhance the user interface and
  when is it detrimental to the user interface?
• Monitor Reports interrupt
• the users activity
• the audio channel (reportages, interactive guide)
• When to interrupt? How should interruptions be
  handled?
• Objective to minimise disruption on the user?

4
Social Interfaces (Nass and Reeves)

• People react socially to mediated interfaces in
  much the same way as they react in face-to-face
  situations
• If interfaces exhibit supportive modalities that
  cue
• social responses
• Then people perceive the interaction as more
  natural
• Humanised Interface

5
Aspects of a Humanised Interface

• Interruptions
• (urgency and priority)
• Politeness
• (appropriate level)
• Linguistic Variation
• (user presence and user activity)

6
Thesis Hypothesis
The incorporation of speech, conveyed using human
discourse characteristics, can improve a
computerised work environment, enabling the
computer to be perceived as a work companion.
7
Thesis Hypothesis (1/2)

• 1 Perception of a work companion
• use human discourse characteristics
• linguistic variation
• politeness
• The use of human discourse characteristics such
  as linguistic variation and politeness promotes
  the perception of a work companion.

8
Thesis Hypothesis (2/2)

• 2 Work Environment characteristics
• proper management of interruptions
• interrupt as a human would do
• dont avoid interrupting, but...
• minimise disruption
• interrupt the audio channel
• A proper management of interruptions occurring
  in the computer-based work environment promotes
  the perception of a work companion.

9
Demonstrator design and prototyping

• Design a Speaking Assistant architecture to allow
  for
• a number of specialised agents (e.g. e-mail,
  diary, printer)
• a speaking agent
• Speaking agent behaviour
• creates spoken monitor reports (reminders,
  notifications) corresponding to messages received
• creates reportages (structured multimedia
  reports)
• interrupts grabs user attention
• fades out on-going reportage
• presents a monitor report
• fades in on-going reportage

10
Towards a Personal Assistant in a Multimedia
Environment
Personal Assistant (Specialised Agents)
Speaking Agent infrastructure architecture
Reportage Interactive guide Multimedia clip ...
user activity monitor
context
diary agent
Dispatcher
Aggregator
Vocaliser
MM Engine
User
printer agent
user presence monitor
e-mail agent
Prioritised Messages
Notification Reminder Reportage
Sentences
Spoken messages
Scheduler Priority heuristics
Attention-grabbing Audio channel interruption
Template-based sentence generation
Intonation Prosody generation
11
Contributions (1/2)

• How to interrupt the user (activity and audio
  channel)
• grab the users attention to notify / remind
• fade out, speak, resume at appropriate locations
• must be handled after generating spoken
  messages
• When to interrupt
• notification mechanism with interruption levels
  that depend on urgency, priority and user
  activity
• must be provided by specialised agents

12
Contributions (2/2)

• What is an appropriate architecture to support
  a multi-agent personal assistant?
• a number of specialised agents
• handle specific parts of the user work
  environment
• send notifications/reminders to be spoken by the
  speaking agent
• sense the user presence
• a speaking agent
• speaks for a number of specialised agents
• dispatches, aggregates, vocalises and interrupts
• a suitable inter-agent protocol
• what information must specialised agents provide?
• Does this architecture promote the perception
  that a system is humanised?

13
Evaluation overview

• Assessment of the relative importance of
  incorporating some human discourse aspects
• at the interface
• Is the system useful? Will people continue to use
  it?
• Are the generated interruptions appropriate?
• Is the way used to interrupt appropriate?
• Does it appear to be a work companion?

14
Evaluation purpose

• What are the beneficial effects of interrupting
  using speech at the interface to convey
  information that is hidden from view?
• What is the role of speech as a contributor to
  the human-like nature of the interaction?

15
Evaluation empirical study hypotheses (1/3)

• H1 When the system applies an interruption model
  based on the urgencies and priorities of
  notifications, users will perceive the generated
  interruptions as useful (important) and
  appropriate (opportune).
• Independent variable mode of interruption
  interruption model based on urgencies and
  priorities / interrupt as events occur.
• Dependent variable users perception of the
  interruption useful / not useful and
  appropriate / not appropriate.
• H2 When the system provides spoken messages to
  convey notifications and reminders, users feel
  less disrupted in their current activity than in
  the case where messages are presented graphically
  in an independent window.

16
Evaluation empirical study hypotheses (2/3)

• H3 Users prefer a fade out/in approach when
  the audio-visual channels have to be interrupted
  over a full stop of the interrupted audio/video
  clips.
• H4 Users are less annoyed to be asked about
  their availability to receive notifications /
  reminders than to have the system interrupting
  without regarding their interest in them.
• H5 When the system conveys longer and more
  detailed messages (with more semantics), users
  understand better the purpose of the notification
  / reminder being conveyed through the spoken
  message than when spoken messages are shorter or
  non-existent (e.g. sound alarms, flashing icons).

17
Evaluation empirical study hypotheses (3/3)

• H6 When the system conveys polite messages,
  users acceptance is improved and the users
  level of irritation is lower than with neutral
  messages.
• H7 When the system uses linguistic variation,
  users acceptance is improved and the users
  level of irritation is lower than with repetitive
  messages.

18
Thesis Outline (1/4)

• Chapter 1 Introduction
• 1.1 Towards a human-oriented system for personal
  information management
• 1.2 Thesis overview
• Chapter 2 An infrastructure for human-oriented
  systems
• 2.1 Introduction
• 2.2 A system for personal information management
  issues and characteristics
• 2.3 The system as a personal assistant
• 2.4 The system as a conversational interface
• 2.5 Summary

19
Thesis Outline (2/4)

• Chapter 3 Aspects of a human-oriented system for
  personal information management
• 3.1 Introduction
• 3.2 Proper management of interruptions
• 3.3 Linguistic variation
• 3.4 Politeness
• 3.5 Summary
• Chapter 4 Design of a conversational assistant
  for personal information management
• 4.1 Introduction
• 4.2 System overview
• 4.3 A conversational assistant architecture
• 4.4 Speaking agent architecture
• 4.5 Summary

20
Thesis Outline (3/4)

• Chapter 5 The conversational assistant prototype
• 5.1 Introduction
• 5.2 Platform
• 5.3 Specialised agents implementation
• 5.4 Messages and the communication protocol
• 5.5 Speaking agent implementation
• 5.6 Using the prototype
• 5.7 Summary
• Chapter 6 Empirical Study
• 6.1 Introduction
• 6.2 Methodology
• 6.3 Results
• 6.4 Discussion
• 6.5 Summary

21
Thesis Outline (4/4)

• Chapter 7 Conclusion and future work
• 7.1 Conclusions
• 7.2 Contributions
• 7.3 Future work
• 7.4 Final remarks
• References
• Appendixes

22
Work Plan