Conversational Assistant for Personal Information Management

1
Conversational Assistant for Personal
Information Management
First year Qualifying Dissertation Seminar

• By Nuno Magalhães Ribeiro
• Hypermedia Multimedia Group
• Department of Computer Science
• The University of York
• nribeiro_at_cs.york.ac.uk
• Supervisor Ian Benest
• Assessor Patrick Olivier
• July 23, 1999

2
Problem

• Future computation engine with infinite storage
  space
• infinite processing speed,
• connected to the worlds electronic information
  via a network of infinite bandwidth
• the information stored will be very large (will
  anyone throw information away?),
• the user interface will need to find and cue the
  user appropriately so that the location of the
  required information is found without the need
  for hours of searching and viewing
• The amount of (personal) information is
  increasing at a fast pace
• difficult to remember information (what we know
  but cant recall)
• difficult to find existing information (what we
  need but cant find)
• time consuming for us to manage every piece of
  information we gather

3
Automated Personal Assistant
4
Research Objectives

• what kinds of reports will the conversational
  assistant provide using speech?
• what are the interrupting and the audio-message
  combination rules to provide information in a
  human-oriented way? Can these be modelled?
• can the context of the tasks be used to to
  synthesize high-level activities from low-level
  events?
• design and validate architecture of software
  agents in support of a conversational assistant
• develop and validate a structure for inter-agent
  communication of messages to be spoken (protocol)
• does a conversational assistant reduce users
  cognitive load and improve management of high
  volumes of information?

5
Speech rationale
6
Conversational Assistant
7
Specialised Agents
8
Conversational Agent
9
Project Work Packages

• Activity 2. Conversational Assistant Architecture
  and technological platform
• Activity 3. Specialised Agents Modelling
• Behaviour, time-dependent messages, kind of
  messages
• Activity 4. Communication Modelling.
• What messages will be exchanged, inter-agent
  protocol
• Activity 5. Conversational Agent Modelling
• rules for interrupting and combining audio
  messages,
• structure of reportage
• Activity 6. Prototype Implementation and Testing.
• Functional prototype, methods for testing and
  test
• Activity 7. Evaluation
• user population, usability study, analyse/assess
  conversational approach

10
Second Year Work Plan

• T1 to build an e-mail agent
• T2 to build a reminder agent (diary agent)
• T3 to build the conversational agent
• T4 to build a prototype including the reminder,
  e-mail and conversational agents, and a
  simulation of a presentation (reportage) in order
  to investigate the interruption rules
• T5 to establish the communication protocol

11
Programming Platforms Options

• Windows NT
• Java
• Microsoft Speech API speech recogniser/synthesis
  er
• Java Speech API IBM speech recogniser/synthesise
  r