A Context Communication Information Agent

1
A Context / Communication Information Agent

• Jason I. Hong
• James A. Landay
• University of California at Berkeley
• Group for User Interface Research (GUIR)
• CHI2000 Workshop
• Situated Interaction in Ubiquitous Computing
• April 03, 2000

2
Motivation

• Support Common Day-to-Day Information Activities
• Story
• some people in front of a whiteboard
• talking to each other
• writing some ideas on whiteboard
• Idea what if there are multiple sensors and
  capture technology?
• Idea what if computer used this info to help
  find related and relevant info?

Searching for relevant items

!!
3
Context / Communication Info Agent

• Proactive software agent
• context and h2h communication
• help find and deliver the right information at
  the right time.
• runs autonomously in the background
• constantly looking for relevant items
• present the most useful info as non-intrusively
  as possible

4
Design Space - Input

• Human-Human Communication
• Speech, Ink, Vision, Text
• Context
• Who you are?
• Who is speaking?
• Where am I?
• When is it?
• What calendar event am I in?
• What todo item am I doing?
• Interaction
• Person never interacts with system directly
• System just leverages what a person does naturally

Input
Search
Presentation
5
Design Space - Search

• Structuring queries from input
• Continuous streams of input
• Simple approach - spot keywords
• But, given richer sources of input, can we
  formulate better queries?
• Interesting patterns of communication?
• Which kinds of communication and context are
  useful for this task?
• Imperfect recognition is good enough
  recognition sufficient?
• Information sources to search
• Personal Information
• Clndr, Contacts, Email, Prsnl web, Notes
• Group Information
• Clndr, Group Notes, Contacts, Group web
• Global Information
• web

Input
Search
Presentation
6
Design Space - Presentation

• Synchronous / Asynchronous
• Show me as I do my work
• Show me after I get back from lunch
• Aim for minimal attention
• Attention should be on the task
• Tailor output to context
• I'm busy, don't bother me at all
• Show me only if it's really important

Input
Search
Presentation
7
Low-Fidelity Prototype

• Run a low-fidelity prototype
• Quick way of testing system that doesn't exist
  yet
• Parts requiring lots of programming simulated by
  human
• First iteration
• Speech-based agent, listens in on conversations
• Uses web search engines
• Presents combined search results
• Basically an alternative front-end for web search
  engines

Web search
8
Low-Fidelity Prototype

• Combined results
• Categorized by time
• Categorized by topic
• Material explicitly referenced
• Endeavour homepage
• "Do you have a link to that?"
• "There's a paper from Xerox PARC"
• Related material not explicitly mentioned
• UW Portolano
• MIT Oxygen Project

9
Low-Fidelity Prototype Results

• Some survey results
• People liked the general concept
• Search results rated ok, not great
• Didn't want to spend too much time (spent more
  time than we expected)
• Control of when agent was running (turn on and
  off)
• Real-time results
• Multiple ways of organizing, and accessing, and
  filtering the info

10
Prototype

• Implementation
• Speech-based input (IBM ViaVoice)
• Start with keyword-based search
• Uses Google search engine
• Problems
• Speech recognition is poor
• "Interesting" keywords not in dictionary
• Still in the process of improving the software

11
Continuing Work

• Better organization and navigation of results
• Peripheral displays
• Figuring out "right" rate to update results
• Another low-fi prototype using "precise" speech
  recognition
• Improving recognition rate
• ICSI Speech Recognition Engine / ViaVoice 2000
• Crawl local web pages to expand dictionary

12
Comments?
13
Design Space - Refined
Input
Context
Who, Where, When, How, What
Communication
Ink, Text, Speech
Presentation
14
Related Work

• Cyberguide and others
• Letizia
• Remembrance Agent
• MSR Implicit Queries
• XLibris