Hold that thought

1
Hold that thought!

• Applications of automated capture in everyday
  life
• Gregory D. Abowd
• College of Computing GVU Center
• Georgia Institute of Technology

2
Agenda

• Capture
• A general class of ubicomp applications
• 3 examples of capture
• Reflections

3
Hold that Thought!

• A Common Theme
• Many interesting services rely on automated
  capture of everyday activity, with useful
  interfaces to access and replay those memories.

4
Not a New Idea

• Vannevar Bushs 1945 vision
• the memex.
• Leverage computer as recorder
• Allow the human to
• engage in activity
• synthesize experience

5
Weisers motivation

• Sal doesn't remember Mary, but she does vaguely
  remember the meeting. She quickly starts a search
  for meetings in the past two weeks with more than
  6 people not previously in meetings with her, and
  finds the one.
• Sal looks out her windows at her neighborhood.
  Sunlight and a fence are visible through one, and
  through others she sees electronic trails that
  have been kept for her of neighbors coming and
  going during the early morning.

6
Three examples

• Think about how these are different.
• Think about how to do formative and summative
  evaluation.

7
Example 1Conversations with Meghan

• What were we talking about?
• Wouldnt it be nice to have access to near-term
  audio around you at all times?

8
The Personal Audio Loop (PAL)

• The idea

9
The Design Challenge

• What does it take to make this service
• Useful
• Ubiquitous
• Usable
• Acceptable
• Discussion
• Which evaluation techniques would apply?

10
Useful/Ubiquitous

• Portable diary study
• How often?
• When?
• Where?
• Who around?
• How long ago?

11
Usable

• Empirical study How to create a controlled but
  somewhat realistic audio skimming experience?
• Discount Heuristic eval or cognitive walkthrough
  or think aloud

12
Script

• User Hello, I would like to ask you a few
  questions about yourself so I can get to know you
  better. I ask that each of you answer each
  question one at a time. First question is what
  are your full names?
• Julies Answer My full name is Julie Anne
  Kientz.
• User Thanks. The next question is when and
  where were each of you born?
• Julies Answer I was born on at 840 a.m. on
  May 9th, 1980 in Marion, Ohio.
• User Can you tell me a little bit about your
  family? Do you have any siblings?
• Julies Answer I come from a family of two
  children, including myself and my older brother,
  Jeremy, who is 26. My parents have been married
  for over 35 years and still live in Marion, Ohio.
  
• User And what do your parents do for a living?
• (for 15 minutes)

13
Our answers

• What does it take to make this service
• Useful 10-60 minute buffer
• Ubiquitous put it on a cell phone
• Usable simple skimming with landmarks
• Acceptable legal consent rules, near-field
  microphone and NO archiving

14
Example 2RGA Action Films

• Sucking your fingersdoes not adverselyimpact
  potential.

Mom Dad, 1982 Niagara Falls
Peter ND EE 88
15
The Design Challenge

• Create a system to annotate home movies with
  relevant metadata to facilitate browsing and
  searching.
• Effective symbiosis between manual and automated
  techniques.
• Control/accuracy trade-off with speed
• (Multimedia Information Retrieval 2003)

16
The Family Video Archive
17
The Family Video Archive
18
Simplified Sharing

• Albums reflect content with common theme.
• Automatically author a DVD using metadata to
  support navigation.
• Audio commentary as additional bonus track
  (future).

19
Point of Capture Video Annotation

• The ContextCam (Ubicomp 2004)
• When, where easy
• RF-Ultrasound tagging to get who in and around
  field of view
• Event predicted from past annotations
• Encode metadata in video
• Clumsy prototype now
• Ultimate platform
• cell phone

20
Reflections on capture

• What are some of the interesting general design
  problems for capture?
• Timescales
• Access drives capture
• Social and legal issues

21
Timescales

• Short-term
• Within minutes
• PAL, Abaris
• Medium-term
• Within days/weeks
• Classroom lectures, Abaris, CareLog, Finding Lost
  Objects
• Long-term
• FVA, Abaris, CareLog

22
Access Drives Capture

• Know why you want something recorded beforehand.
• What research methods work?
• PAL diary studies
• FVA prototype motivates point of capture
  annotation
• Autism stakeholder interviews and artifact
  exploration reveals desire for data concealed by
  educational priorities.

23
Social and Legal Issues

• Cameras and microphones worry people.
• Dont shy away from the challenge!
• Understand the market, legal, social and
  technology forces at work and make informed
  proportionality judgments.

24
Example 3My Darling Children
25
Problem

• The science behind intervention therapies for
  children with special needs.
• It is hard to track the impact of therapy on
  these children.

26
Example ABA DT therapy

• A structured, discrete trial approach to building
  life skills.
• A popular, data-intensive therapy for autism.

27
Artifacts of ABA

• LOTS of data
• Time-intensive ( ) to compile
• Useful, but incomplete

28
The Design Challenge

• Support the recording of quantitative and
  qualitative data by a network of caregivers and
  educators.
• Flexible and unobtrusive mechanisms for
  collection leading to trend analysis. (Ubicomp
  2004)

29
Abaris The golden arrow?