Context-Aware Systems in the Wild

1
Context-Aware Systems in the Wild

• Dr. Anind K. Dey
• Assistant ProfessorHuman-Computer
  InteractionCarnegie Melon University

2
Outline

• Background
• Ubicomp and context-awareness
• Context-aware infrastructure
• Context-aware applications
• New area Usability in context-awareness
• Change in approach
• Change in infrastructure support and interface

3
Overview of Ubicomp

• Vision articulated by Weiser

4
Overview of Ubicomp

• Computing embedded into everyday objects and
  environments, enhancing everyday activity

5
Context-Aware Computing

• Context situational elements relevant to
  interaction between user, application,
  environment
• Context-awareness situationally appropriate
  apps adapting to context, increasing value to
  users
• Using sensors and actuators to improve
  human-computer interaction and (computer-mediated)
  human-human and human-environment interaction
• Examples tour guide, reminders, diary retrieval

6
Historical perspective

• Weisers 1991 visionary Scientific American
  article Computer for the 21st century
• Sals alarm clock and coffeemaker
• Windows Electronic trails of neighbors, kids,
  weather
• Rear view mirror coffee shop
• Search for details in meeting information
• Amazing amounts of context
• But how do you build?

7
10 years of context-aware computing research

• Driven by a single problem
• Reminder to buy milk
• When to deliver not time/location specific
• How to deliver appropriate modality

8
Context Toolkit

• 1997 difficult to build simple context-aware
  applications
• No abstractions for acquiring and using context
  from sensors or controlling actuators
• Context coming from a number of distributed
  sources
• No principles for designing applications

9
Design process

1. Specification
2. Acquisition
3. Delivery
4. Reception
5. Action

1. Specification
2. Acquisition
3. Action

10
Thesis Context Toolkit

• Context Toolkit to support programmers in
  building context-aware applications more easily

Application
Application
Aggregator
Interpreter
Interpreter
Widget
Widget
Discoverer
Context Architecture
Sensor
Sensor
11
Some Success

• First of a series of toolkits that actually makes
  it easier to build context-aware applications
• Toolkit in use by over 600 academic and corporate
  institutions
• http//contexttoolikit.sourceforge.net
• Lots of applications created
• CHI 99, ISWC 99, UBICOMP 00, UBICOMP 01, UIST 02,
  CHI 03, HCI Journal 01, ToCHI 04

12
Outline

• Motivation and background
• Ubicomp and context-awareness
• Context-aware infrastructure
• Context-aware applications
• Usability in context-awareness

13
Context-Aware Infrastructures

• CTK privacy, uncertainty, end-user support
• Nexus context modeling, representation, and
  spatiotemporal processing
• JCAF quality of context and context modeling
• Context Fabric privacy, and modeling of context
  entities
• PersonisAD context models of entities and links
  between them, support for scrutability
• Context Spaces context modeling, inferencing of
  situations and probabilistic reasoning

14
Outline

• Motivation and background
• Ubicomp and context-awareness
• Context-aware infrastructure
• Context-aware applications
• Usability in context-awareness

15
Context-Aware Applications

• What is a context-aware application?
• App that uses context to perform some
  behavior/service for its user(s)
• 3 types of behaviors
• Display context
• Automatically execute/adapt services
• Tag captured information for easier retrieval

16
Directly displaying context
17
Execute/Adapt Services

• Select and perform a relevant service from
  multiple services
• Change how a service behaves or is executed

18
Tag Information

• Use context to aid in context-based retrieval
• May be easier to remember context than specific
  information

19
So are We Done?

• Can support building and executing of
  context-aware apps
• Are building interesting apps
• But, we still dont have widespread deployment
• Commercially available apps are really quite
  simple

20
Back to the Milk Reminder

• Just making it easier to build applications is
  only part of the problem
• Once you know what context to capture and how to
  use it, easier to execute
• But how do you figure this out?
• Lots of problems to be addressed on the user side

21
Outline

• Motivation and background
• Ubicomp and context-awareness
• Context-aware infrastructure
• Context-aware applications
• Usability in context-awareness

22
What are the Impediments?

• Cost and difficulty in deploying infrastructure
• Applications are of limited value
• Not modeling the right tasks
• Applications lacking in usability
• Apps lacking in support for end-user control
• Apps lacking in intelligibility

23
Control in Ubicomp

• Performed two studies on how people respond to
  context-aware applications
• Each study 20 people, diary, interviews
• Presented variety of applications, different
  levels of interactivity and elicited responses
• Serious issues with lack of control and privacy
• Despite these, wanted to use these applications
• UBICOMP 03, INTERACT 03

24
Usability is Key

• Dourish, Abowd and Mynatt, Bellotti and others
  lack of control in these environments
• Information collected, synthesized and used
  implicitly
• How do I know whats going on? (intellgibility)
• How do I change whats going on? (control)
• Who gets this information? (privacy)
• Is this another way to SPAM me? (overload)

Rich Gold Dancing Toaster
25
Research Focus Usability
26
Research Focus Feeling in Control

• End-User Intelligibility and Control decide what
  actions an environment should take and understand
  environment state
• CHI 04, Pervasive 06, Ubicomp 06, CHI 07, Ubicomp
  07

27
Research Focus Usability

• Privacy decide what information to release when
• CHI 03, Personal and Ubiquitous Computing Journal
  04

28
Research Focus Usability

• Peripheral Displays present information without
  overwhelming
• CHI 03, CHI 04, Pervasive 04, UIST 04, UIST 05,
  CHI 06

29
User-Centered Approach

• Improved modeling is required real situations,
  real users, real problems --gt real benefits
• Ethnography
• Need finding
• Sense user behavior of interest
• Create models of user behavior
• Present information or execute services based on
  models
• Study impact
• Domain impact
• Impact on infrastructure design
• Mental models
• Interactive techniques, tools for control,
  feedback, modeling, evaluation

30
Challenges

• Leveraging real human context and realizing that
  youre not really getting at human intention
• Star Trek doors
• Anyone remember the original Star Trek series?
• Japanese doors
• Wash doors?

31
Challenges

• Leveraging real human context and realizing that
  youre not really getting at human intention
• Clippy

32
End User Control and Intelligibility

• Control real situations are dynamic and cannot
  always be planned for a priori
• Need to allow constant adaptation
• Home new people, new organization, new roles
• Programmer cant possibly hope to cover these
  situations
• Must support end-users in maintaining control
• Need support for this

33
Control of Smart Homes
34
End User Control and Intelligibility

• Intelligibility users will reject systems they
  do not understand, particularly when these
  systems make mistakes
• Amazon recommender, Clippy
• Complex systems need to describe their behavior
  to some degree

35
Study of Mental Models

• One group received up to 3 features that best
  explain current prediction
• CHI 07

36
Context in Everyday Life

• Selfish research agenda
• Dual income family with children
• Control, intelligibility, privacy, overload
• Getting lost Improving in-car navigation systems
• Control, intelligibility, overload
• No exercise Motivating physical activity through
  reflection and intervention
• Control, intelligibility
• No sleep Context-based memory aids for
  Alzheimers patients
• Intelligibility, privacy, overload

37
Working up to 850 Million People!

• Not-so selfish research agenda
• Dual income family with children
• 50 USA 150 million
• Getting lost Improving in-car navigation systems
• 75 USA 225 million
• No exercise Motivating physical activity through
  reflection and intervention
• 2/3rds USA 200 million
• No sleep Context-based memory aids for
  Alzheimers patients
• 12 million with Alzheimers

38
Research Problem

• Disconnect what should smart homes do and how

39
Research Problem

• Support what families value time, activities,
  relationships

40
Focus on Dual-income Families

• Largest segment of US population and growing
• Live logistically complex lives that drive
  aggressive and experimental use of communication
  technology

41
Why Families Feel Out of Control

• Swamped with responsibilities from kids
  activities and jobs

42
Findings Master Busyness

• Parents want to master managing numerous
  activities

43
Challenge Wicked Problem of Activity Management
44
Soccer Example
GAME OR PRACTICE? HOME OR AWAY? WHAT TIME?
45
Soccer Example
WHO PICKS UP? WHO DROPS OFF? WHERE?
46
Soccer Example
SHIN GUARDS, KNEE PADS. CLEATS OR FLATS?
47
Soccer Example
PRACTICE UNIFORM? HOME OR AWAY UNIFORM?
48
Soccer Example
CLEAN CLOTHES THE NIGHT BEFORE
49
Soccer Example
JUICE BEFORE OR AFTER? ORANGES AT HALFTIME?
50
Breakdowns
LOST ON THE CALENDAR
51
Breakdowns
LAST MINUTE CARPOOL DECISIONS
52
Breakdowns
MOMS OUT OF TOWN SO DADS IN CHARGE
53
Breakdowns
MOMS OUT OF TOWN SO DADS IN CHARGE
54
Breakdowns
MOMS OUT OF TOWN SO DADS IN CHARGE
55
Breakdowns
CREEPING RESPONSIBILITY
56
Breakdowns
CREEPING RESPONSIBILITY
57
Breakdowns
UNPREDICTABLE ORANGES
58
Breakdowns
SICK CHILD
59
Breakdowns
CASCADE EFFECTS
60
Mapprentice Route Planning

• Current routing systems are impoverished
• Dont take into account time/day, weather,
  traffic
• Dont take knowledge of the user into account
• Can build much smarter systems
• Model driving behavior, use contextual cues, and
  produce better driving directions
• Predict destination and route, suggest alternate
  routes to take care of errands, or to improve
  path
• Diagnose capabilities of elder drivers and
  produce more appropriate routes

In-car Support
Web Support
GPS-enabled cars
61
IMPACT Improving and Motivating Physical
Activity using ContexT

• 60 of Americans overweight/obese
• Leading cause is lack of physical activity
• Ethnography reveals subjects want expert advice,
  have difficulty with motivation and understanding
  data from exercise devices
• Our solution real-time feedback and
  point-of-decision prompts via mobile phone,
  expert agents to guide reflection and provide
  motivation, context
• Motivate lifestyle change to be more physically
  active

Great job!!
GPS-enabled phone
Activity monitor
Visualization
62
Context-Based Memory Aids for Peoplewith Mild
Cognitive ImpairmentsDey

• 12 million Americans have Alzheimers/MCI
• memory loss, financial impact and caregiver
  burden
• Study of patient/caregiver dyads to understand
  types of memory impairment, impacts of
  impairment, and role of technology
• Our solution rich capture of everyday
  experiences, integrate with contextual cues for
  enhanced playback and review, and integrate use
  into daily life
• More accurate/complete account of life, reduced
  burden, increased care, and greater independence

rich multimedia experience
unobtrusive sensors
engaging memory exercise
63
Summary

• Despite all the work in making context-aware
  applications easier to build, theres lots to be
  done
• Context is only an abstraction for user
  motivation/intent
• Context is a part of everyday life and activities
• A number of difficult usability and technical
  issues that need to be addressed
• Model user needs and practices
• Support end-user control
• Support intelligibility of end-user systems
• Still no solution to the milk reminder problem

64
Questions?

• anind_at_cs.cmu.edu
• www.cs.cmu.edu/anind