Workshop%20on%20Ubiquitous%20Computing%20Trip%20Report

1
Workshop on Ubiquitous ComputingTrip Report
Jason I. Hong Sept 28, 2001
2
Dagstuhl

• Week-long seminars on CompSci topics since 1990

3
Dagstuhl
4
Ubicomp Wkshp

• Explore core issues in ubiquitous computing
• Develop a research roadmap
• Foster a community
• Some of the topics
• Lessons Learned
• Privacy
• Definitions of ubicomp
• Evaluating ubicomp
• Context-awareness

5
BackgroundOriginal Ubicomp Apps

• Badges emit infrared signals
• Gives rough location ID
• Teleport
• Redirect screen output from "home" computer to
  nearby computer
• Phone forwarding
• Automatically forward phone calls to nearest phone

Active Badge Olivetti / ATT Hopper, Harter, et
al
6
BackgroundOriginal Ubicomp Apps

• Active badge wireless
• Rough location ID
• Proximate selection
• Interfaces for nearby objects
• Auto-diaries
• People, places, and time
• Triggers
• Alerts on preset events
• Reconfiguration
• Bind device to room

ParcTabs Xerox PARC Want, Schilit, et al
7
Roy Want's 10 Lessons Learned

1. People really care about how these technologies
   are used, be prepared for many long discussions
2. The press loves stories about ubicomp
3. Beware the press...
4. Only building something actually allows you to
   explore its full design potential.
5. Successful technology adoption is very dependent
   on the culture of the target users.

8
Roy Want's 10 Lessons Learned

1. Hard to adopt without a new level of utility
2. Build high-quality, customizable products
3. Only really get one chance to impress users with
   technology, second chance is rare.
4. Infrastructures are hard, lots of work to deploy
   and maintain.
5. Listen to user experiences, but filter desires

9
Privacy in Ubiquitous ComputingMarc
Langheinrich, ETH

• Design of technology has significant impact on
  what can and cannot be enforced (Lessig)
• Ubicomp relatively "boring" compared to AI,
  crypto, nanotech, and other media grabbers
• But coming in fast under the radar!
• US Privacy Act, 1974 established Fair Information
  Principles
• 1995 European Union Directive on privacy
• US / EU Safe Harbor program

10
Fair Information PracticesEU Directive / Safe
Harbor

• Openness and Transparency
• No secret record keeping
• Individual Participation
• See and correct your records
• Collection Limitation
• Data collected only for stated purposes
• Data Quality
• Data is relevant and up-to-date

11
Fair Information Practices (cont.)EU Directive /
Safe Harbor

• Use Limitation
• Data used only for stated purposes, by authorized
  personnel
• Reasonable Security
• Proportional to sensitivity of data
• Accountability
• Compliance with fair information practices
• Explicit Consent
• People have to agree

12
Privacy in Ubiquitous ComputingMarc
Langheinrich, ETH

• Notice
• Announce what's going on
• Difficult because of scale, possibly filter via
  P3P
• Choice and Consent
• Difficult because of scale
• Anonymous / Pseudonymous
• Temporary, unlinked identification of people
• Proximity and Locality
• Operate only when owner is nearby
• Access and Recourse
• Trust but verify system is doing what it says

13
Defining Ubiquitous Computing

• In relation to more mature disciplines
• In relation to courses taught in universities

14
Evaluating Ubiquitous Computing

• What's different from traditional evaluation?
• Both from a systems and HCI perspective
• Evaluating many layers simultaneously
• Still many technological constraints
• Battery life, Network bandwidth
• Scale of evaluation
• Space
• Time
• People

15
Evaluating Ubiquitous Computing

• Evolvability / Interoperability
• Set of services, sensors, and devices changes
  over time, not a fixed set that you can design
  for
• Failure Recovery
• What happens if room crashes?
• Introspection and Sensemaking
• What just happened? Why did it happen?
• What can I do here?

16
Dagstuhl
17
Some Issues in Context-Awareness

• Push and Pull
• "Where is Jimmy?" vs "Notify me when Jimmy enters
  the office"
• Ambiguity, validity, probability
• History of context vs garbage collection
• Context hierarchies
• People gt Groups, Rooms gt Building
• Operators
• Filters, interpreters, aggregators, fusion

18
Some Issues in Context-Awareness

• Debugging and Simulation
• Fake collected sensor data
• Tracing sensor data, from source to transforms to
  sink
• User interfaces for context info
• Feedback, feedforward
• Discovery / Replacement
• Where is my data? What if I can't find it? Infer
  from sensors?
• Security / Privacy / Trust
• Who can access the information?
• How am I seen by others?
• Performance

19
Context-Awareness and AI

• Is Context just Knowledge Representation?
• World modeling, more dynamic
• Success in narrow vs broad domains
• Low-level vs high-level
• In terms of sensors, inferencing, tasks
• Anti-lock brakes vs "Avoid accident"
• Layers in Speech recognition
• Phone gt Syllable gt Word gt Sentence
• Corresponding layers for context?

20
(No Transcript)
21
Some Other Highlights

• Personal Server, Intel
• Disappearing Computer Initiative, EU
• Physical disappearance vs mental disappearance
• Pirates Ubicomp Game, PLAY
• Intrabody communication, UW

22
Other Random Thoughts

• Invisible, Calm Computing was better received by
  press than "Big Brother" Location Tracking
• Early ubicomp research heavily systems-oriented
  because no standard platforms, many requirements
• Devices, net access, tracking, applications
• Ubicomp has many failure modes
• Hardware, software, infrastructure, social
• Most of ubicomp has focused on individuals versus
  groups

23
Why Ubicomp Failed (A hypothetical 10 year
retrospective)

• Technical Issues
• Couldn't scale, instability of system
• Power, too many batteries
• Modeling, couldn't get an interoperable world
  model
• Interoperability, nothing works with each other
• Critical mass, not enough sensors to be useful
• Usability and Usefulness
• Ubi-maint, Ubi-beeping, Ubi-overload
• Social Acceptance
• Never addressed privacy adequately

24
Gutenberg Museum, Mainz

• Lots of form factors
• Foldouts
• Small, large books
• Paper cutouts
• Full justify
• What's missing?
• Titles
• Page numbers
• Chapters / Headers
• Like early web!

25
(No Transcript)
26
The Context FabricInfrastructure Support for
Context-Aware Computing
Jason I. Hong Sept 28, 2001
27
Motivation

• Modern computers are divorced from our reality
• Unaware of who, where, and what around them
• Leads to mismatch
• Computers have extremely limited input
• Aware of explicit input only
• A lot of effort to do simple things (or to
  remember)
• Context-Aware Computing
• One line of ubicomp research
• Making computers more aware of the physical and
  social situations they are embedded in

28
Why Context-Aware Computing?
Context Types
Existing Examples
Human Concern
29
Some Technology Trends

• Sensors
• GPS, Active Badges, Active Bats
• Smart Dust
• Cameras and microphones
• Recognition algorithms
• MSR Radar, location from 802.11
• Smart Floor footstep force
• Wireless technologies
• Bluetooth, 802.11, cell phones

30
Why Context-Aware Computing?
Existing Examples
Context Types
Potential Examples
Human Concern
Activity
Convenience
Identity Time Location Proximity
Activity History
Auto Lights On / Off
Auto Cell Phone Off In Meetings
Activity
Finding Info
File Systems
Tag Photos
Identity
Memory
Calendar Reminders
Proximal Reminders
Identity Time
Safety
Smoke Alarm
Health Alert
Time
Efficiency
Barcode Scanners
Service Fleet Dispatching
31
Overview

• Background and Motivation
• Some Views on Context-Awareness
• Two Scenarios
• The Context Fabric
• SpeakEasy

32
What's Context?

• Some views on Context
• Situated Action, Activity Theory, Phenomenology,
  Distributed Cognition, Linguistics
• Sophisticated views on how people understand and
  interact with the world, and with each other
• A computational view (bottom-up)
• Increase input channels into computer
• Push towards implicit acquisition
• Create better models to take advantage of input
• Using the input models in useful ways

33
Desktop View
We sort of know how to handle context-awareness
here
34
Ubicomp View
35
What's Different?

• Harder to get input
• Acquisition is distributed, not fixed set of
  input
• Acquisition is fuzzier (sensors)
• Need statistical approaches (HMMs, Bayesian Nets)
• Modeling is distributed
• What to model? How to model?
• Dynamic set of sensors, services, and devices
• OnStar custom-built with complete control
• Privacy and Security of greater concern here
• A lot more implicitly captured data

36
What's Similar?

• Introspection
• Why did it do that?
• What does it know about me?
• Bad guesses
• Context for systems (Paperclip)
• Context for people (Yahoo IM status)
• Sensemaking
• What can I do here? What limitations?

37
Overview

• Background and Motivation
• Some Views on Context-Awareness
• Two Scenarios
• The Context Fabric
• SpeakEasy

38
Two Example Scenarios

• Scenario 1 "Where is an open alcove in Soda?"
• One-time query or repeat notification?
• How to model the situation?
• What is the granularity of an alcove?
• What data to store? Exact people? Some people?
• History
• Store history? Where does it live? For how long?
• Privacy
• Who can make this query? Access control? Physical
  presence?
• Where does the data live? And how to process it?

39
Two Example Scenarios

• Scenario 2 "Where is Jason?"
• Multiple sources
• Cell phone, desktop computer, Yahoo Messenger
• Ambiguity, reliability, age of different sources
• Where does the data live?
• How is it protected?
• Proximity? Group by Family, Friends, Strangers,
  etc?
• Views
• Earth, North America, Berkeley, Chez Panisse
• History
• Where does it live? How long to store?

40
Two Example Scenarios

• Possible to create custom-apps in each case
• Need something that
• Works independently of set of devices, sensors,
  services (as much as possible)
• Scales space, time, people
• Provides lots of support for
• Acquisition, modeling, processing, protection of
  context

41
Context Fabric

• Infrastructure for building context-aware apps
• Distributed data model of people, places, things
• Context Specification Language (like SQL)
• Other non-functional goals
• ie, it would be really nice if
• Agnostic of sensor, CPU, OS, programming lang,
  network, discovery service, service platform
• Able to evolve and incrementally deploy

42
Context Data Model

• Question 1 - How is context represented?
• Entities
• Like nouns, people, places, and things
• Attributes
• Like adjectives or properties
• Ex. identity, on / off, location, activity
• Relationships
• How one entity relates to another entity
• Aggregates
• Actions, Groups of people

43
Context Data Model
Name Jason Hong Org Xerox PARC Role Consultant
Devices a, b Location c
44
Context Data Model
Action Name Arrive Time 830AM Room 2509
History
45
Context Data Model

• Question 2 How long does info live?
• Seems application dependant
• Question 3 Ontology?
• Is there a single way of representing and talking
  about the world?
• No universal ontology, situation and app
  dependant
• Idea Support multiple schemas simultaneously
• PARC Schema for one set of context
• Berkeley Schema for another set
• What is the granularity of a place? Of a thing?
• Make it like databases and XML, let specific
  domains define

46
Context Data Model

• Question 4 Where exactly does context live?
• Entity, Attribute, Relationship, Aggregates are
  logical
• Idea Use tuple-spaces, a shared place where
  distributed apps can put and get data
• Exists independently of sensor, process, app,
  device

Sensor
write
App
take
App
read
47
Context Data Model

• Question 5 How to scale up to the world?
• Idea Multiple distributed tuple-spaces
• Logical entity
• E.g. "Mark's context" or "HP Auditorium context"
• Physical entity
• E.g. "Part of Mark's context on device xyz"

48
Context Data Model

• Question 6 Protecting context data?
• Access Control Lists on who can read, write
  attributes
• Provide policies based on groups
• Family, friends, co-workers, strangers
• Temporary access too (1 hour, 1 day)
• Certificates for authentication of requestor
• Different views on data
• Family and friends see "Jason is at Chez Panisse"
• Co-workers see "Jason is out of the office"
• Strangers see "Jason is in the Solar System"

49
Context Data Model

• Question 7 Is this the AI tarpit?
• What can and cannot be modeled?
• Question 8 What does all this really buy us?
• Separates acquisition, model, usage
• Easier to evolve, more resilient to failure
• Multiple schemas provide flexibility
• Context data lives separately from process,
  application, device, you can always get to it
• Templates for basic privacy policies
• Family, friends, co-workers, strangers

50
Context Data Model

• "Where is an open alcove in Soda?"
• Each organization can define "place" locally
• E.g. Berkeley Computer Science

51
Context Specification Language (CSL)

• Problem Difficult to coordinate data and
  services to get the right context data
  procedurally
• Idea Declaratively specify what you need
• Query
• "What are the nearby movie theaters?"
• How many people are in the room right now?
• Events
• Notify me every time a person enters the room.

52
Context Specification Language
What are the nearby movie theaters?
Person
Location, GPS
Person
Context Output
Movie Theaters
Location, Cell
Person
Proximity
53
Context Specification Language

• Question 9 How is context data manipulated?
• Context operators, like database operators
• Select, project, join, etc
• Interpreters
• E.g. motion gt person ID, or wireless data to
  location
• Fusers
• Merge multiple streams together, e.g. sensor
  fusion
• Data format transducers
• Celsius to Farenheit
• Exchange
• Convert push service to pull, or pull to push
• Filters
• Get the top five, or eliminate things that can't
  be right

54
Context Specification Language

• Question 10 How to actually process it?
• Parse the query
• Generate a few equivalent plans for getting the
  data
• Find a decent plan
• Execute it
• Databases have a nice clean algebra
• This one is definitely messier, with unstructured
  data stored all over the place
• Question 11 Is this overkill? Maybe use
  templates for common types of queries?
• "What are the nearest X?" "Where is Y?"

55
Context Specification Language

• Question 12 What does this buy us?
• Introspection, "what's going on?" and "why did it
  do that?"
• More robust to changing set of sensors, services,
  devices
• Question 13 Is this the AI tarpit again?
• Don't want to solve Natural Language Problem!
• However, it would be great to have a query
  language merging physical and virtual information

56
Context Data ModelCurrent Status

• Context Data Model
• Looking at Semantic Web, Knowledge Representation
• Looking at different tuplespace implementations
• Context Specification Language
• Working out all the basic queries
• Need more work on how to assemble paths
• Overall
• Working out more end-to-end stories
• Figuring out one or two issues to go deeply into
• Privacy and scale

57
Overview

• Background and Motivation
• Some Views on Context-Awareness
• Two Scenarios
• The Context Fabric
• SpeakEasy

58
Speakeasy Supporting the Ubiquitous Computing
User Experience

• Mark Newman, Keith Edwards, Jana Sedivy, Chris
  Neuwirth, Karen Marcelo, Trevor Smith, Jason Hong

59
Motivation

• The era of ubiquitous computing is upon us
• many devices per person, becoming interconnected

60
The Speakeasy Vision

• Enable Network Effects
• analogous to phone, fax, web
• Radical Interoperability
• what if anything can talk to anything?
• every new device or service adds value
• Deal with Complexity
• support users sensemaking
• what can I do in this world?
• what the heck is going on?

61
Radical Interoperability

• No need to write specifically for a new component
• Interact with components youve never heard of
• Interact with types of components youve never
  heard of
• Allow interoperability, defer semantics to people
• Our approach mobile code standard interfaces
• ( discovery shared network)
• Identify the minimal set of interfaces. So far
• - Data transfer transformation - Context
• - Status notification - User interface
• -

62
User Experience

• Context
• - Modeled as People, Places, Things (Components)
• - People
• What components have I used before?
• What components belong to me?
• - Places
• What components are in this place?
• - Components
• Where am I?
• What can I do?
• How have other people used me?
• Key Idea
• - Provide key information to people, little
  inference

63
End
64
Questions?

• "The most profound technologies are those that
  disappear. They weave themselves into the fabric
  of everyday life."
• Mark Weiser
• Context the circumstances in which an event
  occurs a setting to join to weave

65
Other Hard Questions

• Can we rely on computers to make decisions?
• Simple ones with little harm, e.g. Yahoo IM
  status
• Continuous ones with high benefit, e.g. smoke
  alarm
• What about complex ones, e.g. glass cockpits?
• Fair information practices?
• Notice, choice, access, security
• Privacy-Value tradeoff too
• Weak identity versus strong identity might help
• Weekly email summary of what happened
• "Privacy is ultimately a psychological construct,
  with malleable ties to specific objective
  conditions" (Grudin)

66
Other Hard Questions

• How to debug and simulate applications?
• Can't expect everyone to have lots of sensors

67

• Push and Pull
• Dealing with ambiguity, validity, and probability
• Allowing for statistical methods like HMMs and
  Bayesian Nets
• History of context over time and garbage
  collection of
• How much to save? Some is permanent, others
  transient
• Context hierarchies
• People gt Groups
• Architectural operators
• Filters, aggregators, fusion
• Debugging and Simulation Support
• Fake collected sensor data
• Tracing sensor data, from source to
  transformations
• UI to context info
• Feedback and feedforward
• Discovery / Replacement / Inference
• Where is my data?
• What if I can't find it? Replace it?
• Or infer from a set of sensors
• Security / Privacy / Trust

68
Context Data Model

• Question x Refining context data?
• Question 7 Metadata for context data?
• Precision / Accuracy / Granularity
• Source / Trace chain
• Owner
• Age
• Volatility
• Time to Live
• Trust level of sensor / trust level of sensor data

69
Some definitions of context

• "Any information that can be used to characterize
  the situation of an entity, where an entity is a
  person, place, or object that is considered
  relevant to the interaction between a user and an
  application, including the user and the
  application themselves. Context is typically the
  location, identity, and state of people, groups,
  and computational and physical objects." Abowd
  and Dey
• "Context refers to the physical and social
  situation in which computational devices are
  embedded" Moran and Dourish

70
Some definitions of context

• Distributed cognition
• Need to go beyond physical attributes
• Look at state of digital resources, peoples
  concepts, task state, social relations, local
  work culture (Kirsh context essay)
• Model underlying system of states, structures,
  and relations
• Summary model the key attributes of the whole
  system, the deep structure (instead of shallow
  surface structure) from individuals to offices to
  social structs and work practices

71
Some definitions of context

• Situated action
• actions are fluid, moment-by-moment, improvised,
  often unplanned, and highly context-dependent
• the context in which actions take place is what
  allows people to find it meaningful (Dourish
  context essay)
• Phenomenology
• How people experience the world and find meaning
  within it
• Shared meanings, common understandings
• How does the meaning of the world reveal itself
  to us through our actions within it?
• Thus, meaning (and hence context) arises from the
  ways in which we engage with and act within the
  world