Sensorbased interaction

1
Sensor-based interaction

• Designing for implicit interaction

2
Agenda

• Questions
• Job announcement User Insight
• Friday April 16, Industry Panel (need questions)
• Ubiquitous Computing
• Everyday interaction
• Evaluation challenges
• Sensor-based interaction
• Revisit context
• Examples of implicit interaction
• Designing interactions for sensing

3
For more information

• CS 7470 Mobile and Ubiquitous Computing
• Conferences
• Ubicomp (since 1999)
• Pervasive Computing (2002, 2004)
• IEEE PerCom (2003 2004)
• Journals
• Springer Verlag Personal and Ubiquitous Computing
• IEEE Pervasive Computing (a gift!)

4
HCI Challenges in Ubicomp

• Applications
• the essence of Ubicomp
• Natural interaction
• Context-aware computing
• Automated capture and access
• Everyday computing

5
Everyday Computing

• Supporting the continuous ebb flow of action in
  everyday life
• Scaling w.r.t. time
• Activity unstructured, informal, continuous
• Task well-defined, production, begin/end

Write presentation blank document enter
text format print and save
Write about ubicomp multiple drafts multiple
documents reuse outside work evolving thought
6
Designing for Everyday Activities

• No clear beginning or end
• Closure vs. flexibility and simplicity
• Interruption is expected
• Design for resumption
• Concurrent activities
• Monitoring for opportunity
• Time is important discriminator
• Interpret events
• Associative models needed
• Reacquire information from multiple pts of view

7
Evolution of Augmented Whiteboard

• LiveBoard - Natural input
• Tivoli - Meeting capture
• Dummbo
• Context in support of impromtu meetings
• Flatland (CHI 99)
• lightweight behaviors
• visibility
• associative storage
• browsing in time

8
Challenge of Evaluation

• Bleeding edge technology
• Novelty
• Unanticipated uses
• Quantitative metrics

9
Evaluation Strategies

• Bleeding Edge
• Audio Aura Scenarios
• Lightweight Ethnography
• Flatland
• Long-Term Studies
• Classroom 2000

10
Sensor-based interaction

• Revisit context
• Examples of implicit interaction
• Designing interactions for sensing

11
What is Context?

• Any information that can be used to characterize
  the situation of an entity
• Who, what, where, when
• Why is it important?
• information, usually implicit, that applications
  do not have access to
• Its input that you dont get in a GUI

12
How to Use Context

• To present relevant information to someone
• Mobile tour guide
• To perform an action automatically
• Print to nearest printer
• To show an action that use can choose
• Want to phone the number in this email?

13
Examples of sensing
14
Real-time audio buffering

• MERL Cambridge
• Buffering phone conversations
• Uses near-field capacitive sensing

15
Writing and sensing
16
General platform for sensing with handhelds
(Microsoft)
17
Interaction Techniques

• Voice memo detection
• Display mode detection
• Tilt scrolling
• contrast compensation
• interaction with display mode
• Power management

18
SHARP

• System for Human Activity Recognition and
  Prediction
• The invisible man approach

19
Designing for implicit interaction

• The intentional spectrum
• Matching actions to intentions
• Revisiting interaction models
• Contrasting physical and electronic worlds
• Designing for implicit interaction
• Principle of appropriate intelligence

20
Car courtesy lights

• turn on
• when doors unlocked/open
• turned off
• after time period
• when engine turned on

driver's purpose is to get into the car
incidentally the lights come on
21
Pepys

• Rank Xerox EuroPARC
• active badges
• automatic diaries

Allan's purpose to visit Pauls office
incidentally diary entry created
22
MediaCup

• cup has sensors
• heat, movement, pressure
• broadcasts state (IR)
• used for awareness
• user is moving, drinking,

Han's purpose to drink coffee
incidentally colleagues are aware
23
shopping cart

• goods in shopping cart analysed
• e.g. Amazon books
• used to build knowledge about books
• people who like X also like Y
• used to give you suggestions
• you might like to look at , special offer

my purpose to buy a book
incidentally shown related titles
24
onCue

• intelligent toolbar
• analyses clipboard contents
• suggests things to do with it

user's purpose to copy text elsewhere
incidentally alternative things to do
25
The Intentional Spectrum
intentional
press light switch
26
Fluidity
intentional
expected
incidental
27
Interaction Models

• intentional cycle
• Norman execution/evaluation loop
• some exceptions
• multiple goals, displays, opportunistic
• guidelines
• feedback, transparency

goal
evaluation
execution
system
28
Assumptions underlying physical world interactions

• physical things (inanimate)
• directness of effect
• locality of effect
• visibility of state
• computational things (also animate)
• complex effects
• non locality of effect
• distance networks time delays, memory
• large hidden state

29
Dealing with uncertainty

• traditional system
• due to design flaws
• sensor-based
• intrinsic to design
• uncertain readings, uncertain inference
• usually control non-critical aspects of
  environment

30
Appropriate intelligence

• Be right as often as possible, and useful when
  acting on these correct predictions.
• Do not cause inordinate problems in the event of
  an action resulting from a wrong prediction.