Lecture 14 Pervasive Computing Applications

1
Lecture 14Pervasive Computing Applications

• Wireless Networks and Mobile Systems

2
Lecture Objectives

• Understand characteristics and technical
  challenges of pervasive computing applications
• Understand system middleware and support for
  context-aware ubiquitous computing
• Exemplify pervasive computing applications

3
Sources

• T. Kindberg and A. Fox, System software for
  ubiquitous computing, IEEE Pervasive Computing,
  Vol. 1, No. 1, 2002, pp. 70-81.
• A. Smailagic and D. Kogan, Location sensing and
  privacy in a context-aware computing
  environment, IEEE Wireless Communications, Vol.
  9, No. 5, Oct. 2002, pp. 10-17.
• S.S. Yau, et al., Reconfigurable
  context-sensitive middleware for pervasive
  computing, IEEE Pervasive Computing, Vol. 1,
  No. 3, 2002, pp. 33-40.
• M. Roman, et al., A middleware infrastructure
  for active spaces, IEEE Pervasive Computing,
  Vol. 1, No. 4, 2003, pp. 74-83.
• B. Johanson, et al., The interactive workspaces
  project experiences with ubiquitous computing
  rooms, IEEE Pervasive Computing, Vol. 1, No. 2,
  2002, pp. 67-75.

4
Agenda

• Characteristics of pervasive computing
• System support to enable pervasive computing
• Middleware for context-aware pervasive computing
  applications
• Examples of pervasive computing applications
• Smart homes/rooms
• UIs Gaia-enabled Active Spaces
• Stanfords Interactive Workspaces project
• Experimentation
• SLP-based pervasive Pocket TV application

5
Pervasive Computing Characteristics (1)

• Pervasive computing is
• An environment in which people interact with
  embedded (and mostly invisible) computers and in
  which networked devices are aware of their
  surroundings and peers and are able to provide
  services or use services from peers effectively
• The creation of environments saturated with
  computing and wireless communication, yet
  gracefully integrated with human users

6
Pervasive Computing Characteristics (2)

• A pervasive (ubiquitous) system is characterized
  by
• Physical integration integration between
  computing nodes and the physical world, e.g., a
  whiteboard that records whats on
• Instantaneous Interoperation devices
  interoperate spontaneously in changing
  environments, e.g., a device changes its partners
  as it moves or as the context changes

7
Non-Examples of Pervasive Applications

• Accessing email over a phone line from a laptop
• Neither physical integration nor spontaneous
  interoperations
• The laptop maintains the same association with a
  fixed mail server
• A collection of wirelessly connected laptops
  based on IEEE 802.11 at a conference
• Discovery of the local network can be considered
  as physical integration of laptops with a 802.11
  access point
• The physical integration in pervasive computing,
  however, should involve a part of the environment
  that has a non-electronic function (e.g., a white
  board, a cup, etc.)
• No spontaneous interoperation is possible without
  considerable human manual intervention

8
Borderline Examples of Pervasive Applications (1)

• A smart coffee cup and saucer
• Physical integration The cup is a regular cup
  (a non-electronic function) that contains
  sensing, processing and networking elements that
  let it communicate with the saucer of its state
  (full or empty, held or putdown, hot or cold)
• Instantaneous interoperation Not satisfied if a
  specialized protocol is used between the cup and
  saucer as the owner would not be able to use the
  coffee cup without the saucer

9
Borderline Examples of Pervasive Applications (2)

• P2P games
• Physical integration client devices can have
  sensors that allow physical integration
• Instantaneous interoperation not satisfied if
  preconfigured components are used in the game
• A more convincing example would involve players
  with generic game playing pieces that let them
  spontaneously join local games not encountered
  before

10
Borderline Examples of Pervasive Applications (3)

• A Web-based object discovery system
• Devices (e.g., laptops) have small, embedded web
  servers that allow objects to be accessed by
  physical hyperlinks
• The user is presented with a web page when it
  senses an identifier of the object
• Physical integration
• smart devices can use web functionality to allow
  advertised objects to be integrated into the
  physical world as web pages without the need to
  reconfigure browsers
• Instantaneous interoperation
• not exactly satisfied since the system requires
  human intervention to keep it going. The human in
  the loop changes the browsers associations to
  advertised web services

11
Examples of Pervasive Applications (1)

• A magic mirror that shows personal data and
  actions of those users who face it in a meeting
• Physical integration the mirror can sense the
  presence of users in a meeting and can record
  their actions, in addition to its normal physical
  function
• Instantaneous interoperation the mirror would
  interact with the rooms other components the
  moment you switch it on and would make
  spontaneous association with all relevant local
  sources of information about users

. . .
12
Examples of Pervasive Applications (2)

• A visitor brings his/her laptop into a meeting
  room and without manually configuring it in any
  way uses it to send his presentation to the
  rooms projector
• Physical integration the projector can be
  activated from any laptop in the room
• Instantaneous interoperation A laptop can
  spontaneously interact with the projector and
  control the presentation with a proper UI
• Can be made context-sensitive, e.g., allowing
  only a particular visitor to do so

13
Other examples Miró display
Cornell experiment reflections on collective
experience
14
Other examples Gust of Presence
Delft Univ. of Technology Gust of Presence
project
15
The Semantic Rubicon
Little evidence exists to suggest that
software alone can meet ubiquitous computing
challenges, although it is desirable to have
minimum or no intervention
Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 1, 2002.
The semantic Rubicon is the division between
system and human for high-level decision-making
or physical world semantics processing The
division should be exposed in system design and
the criteria and mechanisms for crossing it
should be clearly indicated
16
System Support

• Discovery
• Requiring a common syntax and vocabulary model
  for specifying services

Service Type Mismatched
ServiceTypephotoPrint
ServiceTypeprinting
Missed opportunity
ServiceTypedigitalFrame
17
System Support (2)

• Interaction
• Requiring a common interoperation model for
  components to interact
• Event systems publish, subscribe, and handle
  events
• Tuple space add, read, and remove tuples in a
  tuple space

(Standardized) Data Oriented Interaction Service
ServiceTypePrint DataTypeJPEG
ServiceTypeprint DataTypeJPEG
A common service A tuple space or an event
service
ServiceTypedigitalFrame DataType JPEG
18
System Support (3)

• Adaptation
• Without human intervention for achieving calm
  computing as well as spontaneous interaction
• Being able to display or manipulate data or UI
  from other devices in a heterogeneous environment
• Possible solutions include
• Dynamic downloading of mobile code to handle UI
  and communication (as in Jini) requiring JVM
• Dynamic UI generation/transformation (as in
  Stanfords Interactive Workspaces) requiring a
  common UI description language to describe UI
  elements, e.g., This control is a Boolean
  toggle
• Another example is to use XML for specifying UI
  for web services

19
Human Interface Adaptation

• Use the same high-level UI description language
  to generate device-specific UIs to control
  lights and displays in a smart room for (a) a
  desktop HTML browser, (b) a Java-equipped device,
  and (c) a Palm device

Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 1, 2002.
20
System Support (4)
Context aware

• Physical Integration
• Context-awareness support is required to allow an
  application to access different types of contexts
  (e.g., locations, identities, QoS conditions)
  while hiding how the information was sensed or
  collected
• Examples
• Show me the agenda! - can be made
  context-sensitive
• Show me related Web pages for museum exhibits!
  can be based on the location and ids of
  exhibits sensed through barcode or Infrared
  beacon

I know who I am photographing
I know you provide photo printing services
tag
tag
21
Positioning for Location-aware Computing

• Outdoor GPS (Global Position Sensing) normally
  to within 1-5 m accuracy with differential
  correction
• Indoor
• Active Bat (shown below) developed by ATT Lab
  Cambridge based on ultrasonic sensors (to within
  3 cm accuracy)
• CMU-TMI (shown next) developed by CMU based on
  802.11 for location tracking(to within 1-5 m
  accuracy)

Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 1, 2002.
22
System Support Location Tracking in Smart Spaces

• CMU has developed and deployed a Triangulation,
  Mapping and Interpolation (CMU-TMI) algorithm
  based on 802.11 for location tracking
• Physical locations of access points are known in
  advance
• A mobile device scans all access points (at least
  3) within range to determine their signal and
  noise strengths
• The signal strengths are used to infer the
  distance between the device and the access points
  using an approximate relation obtained a priori
• Triangulation technique is applied to calculate
  the physical location

23
System Support (5)

• Programming Framework
• A programming framework for pervasive computing
  can be placed at the application or middleware
  level
• Application level
• Providing only application-level coordination
  mechanisms, e.g., through a tuple-space API
• Middleware level
• Providing more tightly integrated facilities for
  achieving context awareness, QoS resource
  management, and adaptive control.
• Determining the users task and intention, and
  facilitating associations between components to
  assist the user in these activities

24
System Support (6)

• Robustness
• Failure is a common case in mobile and pervasive
  computing
• Recovering from failures ? always prepared to
  reacquire lost resources
• Expiration-based schemes and soft states
• Periodic service advertisement and lease in the
  common data-oriented interaction service
• Separating failure-free (logic) from
  failure-prone (e.g., accessing a service or file
  while moving) application code
• More effective failure handling can be done
  implicitly by the middleware, e.g., retrying an
  idempotent operation (binding to a remote file)
  for a number of times
• Security trust between devices, access control

25
Middleware for Pervasive Computing

• Context-Aware middleware for pervasive computing
  must address two broad characteristics
• Tradeoff between awareness and transparency
• For pervasive applications, environment awareness
  is key to their effectiveness
• Cooperation between development support and run
  time services
• Let developers focus on application logic

26
RCSM A Reconfigurable Context Sensitive
Middleware

• ASUs RCSM addresses the following specific
  characteristics
• Uniform development support
• providing a uniform way to express context
  awareness without restricting to a specific
  language, OS, or environment
• Application-specific context acquisition,
  analysis and detection
• allowing developers to express the need for
  context data in a platform-independent way
  without knowing how data are obtained
• Context-triggered actions
• transparently invoking actions whenever the
  specified contexts are valid
• Transparent support for ad hoc communication
• abstracting the details of ad hoc communication
  from applications including proactively
  discovering new devices, establishing new
  communication links and notifying the application
  when a device is found

27
RCSM A Reconfigurable Context Sensitive
Middleware (2)

• ASUs RCSM provides a context-aware interface
  description language (CA-IDL) API for an
  application to
• Define contexts used by the application
• Specify actions taken based on context values
• Compilers are provided to build adaptive object
  containers (ADCs) based on CA-IDL

Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 3, 2002.
28
Pervasive Computing Example 1 UIs Gaia-enabled
Active Spaces (1)

• Gaia is a distributed middleware infrastructure
  that coordinates software entities and
  heterogeneous network devices contained in a
  physical space
• The physical space is extended into an active
  space by adding coordination via a context-based
  software infrastructure

Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 4, 2002.
29
Pervasive Computing Example 1 UIs Gaia-enabled
Active Spaces (2)

• Event manager service for event channel
  registration/notification
• Context service for querying/registering context
  information such as peoples locations
• Presence service for maintaining soft states of
  resources including digital entities
  (applications and services using heartbeats) and
  physical entities (devices and people using
  sensors)
• Space repository for displaying/retrieving
  information about all entities in the active
  space
• Context file system associating data with
  context and format info to allow
  context-sensitive data retrieval presented with
  the right UI format, e.g., make personal data
  available to applications conditioned on presence

Example Presentation Manager Application
Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 4, 2002.
30
Pervasive Computing Example 1 UIs Gaia-enabled
Active Spaces (3)

• EX Presentation Manager Application
• Speaker X with an RF active badge and a handheld
  walks into a Gaia-enabled room
• Presence service detects the badge and sends a X
  is here event with the user profile info of X
  contained
• Space repository receives the event and displays
  the user info.
• Context file system obtains the speakers mount
  point and mounts the speakers presentation file
  ready for access
• The handheld (controller) detects the directory
  server and then sends a heartbeat event to the
  heartbeat channel
• Presence service detects and sends a new device
  found event
• Space repository receives the event and displays
  the handheld info
• Both speaker X and handheld are now entities of
  the active space

A prototype of Gaia-enabled active space

• Context Speaker X enters the room
• Action start the presentation manager
  application
• Read the presentation file
• Allowing X to control presentation by control
  events generated from the touch screen or
  handheld

31
Pervasive Computing Example 2 Stanfords
Interactive Workspaces (1)

• Stanfords Interactive Workspaces is based on a
  tuple space data model
• Event Heap Allowing name-type-value events (with
  expiration times) to be posted and retrieved
• Data Heap Allowing data with attributes (e.g.,
  format) posted
• iCrafter Providing service advertisement/invocati
  on and a UI generator that returns the best
  interface for the device

Image extracted from IEEE Pervasive
Computing Magazine, Vol. 1, No. 2, 2002.
32
Pervasive Computing Example 2 Stanfords
Interactive Workspaces (2)

• Room-based cross-platform interfaces
• The room-control system stores the geometric
  arrangement of screens and lights in the room in
  a configuration file and will automatically
  provide controllers on any device supporting a UI
  format available through iCrafter

33
Experimentation SLP-based Pervasive Pocket TV

• SLP (RFC 2608) is an IETF standard that provides
  a scalable framework for automatic resource
  discovery on IP networks. Three entities are
  defined in a SLP system
• User agent (UA)
• Server agent (SA)
• Directory agent (DA)
• A UA initiates service discovery on behalf of one
  or more applications. It can send queries to all
  SAs via multicast if a DA does not exist or to a
  DA to discover services via unicast.
• A SA works on behalf of one or more services. It
  responds directly to UA queries via unicast. If a
  DA exists, a SA can register its services with
  the DA to expose its services.

34
Experimentation Pervasive Pocket TV (2)

• A DA serves as a centralized information
  repository. It accepts SA registrations and
  answers UA directory service queries. The DA
  support is optional and is introduced for
  performance and scalability considerations.

Application
Service
query
UA
SA
reply
service registration
directory service query
DA
directory service reply
acknowledgment
35
Experimentation Pervasive Pocket TV (3)

• DA Discovery Passive versus Active

Multicast SrvRqst (servicedirectory-agent)
UA/SA
DA
Unicast DAAdvert
Multicast DAAdvert
UA/SA
DA
36
Experimentation Pervasive Pocket TV (4)

• Three configurations

Small No DA
Multicast SrvRqst
Multicast SrvRqst
SA
UA
SA
Unicast SrvRply
Unicast SrvRply
Unicast SrvReg
Medium 1 DA
SA
Unicast SrvAck
Unicast SrvRqst
UA
DA
Unicast SrvRply
Unicast SrvAck
SA
Unicast SrvReg
Large Multiple DAs
UA
SA
SA
DA
SA
DA
UA
SA