Perspectives on the Anatomy of Ubicomp Experiences, ECT and Physical Configuration Management

1
Perspectives on the Anatomy of Ubicomp
Experiences, ECT and Physical Configuration
Management

• Chris Greenhalgh
• 2005-07-22

2
Content

• Classes of Ubicomp Application
• Example Blowaway
• Various perspectives
• ECT
• Physical Configuration Manager
• Behaviours

3
Classes of Ubicomp Application (1)

• presentation/filtering of "contextually relevant"
  information
• esp. location-relevant, automatically, on a
  personal display
• e.g. touring guide, museum guide
• e.g. interruption management
• including localised task support
• e.g. navigation, wayfinding (GPS navigation
  system)
• e.g. maintainance (photocopier -),
  crisis/disaster activities

4
Classes of Ubicomp Application (2)

• capture of local (esp. physical world)
  information
• recording and recall
• e.g. prosthetic memory, smart-camera, memories
  for life, contextual reminders
• e.g. surveying, scene-of-crime, site-specific
  planning
• e.g. authoring
• remote monitoring and anomaly detection
• e.g. elder-care, health-care, environmental
  monitoring, security (prison or fortress)
• also e.g. "awareness" less intrusive/oppressive?!
  

5
Classes of Ubicomp Application (3)

• device/location-independent interaction
• e.g. mobile (software) interfaces and
  applications
• payment?!, mobile/distributed access to
  applications/services/processes, e.g. food
  ordering
• e.g. ensemble/distributed interaction (esp. ad
  hoc/dynamically configured)
• using nearby interfaces with personal devices,
  offloading personal storage/computation,
  personal-area networks/convergence

6
Classes of Ubicomp Application (4)

• augmented (physical) artefacts and environments
• artefacts, e.g. phicons, tagged paper, AR
  (CYSMN?!), smart-its chemical containers
• systems, e.g. tangible interfaces
• envionrments, e.g. responsive environments, art
  installations, "smart" home

7
Classes of Ubicomp Application (5)

• social/collaborative
• communication
• e.g. telephony
• virtual co-location (and collaboration), e.g.
  CYSMN, TheMacRoom
• social prosthesis
• e.g. p2p "jewelery" for introduction/identificatio
  n, wireless game trading/gift extensions
• more general brokering?

8
Example Blowaway a large augmented
artefact/tangible interface

• Illustrations from http//www.blowaway.org/

9
The Blowaway Team (1) Hardware (sensors
interfacing)
10
The Blowaway Team (2)Table design (physical
elements)
11
The Blowaway Team (3) Programming
12
The Blowaway Team (4) Design (concept content)
13
The Blowaway Team (5) Audio (FX soundtrack)

• -)

14
The hardware perspective

• Physical input/output requirements
• Choice of sensors/actuators/displays
• Physical tailoring/linkages/etc
• Signal distribution/conversion/connect/soldering
• Computer interfacing
• Component choice, PCB design/fab, PIC programming
  
• or OTS or both
• Device driver?

15
The physical artefact/environment perspective

• Casing
• Cabling
• Support/structure/mounting
• Finish
• Setting/backdrop
• Situation

16
The software perspective

• Game engine
• Rendering framework
• Input interfacing
• Physics/behaviour framework
• Game logic/progression framework
• Tools
• Layout design

17
The content perspective

• Media assets
• (visual) Sprites, backdrops, animations
• Background soundtrack, spot FX
• Organisation
• Allocation of assets to game objects
• Map layouts, enemy placement
• Behaviours/constraints/tuning
• Depends on game engine (may be built-in)

18
The production perspective

• Deadlines
• Scheduling
• Contingencies
• division of labour
• Total experience management

19
Example Uncle Roy
CONTROL ROOM
FRONT OF HOUSE
Game Manager
ManagementClient
Registration Client
PIG SERVER
ONLINE PLAYER TERMINAL
STREET PLAYER PDA
EQUIP
PDA FLASH Client
XML/EQUIP STUB
Shockwave Client
DATABASE
GPRS
INTERNET
AUDIO encoder
GPRS
WEB/PHP SERVER
OFFICE
EQUIP AR PROCESSOR
Augmented Video View
CCTV WEBCAM
20
The orchestration/experience perspective

• Player recruitment
• Induction/debriefing
• Security
• Monitoring/trouble-shooting
• Intervention/guidance

21
The ethical perspective

• Trust
• Security
• Potential abuse
• Re/use and informed consent
• Health and safety
• Perception
• Effect
• Responsibility

22
The (current) ECT perspective component
composition configuration

• Create/locatecomponentson appropriate
  hostse.g. device drivers,software
  services,behaviour, glue
• Set configuration property values(e.g. COM
  port)
• Create links betweenproperties to
  createdata-flow graph
• Train/script behaviour

Host A
Devices
SmartIT RF
RS-232
outTouch
SmartIT
Link
CaptureState
Camera
VFW, USB
SmartIT w/sensor
url
Serial SmartIT
url
Media viewer
minimum
maximum
VGA
current
Host C
Installation Data-space (Host A)
Components
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Sussex Interactive Skipping what you would
see if you could look down the serial cable
28
ECT

• Reuse of transferable components
• Esp. device drivers
• Some generic logic components
• Supports multi-machine distribution
• E.g. RCA shared lamps
• Supports recording replay for analysis
• Supports run-time introspection
• Authoring
• Monitoring/debugging
• Supports non-programmer assembly of intallations

29
ECT -

• Hard(er) to write good components
• Low-level (generic!) inputs/outputs (e.g. tag
  ids)
• Performance overhead
• Limited component interaction modes
• Mainly 11 linked property values
• No high-level support for dynamic configurations
• Software-oriented perspective
• No content model(s)
• write a (e.g.) processing component!

30
ECT/Ubicomp Applications

• Good for
• Plumbing fixed set of I/O devices to
  opaque/monolithic behaviour component(s)
• E.g. responsive artefacts/environments
• (Currently) Bad for
• Dynamic configurations and distributed behaviours
• E.g. systems with mobile/handheld devices
• Supporting content

31
Suggestion The Physical Aspects of the System
should Explicitly Modelled and Accounted for

• Hypothesis
• Applications are more naturally and more readily
  expressed and understood in terms of (primarily)
  physical aspects

32
Potential utility

• Support for configuration and assembly
• e.g. matching devices and driver components
• Support for understanding
• Monitoring, debugging, orchestration
• Record and reuse/analysis
• Support for behaviour specification
• Support for other content stereotypes

33
Work in progress ECT Physical Configuration
Manager
34
What we see in the dataspace
35
Modelling the PC, Reader artefact
36
Note things the system does not know

• Exactly what hardware is present
• What is plugged into a port (when the protocol is
  limited)
• What thing the RFID tag is attached to
• What particular ID an unseen tag has
• What the physical arrangement of things is
• What the significance of things is

37
Support for configuration and assembly (1)
rule-based request for component
38
Support for configuration and assembly (2)
rule-based configuration
39
(No Transcript)
40
Support for understanding (1) rule-based
mapping of hw?sw
41
Support for understanding (2) rule-based
inference of physical state
42
Support for behaviour specification behaviours
specified w.r.t. physical state
43
Brief reflection on ubicomp behaviour content

• Behaviour is expressed as sets of contingencies,
  encoded in different ways
• If A then B
• Content is that part of the behaviour which is
  readily substitutable within the framework of the
  behaviour engine

44
physical
Model of the world
digital
Content
Behaviour engine
Output
Input
artefacts
History
people
environment
45
ECT Ubicomp behaviours

• Often based on the state of physical things
• People, places, activities,
• ECT property values are too low level
• E.g. a list of IDs, a float, a boolean, a
  lat/long
• A lot of extra context is implicit in the
  particular property it is
• E.g. on a particular reader, from a particular
  sensor, of a particular person or device
• Current direct connected and encapsulated
  behaviour component approach
• Forces mapping of behaviour to these low-level
  values
• Requires behaviour programmer to keep track of
  this in their head

46
And

• ECT has no standard behaviour engine(s)
• Other than programming/scripting languages
• ECT (therefore) has
• No standard content interface(s)
• No standard history interface(s)/facilities
• for use within-experience
• These all presume a higher-level model of the
  world within the system
• But tend to embody a particular
  application/interaction stereotype discuss

47
So

• what