Gaetano Borriello Department of CS

1
The Portolano Expeditionin Invisible Computing

• Gaetano BorrielloDepartment of CSEUniversity
  of Washington
• University of CaliforniaBerkeley, CA9 February
  2000

www.cs.washington.edu/research/portolano
2
Invisible computing photography scenario
3
An Earlier Age of Exploration

• Prince Henry the Navigator (1394-1460)
• Established school for navigatorsin 1450 at
  Sagres
• Portolano charts - first truerepresentations of
  coastlines
• Published in time for next voyagefurther down
  African coast

4
Principle Themes

• Invisibility
• not enough to be mobile, pervasive, ubiquitous,
  etc.
• users attention is the valuable resource
• minimize user configuration/maintenance/interactio
  n
• robust, reliable, safe, and trustworthy
• devices, middle-ware, and applications ?
  services
• Active fabric
• plug-and-play, discovery, composability
• data-centric, heterogeneous, active networking
• data and code mobility
• self-organizing, self-updating, self-monitoring
  systems
• active databases and information management
• External user community

5
World View

• What will be
• lots of task-specific devices
• rich connections to physical world
• world-wide information/computation utilities
• ever-increasing computation, storage, and
  bandwidth
• computing/communication capabilities in
  everything
• What wont be
• unlimited power sources
• homogeneous connectivity
• continuous connectivity
• increased user mind-share devoted to computing
  concerns

6
Expedition Goals

• Connecting the physical worldto the world-wide
  information fabric
• instrument the environment sensors, locators,
  actuators
• universal plug-and-play at all levels devices to
  services
• optimize for power computation partitioning,
  comm. opt.
• intermittent communication new networking
  strategies
• Get computers out of the way
• dont interfere with users tasks
• diverse task-specific devices with optimized
  form-factors
• wide range of input/output modalities
• Robust, trustworthy services
• high-productivity software development
• self-organizing, active middleware, maintenance,
  monitoring
• higher-level, meaningful services

7
Application domains (and collaborators)

• Labscape (UW Cell Systems Initiative)
• instrumentation of the workplace
• collect data to replay meetings/experiments
• data mining to support investigations/recollection
  s
• Personal devices/networks (Microsoft Research)
• body-area networking (RF and skin)
• borrowable, scrap devices
• Here-to-there (Intel and Ford)
• in-building/in-room location tracking
• continuous access to data
• management of personal devices and services

8
Research Themes

• Low-power intermittently connected devices
• Intentional user interfaces
• Data-centric networking
• Self-organizing information systems
• Invisible software development/deployment
• Seamless data/information management

services active fabric devices
9
Low-power Intermittently Connected Devices
wireless (heterogeneous)wired
devicegateway
web (data storesand services)
bodyserver
10
Low-power Intermittently Connected Devices

• Maximize lifetime of devices
• partition computation/storage into infrastructure
• code distribution (carried, delivered, gathered)
• distributed caching
• data-centric (not connection-centric) networking
• Devices for personal use
• borrowable (scrap) devices, (private) body
  communication
• body-area-server as cache/gateway, distribution
  point
• location sensing
• Devices in the environment
• simple and regular communication
• minimal state
• automatically deployed proxy services (for
  devices/data)

11
Intentional user interfaces

• Tags (RF cap, ind., bi-static)
• Phycons (IR signalling, pos.)
• Location sensing(RF sig. strength, timing)

PROJECTOR
Video Tape
Volume
12
Spot-on RF location sensing
Ethernet
RFbasestation
Spot-on mobilenode
Accelerometer RF signal strength
13
Intentional user interfaces

• Services
• of the person
• of the space
• interactions between subscribers
• Sensors/actuators
• tags, phycons, locators
• speech I/O
• video I/O
• User interactors post events, services react
• self-describing elements, agreement on event
  semantics
• link to services through discovery,
  auto-configuration
• setting expectations if its invisible, how to
  know its working?

14
Data-centric networking

• Data has a life of its own
• does not require connection from end-to-end
• Builds on active networking infrastructure
• uses computational resources of network nodes
• Needed due to intermittent connections
• from power or range limitations
• Application-specific data routing
• discovery, replication in addition to routing
• Aids in pushing code from services to devices
• data aggregation, transcoding

15
Connection-centric networking
16
Data-centric networking
17
Data-centric networking

• Supports low-power devices
• drop-in and run saves transceiver power
• ambient power harvesting becomes possible
• fosters power-oriented application partitioning
• 3-point acknowledgements
• receipts match-up with acks on return to base
• enhanced privacy/security (also through
  anonymizers)
• Requires active networking infrastructure
• ship code in data bundle
• need business model for charging for resources

18
Self-organizing Information Systems
distributedtuple-space
set up nodesqueries/actuations marshal resources
services
active fabric
devices
data finds repository propagates/replicates
19
Self-organizing Information Systems

• Active fabric
• active networking distributed data-driven
  computation
• reflective self-monitoring, self-measuring
• discovery protocols link device events to
  services
• active names
• Adaptive but stable, secure
• code mobility, caching
• control systems
• redundancy in communication/storage/computation
• privacy anonymizers, three-point transactions
• seamless flexibility/upgradability
• construct ad-hoc distributed systems from basic
  elements

20
Seamless Data/Information Management
Layered DataIntegration
services(pattern detectors)
sensors(input, e.g., tags, location, web info,
medical)
distributeddata proxies
actuators(output, e.g., alerts, on/off,
displays, robots)
21
Seamless Data/Information Management

• Query decomposition
• services based on sensor fusion (physical and
  virtual)
• redundant data sources
• learning for reliability/performance
• Pattern detectors
• code/data migrate into position
• standing queries
• Service/proxy/device manager
• watch over interactions, enhancement hints
• aid in structuring of layers, device collections,
  updates
• proxies for devices --- proxies for data

22
Model-based Design

• Pluggable reusable components
• Data-flow and control-flow connectors
• events and state changes

objects state
23
Software Development/Deployment
automatically synthesize code for communication/co
ordination/delivery
24
Software Development/Deployment

• Raise level of abstraction
• components with exportable interfacesfor
  data-flow and internal state
• coordinators to link interfaces
• Auto-generation of code
• delivery through device and/or service
• optimizations to architectural decomposition
• migration of functionsto support power/bandwidth
  tradeoffs
• simple run-time systems (vs. OS)
• Packaging of code
• into computing elements and to go with data in
  active fabric
• automatically build needed elements (eg,
  transactional store)

25
Self-monitoring

• If its invisible, how do you know its working?
• Functional code is not enough
• Checking code is needed
• Determine failure points
• find alternatives (from app. hints)
• change data policies
• retransmit from redundant stores
• inform user in a meaningful way

26
Labscape

• External user community
• motivated technical users
• support scientific endeavours
• Biological laboratory automation
• Cell System Initiative at UW Medical School
• experiment capture
• distribution of knowledge
• replay/reconstruction via data mining
• eventually lead to experiment design and
  execution
• rich sensor/actuator space
• rich user interaction modes
• task-specific with well-defined ontology

27
LabScape - one of our driver applications

• Biology is a hard science with a soft
  infrastructure
• capture and use of knowledge is key
• from loosely connected to highly integrated
  collaboration
• invisible infrastructure for building knowledge
  base

28
LabScape is an Ideal Application Driver for
Portolano

• Sophisticated butnon-IT user base
• Failure of LIMS desktop applicationsapproach
  to laboratoryautomation
• pen and paper labnotebooks still the norm
• Rich but rigorous ontology
• Green-field opportunityto explore invisible
  computingin the Cell Systems Initiative
  facility

29
LabScape Major Components

• Descriptive Model The Knowledge Base
• heterogeneous
• globally linked reviewed and raw
• semantically precise, without excluding narrative
• contradictory and negative information
• Experiment Manager Knowledge Capture and Use
• remote utilization of capital intensive resources
• capture data not known to be important
• collaboration across scientific community
• build the database by doing the work!
• access and augment the knowledge base

30
Scenario The Scientific Process

• Tom and Aparna are collaborative molecular
  biologists
• Aparna gets a hit on her standing query someone
  appears to have validated a hypothesis about the
  synthesis of a particular protein using a new
  experimental technique. Its Tom.
• Aparna wants to reproduce the experiment. Gets
  Toms protocol and assigns a student to perform
  the experiment.
• The student sees something questionable in Toms
  protocol and changes the experiment on the fly.
• Aparna reviews her students lab techniques and is
  pleased. Adds new hypothesis and publishes the
  raw data.
• Tom notices a new reference to his experiment and
  learns that Aparna claims to have invalidated his
  results. A quick query shows exactly what was
  different between the two protocols. Tom is
  pleased that his experimental techniques are
  being repeated.
• Tom sends a message to Aparna, thanks her the
  improvement and feedback. Tom reduces the
  confidence level of his hypothesis but does not
  remove it from the system. They publish their
  results by submitting the knowledge base for
  review.

31
Scenario
32
Scenario now cast in invisible terms
Invisible Computing Infrastructure
33
Prototype products in invisible computing

• Preferences holder with short-range RF
  communication
• GPS-enabled notes on a PDA
• Voice-programmable VCR
• Shoe-embedded pedometer to calculate distance
  walked/run
• Real-time rapid transit information on a PDA
• Adaptive remote control for home appliances
• Novel input modes for PDAs (tilting, pressure,
  orientation)
• PDA Calendar that checks traffic conditions and
  plans best route
• Plug-and-play home automation IR
  location-sensing RF to server
• Training bike with GPS, speed, heart-rate
  sensing, RF to PDA
• Medical tricorder logs blood-pressure/heart-rate/b
  lood-ox
• Prescription entry and mgmt system (PDA, service,
  pager)

34
Scrap remote control devices

• Remote control with touch screenslying around
  the house (WinCE palm-top)
• Point at an appliance and get UIand state (via
  IrDA)
• Controls appliance and/or asks formore command
  options

35
Intellipilot

• New input modalities for PDAs
• Use accelerometers, compass, shaft encoders,
  pressure sensors, light sensors, etc.
• New applications for PDAs (pedometer)

36
Calendar

• Intelligent agents
• Mobile clients
• Horizontally-layeredservices
• Route planning betweenlocations
• Reminders based on current traffic conditions
• Web database asrepository
• Java agents use database(get info, add entries)

37
Sitzmark

• Training datacollection
• Collect bike data(GPS, speed)
• Collect rider data(heart rate)
• PDA with RF
• PC data displayprogram(linked to maps)
• Database forlogginglong-term data

38
Current Exploratory Projects

• Body-area networking and body server/protocols
• Borrowable (or scrap) devices
• In-building/room fine-grain location tracking
• Modular heterogeneous sensor/actuator
  architecture
• Embedded web servers/gateways
• Generalized Jini-style proxy services
• Software partitioning/mapping app. dev. and
  depl.
• Active networks enhancements
• Self-organizing network overlays
• Active names
• Labscape infrastructure
• Physical icons, tags, and sensors
• Integrating speech with sensors/tags
• Speech/conversation/meeting browser
• Speaker/language modeling/identification

39
Current Projects

• Arcade
• sensor/service plug-and-play
• data lifetimes, distributed tuple-spaces, data
  proxies
• Hendrix
• migrating mediated data streams
• follow-me audio using speakers in environment
• Spot-on
• multi-grain, multi-technology location sensing
• location service for people/objects
• Active fabric
• Jini proxy services for low-power devices
• enhanced active network nodes (multiple address
  spaces)

40
Summary

• Creating a new world of invisible computing
• where computing fades into the background
  (Weiser91)
• low cognitive load
• Integrated approach from applications to I/O
  devices with an active middle-ware, database, and
  networking fabric in between
• Application domains to drive vertically
  integrated research with strong
  collaborators/users
• www.cs.washington.edu/research/portolano