Scalable Coordination Algorithms for Deeply Distributed Systems

1
Scalable Coordination Algorithms for Deeply
Distributed Systems

• PIs
• Deborah Estrin (UCLA and USC-ISI)
• John Heidemann (USC-ISI)
• Ramesh Govindan (USC-ISI)
• http//www.isi.edu/scadds
• Technical staff
• Fabio Silva (USC-ISI)
• Students (SCADDS USC-ISI, and UCLA)
• Alberto Cerpa, Jeremy Elson, Deepak Ganesan,
  Lewis Girod, Chalermak Intanagowat, Ya Xu, Yan
  Yu, Jerry Zhao

2
Outline

• Diffusion
• Testbed measurements (Silva, Intanago)
• In network processing
• Nested Queries (Silva, Intanago)
• Aggregation (Intanago)
• Tracking (Ganesan, Work in progress)
• Scaling mechanisms
• GEAR (Yu) and GAF (Xu) routing
• TinyDiffusion (Ganesan)
• Tiered testbed update
• PC-104, UCB Motes with TinyOS, Tags
• MAC (Ye)
• Plans for Q2-3 01

3
Experiments on our PC104 testbed

• Initial experimental measurements of diffusion
  (e.g., for comparison with simulation)
• Compare bytes sent by diffusion with and without
  aggregation (simple in network processing)
• Measurement Setup
• A 5-hop network of 14 nodes on 2 ISI floors
  (testbed is actually 30 nodes and growing)
• Radio 13kbps radiometrix
• 1 sink and 1-4 sources (each source sends 112
  bytes every 6 seconds)

4
Experimental Results

• Bytes sent by diffusion per event vs. Number of
  sources

Diffusion without suppression
Diffusion with suppression
5
Comparison to Simulation

• Bytes sent by diffusion per event vs. Number of
  sources

Diffusion without suppression
Diffusion with suppression
6
Differences between Simulations and Experiments

• MAC differences
• Modified 802.11 for simulations to represent
  hybrid TDMA-Contention
• Radiometrix MAC for experiments
• Channel differences
• No obstacles used in ns-2 simulations
• Note we have added ability to include simple
  terrain but didnt try to replicate indoor exp
  terrain in sims
• More packet losses and collisions in experiments
• Collisions in experiments act as unintentional
  suppression (make no suppression look better than
  it will with better mac)

7
In network processing Nested Queries

• Edge processing overwhelms power and bandwidth
  consumption
• Nested queries where low-energy sensors trigger
  high-energy sensors

8
Experimental Validation Testbed Measurements

• Higher delivery ratio for nested query indicates
  that localizing data traffic benefits
  performance.
• Audio Events Successfully Delivered vs. Number
  of light sensors

9
Reinforced Aggregation

• Promote In-network Data Aggregation near the
  Sources for Better Energy Savings
• Two Approaches for Reinforced Aggregation
• Greedy Tree Approach
• Incremental approach -- Adds minimum number of
  links on the existing tree
• Iterative Approach
• Selects aggregation points such that energy
  dissipation for delivering aggregated data is
  approximately minimized

10
Greedy Tree Approach (incremental)

• Each node enumerates additional cost for
  supplying additional data samples of the same
  data type for previously reinforced path
  (on-tree)
• On-tree nodes dont increment cost for additional
  data samples
• Sink node selects path for particular data
  samples based on cost advertised on the existing
  tree, and that advertised on other (possibly
  shorter) paths
• Advertised cost along existing tree reflects
  sharing
• Each node maintains message cache
  messageenergylast hop

Source 1
Source 2
B
d21B d21A d10B
A
D
d21B d21A d11B
C
d21D d22C d12D
E
Sink
11
Iterative Approach

• Each node advertises cost for each data sample
• Each node also advertises cost for each aggregate
  (multiple data samples belonging to same data
  type)
• Sink reinforces aggregate with minimum
  advertised energy cost
• Each node maintains message cache
  messageenergylast hop

Source 1
Source 2
d21A
B
A
d10B d121B
d21A
D
C
d11B d122D
d22C
E
d12D
Sink
d123D
12
Planned Tracking-based in network processing

• Work in progress on other primitives such as
  tracking (example motivated by Xerox and U Wisc)
• Edge processing
• Node A with detection subscribes to other nodes
  that it (A) believes might see tracked object
  and contribute most to location/tracking
• In network processing
• Node A with detection sends out interest
  containing attributes and function that
  characterizes locations/nodes that might see
  tracked object and contribute most to
  location/tracking

13
Scaling Mechanisms

• Flooding of interests
• Geographic and Energy informed routing of
  interest messages
• Exploiting redundancy
• Geographic Adaptive fidelity applied to topology
  used for flooding interests
• Optimizations for large numbers of listeners
• Pushed data (e.g., needed by Univ Wisc API)
  discussed in Integration meeting (see John H.
  slides)
• Optimizations for much smaller/constrained nodes

14
Geographical and Energy Aware Routing (GEAR)

• Motivation
• Reduce overhead of interest and low rate data
  flooding in directed diffusion
• Basic ideas
• Leverage geographical information to restrict
  flooding, and recursively disseminate data inside
  the target region.
• Extend overall network lifetime using local
  techniques to balance energy usage
• Reuse routing information across multiple user
  queries.
• Extension of GPSR, LAR, other geographic routing

15
GEAR

• Forward the packets towards the target region
• Greedy mode
• minimizing cost function (fmix function of
  distance and energy)
• Route around communication holes with energy
  aware neighbor estimation
• Disseminate the packet within the target region
• Geographic Recursive Forwarding
• recursively re-send packets to sub-regions of the
  original geographic region
• Restricted Flooding
• apply in low density case.

16
Simulation results multiple traffic pairs
packets delivered before network partition vs.
nodes
GEAR GEAR Geo-only
17
Simulation results multiple traffic pairs
connected pairs broken down per received data
packet vs. nodes
GEAR GEAR Geo-only
18
GEAR Plans

• Prototype Implementation on our testbed in
  progress (Yu)
• Planned experimentation w/CSIP support
• Desired data is characterized by geographic
  attributes
• Xerox and U. Wisc as users/collaborators
• Planned addition of data-dissemination-cost
  attribute
• Support CSIP informed decision re. data
  contribution (to task) vs. dissemination cost

19
TinyDiffusion

• Implementation of Diffusion on resource
  constrained UCB motes
• 8bit CPU, 8K program memory, 512 bytes data
  memory
• Subset of full system
• retains only gradients, and condenses attributes
  to a single tag.
• Entire System runs for less than 5.5 KB memory
• TinyOS adds 3.5K and 144 bytes of data. (incl.
  support for Radio and Photo Sensor)
• Diffusion adds 2K code and 110 bytes of data to
  TinyOS.

20
TinyDiffusion Functionality

• Resource Constraints
• Limited cache size currently 10 entries of
  2bytes each
• Limited ability to support multiple traffic
  streams. Currently supports 5 concurrently active
  gradients.
• Tiered Deployment
• PC104s running diffusion interface with mote
  clusters using TinyDiffusion.
• Motes enable dense sensor deployment but can
  support limited in-network processing
• Logical Header format of TinyDiffusion is
  compatible with the Diffusion header.

21
Gateway Architecture
MOTE ATMEL 8586 4MHz MCU 8K program memory 512
Bytes Data Memory RFM Radio 900 MHz
PC104
Mote-NIC
MOTE
PC104 AMD ElanSC400 66MHz CPU 16MB RAM Form
Factor 3.6"  x  3.8"  x  0.6"
Serial
22
Tiered Testbed

• PC-104(linux) with MoteNIC
• Tags, Sensor Card
• UCB Motes w/TinyOS
• Yet to come SmartDust (highly specialized nodes)

PC/104
Tag
UCB Mote
23
Shoebox Testbed v2

• Featuring
• PC-104 w/Pentium 266
• Mote-NIC
• Ethernet fordebugging andmeasurement
• Linux 2.4.2w/glibc 2.1.3
• Plastic
• shoeboxes
• from local drugstore

24
Sensor Card

• The sensor card is a small (2x4)
  microcontroller board with several on-board
  sensors and emitters
• Microphone
• Light sensor
• Accelerometer
• Designed to perform simple sensing tasks at low
  power.
• Currently it is connected to the PC-104 platform
  by serial.
• Data is preprocessed on the sensor board and fed
  back to the PC-104 for analysis and
  communication.
• The next version of the PC-104 platform will have
  the capability to be awakened by a peripheral
  such as the sensor card.

25
Plans Q2-4 01

• Diffusion Experimentation
• larger scale experiments and tuning
• port to WINSng 2.0 platform
• TinyDiffusion experiments and interoperate with
  Diffusion
• In Network Processing
• Develop primitives for tracking
• Implement in network aggregation
• Scaling enhancements
• Geographic/Energy Adaptive Routing in Diffusion 3
• Adaptive fidelity experiments (applied to
  interest flooding)
• Data Push (Univ. Wisc. API)
• Bulk transfer capability (e.g. for mobile code,
  larger sensor data)
• SenseIT experimentation support
• November Demo participation
• Emulation of diffusion over wired networks for
  debugging

26
Related (other funding) Projects

• Active cooperative localization
• for sensor network self configuration when
  no/subset GPS
• ASCENT
• Self-configuring topology for densely deployed
  networks
• Adaptive beacon placement/activation
• For proximity based localization
• Computation primitives and constructs
• Beyond nested queries (w/ Culler)
• Application projects
• Habitat monitoring (Biocomplexity mapping)
• Ecophysiology (w/ Culler, Pister, Rundel)

27
Publications/Submissionshttp//www.isi.edu/scadds

• Mobicom submissions (adaptive fidelity,
  multipath, adaptive beacon placement)
• SOSP submission (naming-based architecture)
• IROS (Robotics) submission (localization)
• ICDCS (address-free, beacon placement)
• IPDPS (time synch)
• Sigcomm submission (self-config topology
  experiments)