Networks of Tiny Devices Embedded in the Physical World

1
Networks of Tiny Devices Embedded in the
Physical World

• David Culler
• Computer Science Division
• U.C. Berkeley
• www.cs.berkeley.edu/culler
• Intel Research
• Berkeley

2
Technology Push

• Complete network embedded systems going
  microscopic

Power
3
Application Pull

• Complete NW embedded systems going microscopic
• Huge space of new applications

Monitoring Managing Spaces
Ubiquitous computing
4
Bridging the Technology-Application Gap

• Power-aware, communication-centric node
  architecture
• Tiny Operating System for Range of
  Highly-Constrained Application-specific
  environments
• Network Architecture for vast, self-organized
  collections
• Programming Environments for aggregate
  applications in a noisy world
• Distributed Middleware Services (time, trigger,
  routing, allocation)
• Techniques for Fine-grain distributed control
• Demonstration Applications

5
Critical issues

• Highly constrained devices
• power, storage, bandwidth, energy, visibility
• primitive I/O hierarchy
• Observation and action inherently distributed
• many small nodes coordinate and cooperate on
  overall task
• The structure of the SYSTEM changes
• Devices ARE the infrastructure
• ad hoc, self-organized network of sensors
• Highly dynamic
• passive vigilance most of the time
• concurrency-intensive bursts
• highly correlated behavior
• variation in connectivity over time
• failure is common

6
A de facto platform for EmNets

• Developed a series of wireless sensor devices
• TinyOS concurrency framework
• Messaging Model
• Networking stacks (RF and Serial)
• Multihop routing
• Several Key components
• sensing, logging, data filters, broadcast
• Simulation tools
• DARPA NEST OEP

USC robomote
7
Many Research Groups using it

• UCB
• NEST
• SensorWeb
• Blackout
• Glaser structures
• CBE
• BFD
• BRWC
• UCLA
• USC, ISI
• Rutgers winlab
• Intel
• Bosch
• Crossbow

• U Wash
• Rutgers
• UIUC
• NCSA
• U Virginia
• Ohio State
• UCSD
• Dartmouth
• MIT
• UT Austin, ASU, Iowa
• Accenture
• and many more

8
The MICA architecture
51-Pin I/O Expansion Connector

• Atmel ATMEGA103
• 4 Mhz 8-bit CPU
• 128KB Instruction Memory
• 4KB RAM
• 4 Mbit flash (AT45DB041B)
• SPI interface
• 1-4 uj/bit r/w
• RFM TR1000 radio
• 50 kb/s ASK
• Focused hardware acceleration
• Network programming
• Rich Expansion connector
• i2c, SPI, GIO, 1-wire
• Analog compare interrupts
• TinyOS tool chain
• sub microsecond RF synchronization primitive

8 Analog I/O
8 Programming Lines
Digital I/O
Atmega103 Microcontroller
DS2401 Unique ID
Coprocessor
Transmission Power Control
Hardware Accelerators
SPI Bus
TR 1000 Radio Transceiver
4Mbit External Flash
Power Regulation MAX1678 (3V)
2xAA form factor
9
Rich Sensor board
Microphone
Sounder
Magnetometer
1.25 in
Temperature Sensor
Light Sensor
2.25 in
Accelerometer
10
More Sensors and Actuators

• Motor-Servo board interfaces any combination of
  two motors, servos, and solenoids to a toy car
  platform
• whisker board for obstacle detection
• digital accelerometer (ADXL202) board for crude
  odometry
• GPS Board
• Weatherboard
• light, temp, humidity, barometric pressure,
  occupancy (thermopile)

11
Getting Ready for Outdoors
12
A Operating System for Tiny Devices?

• Traditional approaches
• command processing loop (wait request, act,
  respond)
• monolithic event processing
• bring full thread/socket posix regime to platform
• Alternative
• provide framework for concurrency and modularity
• never poll, never block
• interleaving flows, events, energy management
• gt allow appropriate abstractions to emerge

13
Tiny OS Concepts

• Scheduler Graph of Components
• constrained two-level scheduling model threads
  events
• Component
• Commands,
• Event Handlers
• Frame (storage)
• Tasks (concurrency)
• Constrained Storage Model
• frame per component, shared stack, no heap
• Very lean multithreading
• Efficient Layering

Events
Commands
send_msg(addr, type, data)
power(mode)
init
Messaging Component
Internal State
internal thread
TX_packet(buf)
Power(mode)
TX_packet_done (success)
init
RX_packet_done (buffer)
14
Application Graph of Components
Route map
router
sensor appln
application
Active Messages
Radio Packet
Serial Packet
packet
Temp
photo
SW
Example ad hoc, multi-hop routing of photo
sensor readings
HW
UART
Radio byte
ADC
byte
3450 B code 226 B data
clocks
RFM
bit
Graph of cooperating state machines on shared
stack
15
TOS Execution Model

• commands request action
• ack/nack at every boundary
• call cmd or post task
• events notify occurrence
• HW intrpt at lowest level
• may signal events
• call cmds
• post tasks
• Tasks provide logical concurrency
• preempted by events
• Migration of HW/SW boundary

data processing
application comp
message-event driven
active message
event-driven packet-pump
crc
event-driven byte-pump
encode/decode
event-driven bit-pump
16
Dynamics of Events and Threads
bit event gt end of byte gt end of packet gt
end of msg send
thread posted to start send next message
bit event filtered at byte layer
radio takes clock events to detect recv
17
Maintaining Scheduling Agility

• Need logical concurrency at many levels of the
  graph
• While meeting hard timing constraints
• sample the radio in every bit window
• Retain event-driven structure throughout
  application
• Tasks extend processing outside event window
• All operations are non-blocking
• lock-free scheduling queue

18
Demonstration applications

• 29 Palms
• Cory Hall network
• ½ million packets over 3 weeks
• Surge network and environment display
• 800 node ad hoc network
• CBE
• Glaser Shakes
• Granlibakken retreat watcher
• Robomote
• Group response
• gt continued application focus
• more real and long lived
• more dynamics
• extract architecture and create framework

19
Example TinyOS study

• UAV drops 10 nodes along road,
• hot-water pipe insulation for package
• Nodes self-configure into linear network
• Synchronize (to 1/32 s)
• Calibrate magnetometers
• Each detects passing vehicle
• Share filtered sensor data with 5 neighbors
• Each calculates estimated direction velocity
• Share results
• As plane passes by,
• joins network
• upload as much of missing dataset as possible
  from each node when in range
• 7.5 KB of code!
• While servicing the radio in SW every 50 us!

20
Structural performance due to multi-directional
ground motions (Glaser CalTech)
Mote infrastructure

• .

Mote Layout
14
5  
15
15
13
6  
12
9  
11
8  
Comparison of Results
Wiring for traditional structural
instrumentation truckload of equipment
21
Energy Monitoring/Mgmt System

• 50 nodes on 4th floor
• 5 level ad hoc net
• 30 sec sampling
• 250K samples to database over 6 weeks

22
Energy Monitoring Network Arch
sensor net
control net
802-11
telegraph
PC
PC
modbus
scada term
UCB power monitor net
Browser
23
Meeting Social Network
24
Wealth of Research Challenges

• Large numbers of highly constrained (energy
  capability), connected devices
• able to be casually deployed in infrastructure
  (existing or in design)
• imperfect operation and reliability
• operating in aggregate
• New family of issues across all the layers

application
service
prog / data model
network
mgmt / diag / debug
algorithm / theory
system
architecture
technology
25
Node Communication Architecture
Classic Protocol Processor
Direct Device Control
Hybrid Accelerator
26
Novel Protocol Examples

• Low-power Listening
• Really Tight Application-level Time
  Synchronization
• Localization
• Wake-up
• MACs
• Self-organization

27
Low-Power Listening

• Costs about as much to listen as to xmit, even
  when nothing is received
• Must turn radio off when there is nothing to
  hear.
• Time-slotting and rendezvous cost BW and Latency
• Can turn radio on/of in lt1 bit
• Small sub-msg recv sampling
• Trade small, infrequent tx cost for freq. Rx
  savings

28
Exposing Time Synchronization Up

• Many applications require correlated data
  sampling
• Distributed time sync accuracy bounded by ½ the
  variance in RTT.
• Successful radio transmission requires sub-bit
  synchronization
• Provide accurate timestamping with msg delivery
• Jitter lt 0.1us (propagation) 0.25 us (edge
  capture accuracy) 0.625 us (clock synch)

29
Localization

• Many applications need to derive physical
  placement from observations
• Spatial sampling, proximity, context-awareness
• Radio is another sensor
• Sample baseband to estimate distance
• Need a lot of statistical data
• Calibration and multiple-observations are key
• Acoustic time-of-flight alternative
• Requires good time synchronization

30
Statistical Approach
31
Integrated Architecture

• Chip Area 5 mm2
• AVR core with protocol Accelerators .5 mm2
• 16 Kbytes on-chip ram 4 mm2
• ADC
• 800-1GHz FSK transceiver, -90dBm receive sensy
  .5 mm2
• Expected sleep current 1 uW
• lifetime on a single AA 400 years
• Expected active (processing current)
• Processor _at_ 4 Mhz lt 1 mW
• Radio 1mW power consumption, 100Kbps
• ADC 20 pJ/sample 10 Ksamps/second .2 uW.

32
Networking

• Hands-on Experience with Large Networks of Tiny
  Network sensors
• intense constraints, freedom of abstraction
• Re-explore entire range of networking issues
• encoding, framing, error handling
• media access control, transmission rate control
• discovery, multihop routing
• broadcast, multicast, aggregation
• active network capsule (reprogramming)
• localization, time synchronization
• security, network-wide protection
• density independent wake-up and proximity est.
• Fundamentally new aspects in each

33
The Nodes are the Infrastucture

• Simple Epidemic Algorithm Schema
• if (new mcast) then
• take local action
• retransmit modified request
• Examples Network wakeup, command propagation
• Build spanning tree
• record parent
• Naturally adapts to available connectivity
• Minimal state and protocol overhead
• gt surprising complexity in this simple mechanism

34
Network Discovery Radio Cells
35
Network Discovery
36
Controlled Empirical Study

• Experimental Setup
• 13x13 grid of nodes
• separation 2ft
• flat open surface
• Identical length antennas, pointing vertically
  upwards.
• Fresh batteries on all nodes
• Identical orientation of all nodes
• The region was clean of external noise sources.
• Range of signal strength settings
• Log many runs

37
Example epidemic tree formation
38
Final Tree
39
Power Laws ?

• Most nodes have very small degree (ave .92)
• Some have degree 15 of the population
• Few large clusters account for most of the edges

40
Open Territory gt Many Children

• Example Level 1

41
Open Territory gt Many Children

• Example Level 2 variation in distance

42
Open Territory gt Many Children

• Example Level 3 long links

43
Understanding Connectivity

• 16 transmit power settings
• For each transmit power setting, each node
  transmits 20 packets.
• Receivers log successfully received packets.
• Nodes transmit one after the other in a
  token-ring fashion
• No collisions.
• Define range radius where 75 of enclosed
  nodes receive 75 of packets
• Often good nodes at a distance

probability of reception from center node vs xmit
strength
44
Importance of Asymmetric Links

• Asymmetric Link
• gt65 successful reception in one direction
• lt25 successful reception in the other direction
• 10-25 of links are asymmetric
• Many long links are asymmetric
• in large field it is likely that someone far away
  can hear you
• what does this mean for protocol design?

45
Collisions are primary factor

• Nodes out of range may have overlapping cells
• hidden terminal effect
• Collisions gt these nodes hear neither parent
• become stragglers
• As the tree propagates
• folds back on itself
• rebounds from the edge
• picking up these stragglers.
• Seen in many experiments
• Mathematically complex because behavior is not
  independent beyond singe cell

46
Stragglers

• significant fraction of links point backwards

47
Minimal lessons learned

• Dont think about wireless networks as bunch of
  circles of radius r
• connectivity is a probability distribution
• falls off with distance, but not as simple fading
  law
• shape varies with time and context
• With large, dense arrays the low-probability
  events are common
• Must strike a balance in exploiting structure and
  adapting to observed behavior
• Want simple local rules that have predictable,
  robust global behavior

48
More typical routing for sensor nets

• Current applications dominated by data
  acquisition
• route from many nodes to nearest gateway
• aggregate from many nodes
• routing determined by simple local rules
• Nodes listen to data transfers from neighbors
• carries hop-count info
• monitors link goodness of potential parents
• dynamically selects best node is lesser hop count
• includes hysterisis and continuous rediscovery
• gateways emit null data with 0-hop
• Much to understand about how such algorithms
  manage major change

49
Self-propagating Programs?

• TinyOS components support class of applns.
• Tiny virtual machine adds layer of interpretation
  for specific coordination
• Primitives for sensing and communication
• Small capsules (24 bytes)
• Propagate themselves through network

50
Multihop Bandwidth Management

• Should self-organize into fair, dynamic multihop
  net
• Hidden nodes between each pair of levels
• CSMA is not enough
• Pmsg-to-base drops with each hop
• Investment in packet increases with distance
• need to optimize for low-power fairness!
• RTS/CTS costly (power BW)
• Local rate control to approx. fairness
• Priority to forwarding, adjust own data rate
• Additive increase, multiplicative decrease
• Listen for retransmission as ack

51
Example Multihop Adaptive Transmission Control
Max rate 4 samples/sec - rate 4p Channel BW
20 p/s - cannot expect more than 1/3 thru
parent Monitor number of children (n) a(n) 1/n
b ½ p p a(n) on success (echo) p
p b without rate control, success drops ½
per hop
52
Key Experience

• Really good at building tinyOS subsystems
• non-blocking, split-phase event structures
• Internalized the state of constant change
  paradigm
• ex maintain routing tree by constantly
  rebuilding it
• soft state that is always suspect
• simple one-way protocols
• Operating in the aggregate
• Simple mechanisms to accomplish large goals
• MAC, ATC
• Out of the box on networking abstractions
• Low-power listen, wake-up, statistical sampling,
  weighted aggregation
• Understanding of large scale dynamics

53
Feeding experience back into simulation
54
Rich set of additional challenges

• Efficient and robust security primitives
• Density independent wake-up, aggregation
• sensor gt can use radio in analog mode
• Resilient aggregators
• Programming support for systems of generalized
  state machines
• Programming the unstructured aggregate
• SPMD, Data Parallel, Query Processing, Tuples
• Understanding how an extreme system is behaving
  and what is its envelope
• adversarial simulation
• Self-configuring, self-correcting systems

55
The Law of Miniaturization
Integration
Log R
Mainframe
99
Time

• Each major generation is increasingly smaller,
  more deeply interactive, arrives when previous is
  at its strength
• Vast majority of computing will be small,
  embedded, devices connected to the physical world
• actually the case today, but...
• not connected to us, the web, or each other
  this will change

56
Where to go for more?

• http//www.tinyos.net/tos/
• Jason Hill, Robert Szewczyk, Alec Woo, Seth
  Hollar, David Culler, Kristofer Pister. System
  architecture directions for network sensors.
  ASPLOS 2000.
• David E. Culler, Jason Hill, Philip Buonadonna,
  Robert Szewczyk, and Alec Woo. A Network-Centric
  Approach to Embedded Software for Tiny Devices.
  EMSOFT 2001.