Operating System Design for Tiny Networked Sensors

1
Operating System Design for Tiny Networked Sensors

• David Culler
• Computer Science Division
• U.C. Berkeley
• www.cs.berkeley.edu/culler

2
Emerging Microscopic Devices

• CMOS stuff is not just Moores law
• Micro Electical Mechanical Systems (MEMS)
• rich array of sensors are becoming cheap and tiny

• Imagine, all sorts of chips that are connected
  to the physical world and to cyberspace!

with Pister
3
What can you do with them?
Disaster Management

• Embed many distributed devices to monitor and
  interact with physical world
• Network these devices so that they can coordinate
  to perform higher-level tasks.
• gt Requires robust distributed systems of
  hundreds or thousands of devices.

Habitat Monitoring
4
Characteristics of Network Sensors

• Small physical size and low power consumption
• Concurrency-intensive operation
• multiple flows, not wait-command-respond
• Limited Physical Parallelism and Controller
  Hierarchy
• primitive direct-to-device interface
• Asynchronous and synchronous devices
• Diversity in Design and Usage
• application specific, not general purpose
• huge device variation
• gt efficient modularity
• gt migration across HW/SW boundary
• Robust Operation
• numerous, unattended, critical
• gt narrow interfaces

5
Outline

• Motivation for exploring a Tiny OS
• Tiny wireless sensor experimental platform
• Basics of TinyOS with Demo
• Dig deeper into execution model
• Emerging layers of abstraction
• Directions for exploration
• Wrap-up

6
Isun Motherboard Processor Core

• Atmel AVR
• Clock speed 4 MHz
• Memory
• 8 Kbytes of program memory (flash)
• 512 bytes of data RAM
• 512 bytes of EEPROM on chip (write 4 ms/byte)
• 32 8 bit registers
• IO capabilities
• 32 general purpose IO lines
• Some lines also serve more specialized purposes,
  e.g. UART
• 10-bit 8-channel ADC
• Connector interface means that the IO lines serve
  a more specific purposes
• Interrupts
• No interrupt queuing

7
Radio Circuit

• RFM Monolithics TR1000 916 MHz radio
• On/off keying at 10 kbps (max. 19.2 kbps)
• Capable of 115 kbps using amplitude shift keying
• Capable of turning on in 30 us
• Low-power listening by switching on and off on a
  sub-bit granularity
• Processor interface
• RFM CNTRL 0 and 1 switch between transmit,
  receive, and sleep
• Raw, unbuffered access to transmit (RFM TX) and
  receive (RFM RX)
• Requires DC balanced signal an equal number of
  1s and 0s in the signal
• Sampling on reception and modulating on
  transmission done is software
• Too much noise in received signal to use UART for
  sampling
• too little flexibility offered by UART
• Imposes real time constraints on the system
• Power usage
• Transmit current 7 mA
• Receive current 4.5 mA
• Sleep 5 uA

8
Expansion Connector

• Documented hardware interface
• Swap components on either side of the connector
  while preserving investment in sensors or main
  boards
• Sensor interfaces
• 4 lines dedicated to switching components on and
  off
• 7 analog voltage sensing lines
• 2 I2C busses
• SPI
• UART lines
• Debugging aids
• All radio-related signals RX, TX, base band,
  control signals, signal strength
• Programming interfaces
• SPI and reset signals for the main processor and
  the coprocessor
• Ground, Vcc for both analog and digital circuits
• 12 lines reserved for future use

9
Sensor Boards

• Basic Sensor Proto
• Photo resistor PW1 and ADC1
• Thermistor PW2 and ADC2
• Prototyping area
• Vibration Sensor
• Photo resistor PW1 and ADC1
• Thermistor PW4, ADC6
• 2 axis accelerometer ADC2 and 3, PW2
• Digital temperature I2C Bus 1
• 2 axis magnetometer
• Convert magnetic fields into a differential
  output
• Field range -2 to 2 gauss
• Sensitivity 3.2mV/V/gauss
• Resolution 27?gauss at 10Hz
• Bandwidth 5MHz

10
Isun Networked Sensor

• 1 x 1.5 motherboard
• ATMEL 4Mhz, 8bit MCU, 512 bytes RAM, 8K pgm flash
• 900Mhz Radio (RF Monolithics) 10-100 ft. range
• ATMEL network pgming assist
• Radio Signal strength control and sensing
• I2C EPROM (logging)
• Base-station ready
• stackable expansion connector
• all ports, i2c, pwr, clock
• Several sensor boards
• basic protoboard
• tiny weather station (temp,light,hum,press)
• vibrations (acc, temp, ...)
• accelerometers
• magnetometers

11
Basic Power Breakdown

• But what does this mean?
• Lithium Battery runs for 35 hours at peak load
  and years at minimum load!
• three orders of magnitude difference!
• A one byte transmission uses the same energy as
  approx 11000 cycles of computation.
• Idleness is not enough, sleep!

12
A Operating System for Tiny Devices?

• Traditional approaches
• command processing loop (wait request, act,
  respond)
• monolithic event processing
• bring full thread/socket posix regime to platform
• Alternative
• provide framework for concurrency and modularity
• never poll, never block
• interleaving flows, events, energy management
• allow appropriate abstractions to emerge

13
TOS Approach

• Stylized programming model with extensive static
  information
• Program graph of TOS components
• TOS component command/event interface
  behavior
• Rich expression of concurrency
• Events propagate across many components
• Tasks provide internal concurrency
• Regimented storage management
• Current implementation is very simple
• Broad range of alternative execution mechanisms

14
Tiny OS Concepts

• Scheduler Graph of Components
• constrained two-level scheduling model threads
  events
• Component
• Commands,
• Event Handlers
• Frame (storage)
• Tasks (concurrency)
• Constrained Storage Model
• frame per component, shared stack, no heap
• Very lean multithreading
• Efficient Layering

Events
Commands
send_msg(addr, type, data)
power(mode)
init
Messaging Component
Internal State
internal thread
TX_packet(buf)
Power(mode)
TX_packet_done (success)
init
RX_packet_done (buffer)
15
Application Component Graph
Route map
router
sensor appln
application
Active Messages
Radio Packet
Serial Packet
packet
Temp
photo
SW
HW
UART
Radio byte
ADC
byte
Example ad hoc, multi-hop routing of photo
sensor readings
clocks
RFM
bit
16
Storage Breakdown (C Code)
3450 B code 226 B data
17
Empirical Breakdown of Effort

• can take apart time, power, space,
• 50 cycle thread overhead, 10 cycle event overhead

18
TOS Execution Model

• commands request action
• ack/nack at every boundary
• call cmd or post task
• events notify occurrence
• HW intrpt at lowest level
• may signal events
• call cmds
• post tasks
• Tasks provide logical concurrency
• preempted by events
• Migration of HW/SW boundary

data processing
application comp
message-event driven
active message
event-driven packet-pump
crc
event-driven byte-pump
encode/decode
event-driven bit-pump
19
Dynamics of Events and Threads
bit event gt end of byte gt end of packet gt
end of msg send
thread posted to start send next message
bit event filtered at byte layer
radio takes clock events to detect recv
20
Event-Driven Sensor Data
char TOS_EVENT(SENS_OUTPUT_CLOCK_EVENT)() return
TOS_CALL_COMMAND(SENS_GET_DATA)() char
TOS_EVENT(SENS_DATA_READY)(int data)
TOS_CALL_COMMAND(SENS_OUTPUT_OUTPUT)((data gtgt 2)
0x7) return 1

• clock event handler initiates data collection
• sensor signals data ready event
• data event handler calls output command
• common pattern

21
Component Composition
include modules MAIN SENS_OUTPUT INT_TO_LEDS C
LOCK PHOTO MAINMAIN_SUB_INIT
SENS_OUTPUTSENS_OUTPUT_INIT MAINMAIN_SUB_START
SENS_OUTPUTSENS_OUTPUT_START SENS_OUTPUTSENS_OU
TPUT_CLOCK_EVENT CLOCKCLOCK_FIRE_EVENT SENS_OUTPU
TSENS_OUTPUT_SUB_CLOCK_INIT CLOCKCLOCK_INIT SEN
S_OUTPUTSENS_OUTPUT_SUB_OUTPUT_INIT
INT_TO_LEDSINT_TO_LEDS_INIT SENS_OUTPUTSENS_OUTP
UT_OUTPUT_COMPLETE INT_TO_LEDSINT_TO_LEDS_DONE SE
NS_OUTPUTSENS_OUTPUT_OUTPUT INT_TO_LEDSINT_TO_LE
DS_OUTPUT SENS_OUTPUTSENS_DATA_INIT
PHOTOPHOTO_INIT SENS_OUTPUTSENS_GET_DATA
PHOTOPHOTO_GET_DATA SENS_OUTPUTSENS_DATA_READY
PHOTOPHOTO_DATA_READY
INT_TO_LEDS
hardware.h
hardware.h
22
Asynchronous Sensor Interface
TOS_MODULE PHOTO JOINTLY IMPLEMENTED_BY
PHOTO ACCEPTS char PHOTO_INIT(void) char
PHOTO_GET_DATA(void) char PHOTO_PWR(char
mode) SIGNALS char PHOTO_DATA_READY(int
data) USES char SUB_ADC_INIT(void) char
SUB_ADC_GET_DATA(char port) HANDLES char
PHOTO_ADC_DONE(int data)
system/PHOTO.desc
23
Active Messages

• Sending
• Declare buffer storage in a frame
• Request Transmission
• Naming a handler
• Handle Completion signal
• Receiving
• Declare a handler
• Firing a handler
• automatic upon arrival of corresponding message
• behaves like any other event
• Buffer management
• strict ownership exchange
• tx done event gt reuse
• rx must rtn a buffer
• built-in wrapper

TOS_FRAME_BEGIN(INT_TO_RFM_frame) char
pending TOS_Msg msg TOS_FRAME_END(INT_TO_RF
M_frame)
24
Send Message
char TOS_COMMAND(INT_TO_RFM_OUTPUT)(int
val) int_to_led_msg message
(int_to_led_msg)VAR(msg).data if
(!VAR(pending)) message-gtval val if
(TOS_COMMAND(INT_TO_RFM_SUB_SEND_MSG)(TOS_MSG_BCAS
T, AM_MSG(INT_READING), VAR(msg)))
VAR(pending) 1 return 1 return
0
msg buffer
25
Completion Event
char TOS_EVENT(INT_TO_RFM_SUB_MSG_SEND_DONE)(TOS_M
sgPtr sentBuffer) if (VAR(pending)
sentBuffer VAR(data)) VAR(pending)
0 TOS_SIGNAL_EVENT(INT_TO_RFM_COMPLETE)(1)
return 1 return 0

• Underlying message layer notifies all sending
  components of completion event
• may need to resume activity after others
  completion
• provides reference to sent buffer to identify
  action
• Here event propagated as output done

26
Handling Active Messages

• Fired (via dispatch) on arrival
• Gains ownership to buffer passed in msg
• accessed as val.data
• MUST return ownership to a buffer
• If lifetime(buffer) lt lifetime(handler), return
  incoming
• otherwise, return free buffer
• no free buffer gt must punt operation and return
  incoming

TOS_MsgPtr TOS_MSG_EVENT(INT_READING)(TOS_MsgPtr
val) ... return val
27
TinyOS Execution Contexts
Tasks
events
commands
Interrupts
Hardware
28
Typical application use of tasks

• event driven data acquisition
• schedule task to do computational portion

char TOS_EVENT(MAGS_DATA_EVENT)(int data)
struct adc_packet pack (struct
adc_packet)(VAR(msg).data) printf("data_event\
n") VAR(reading) data TOS_POST_TASK(FILTER
_DATA) ...
mags.c
29
Filter Magnetometer Data Task
TOS_TASK(FILTER_DATA) int tmp VAR(first)
VAR(first) - (VAR(first) gtgt 5) VAR(first)
VAR(reading) VAR(second) VAR(second) -
(VAR(second) gtgt 5) VAR(second) VAR(first)
gtgt 5 VAR(diff) VAR(diff)-(VAR(diff) gtgt 5)
tmp VAR(first) - VAR(second) if(tmp lt 0) tmp
-tmp VAR(diff) tmp if((VAR(diff) gtgt 5)
gt 85) TOS_CALL_COMMAND(MAGS_LEDg_on)()
VAR(led_on) 255

• 128 Hz sampling rate
• simple FIR filter
• dynamic software tuning for centering the
  magnetometer signal (1208 bytes)
• digital control of analog, not DSP
• ADC (196 bytes)

30
Tasks in low-level operation

• transmit packet
• send command schedules task to calculate CRC
• task initiated byte-level datapump
• events keep the pump flowing
• receive packet
• receive event schedules task to check CRC
• task signals packet ready if OK
• byte-level tx/rx
• task scheduled to encode/decode each complete
  byte
• must take less time that byte data transfer
• i2c component
• i2c bus has long suspensive operations
• tasks used to create split-phase interface
• events can procede during bus transactions

31
Digging into Communications Stack

• Building up from the RFM bit level
• Bit level abstracts away radio specifics
• Byte level radio component collects individual
  bits into bytes
• Packet Level constructs packets from bytes
• Messaging layer interprets packets as messages
• Data pump paradigm used to connect layers

Messaging
Packet Level
Byte Level
RFM Bit Level
32
RFM RFM.comp
TOS_MODULE RFM ACCEPTS char
RFM_INIT(void) char RFM_TX_MODE(void)
char RFM_TX_BIT(char data) char
RFM_RX_MODE(void) char RFM_PWR(char
mode) char RFM_SET_BIT_RATE(char
level) SIGNALS char
RFM_TX_BIT_EVENT(void) char
RFM_RX_BIT_EVENT(char data)

• Bit level interface to the RFM radio
• Abstracts away bit level timing, RFM specific
  control logic (TX vs. RX modes)
• Signals RX and TX bit events
• RFM_SET_BIT_RATE accepts 3 sampling rates
• 0 50us (2 x bit rate)
• 1 75us (1.5 x bit rate)
• 2 100us (1x bit rate)
• 2x Bit rate used for detecting start symbol, bit
  rate 1.5 x used to transition to middle of bit
  transmission, 1x bit rate used to read and write
  data

33
Data Pump Paradigm

• High level components initiate transfer
  (COMMANDS)
• Lower level components signal when more data can
  be handled (EVENTS)
• Collection of bit events aggregated into single
  byte event
• Collection of byte events collected into single
  packet event

Packet Level
Byte Level
RFM Bit Level
34
Radio Byte Level

• RADIO_BYTE.comp, FOUR_B_RADIO_BYTE.comp,
  SEC_DED_RADIO_BYTE.comp, SEC_DED_RADIO_BYTE_SIGNAL
  .comp
• All have similar interfaces
• Transfer individual bits to the radio
• Fires off TX_READY event when it can accept
  another byte

TOS_MODULE RADIO_BYTE ACCEPTS char
RADIO_BYTE_INIT(void) char
RADIO_BYTE_TX_BYTES(char data) char
RADIO_BYTE_PWR(char mode) SIGNALS
char RADIO_BYTE_RX_BYTE_READY(char data, char
error) char RADIO_BYTE_TX_BYTE_READY(char
success) char RADIO_BYTE_TX_DONE(void)

35
General Radio Byte Operation

• Pipelines transmission transmits single byte
  while encoding next byte
• Trades 1 byte of buffering for additional latency
• Separates high level latencies from low level
  latency requirements
• Encoding Task must complete before byte
  transmission completes
• Decode must complete before next byte arrives

Encode Task
Byte 2
Byte 1
Byte 3
Byte 4
Bit transmission
Byte 1
Byte 2
Byte 3
start
RFM Bits
36
Radio Byte FSM with Tasks (Sending)
0
tx_bit c16 / tx_byte_rdy,tx_done
tx_bytes/POST_TASK
2
1
tx_bit c16
tx_bit c16 / tx_byte_rdy
tx_bytes/POST_TASK
3
4
tx_bit c16
tx_bit c16
State Table 0 idle 1 waiting to send out
first byte 2 sending out byte, no next byte 3
sending out byte, waiting to encode next byte 4
sending out byte, done encoding next byte
Encode Task Executed
tx_bit called
tx_bytes accepted
tx_byte_rdy signaled
tx_done signaled
37
Task Scheduling

• currently simple fifo scheduler
• Bounded number of pending tasks
• When idle, shuts down node except clock
• Uses non-blocking task queue data structure

38
DARPA-esq demo

• UAV drops nodes along road, carries one
• hot-water pipe insulation for package
• Nodes self configure into linear network
• Calibrate magnetometers
• Each detects passing vehicle
• Share filtered sensor data with 5 neighbors
• Each calculates estimated direction velocity
• Share results
• As plane passes by,
• joins network
• upload as much of missing dataset as possible
  from each node when in range
• 7.5 KB of code!

39
Working Across Levels

• Encoding
• Low power listening
• Fair and efficient network access
• Proximity detection
• Security
• Tiny virtual machines

40
Packet Encoding / Layers

• Radio requires rough DC balance
• No more than three ones between zeros
• Manchester encoding
• 4b/6b
• Bit error rate significant and increases with
  distance
• CRC, 3-redundant
• or SECDED with DC-balanced coding
• careful choice of generator matrix
• 8-bits gt 17-bits, no CRC

Radio byte components
Radio packet components
41
Low-Power Listening

• Costs about as much to listen as to xmit, even
  when nothing is received
• Only way to save power is to turn radio off when
  there is nothing to hear.
• Can turn radio on/of in about 1 bit
• 30 ms on every 300 ms
• Can detect transmission at cost of 2 bit times
• Small sub-msg recv sampling
• Application-level synchronization rendezvous to
  determine when to sample

sleep
preamble
message
Xmit Recv
b
Optimal Preamble (2/3 Sxb)1/2
Jason Hill
42
Networking issues in TinyOS context

• Two Problems
• arbitration of access to contended radio channel
• even under low duty cycle, due to correlated
  behavior
• self-organized dynamic multihop routing
• Two approaches
• build it into the lower communication layers
• expose to the application to solve

43
Media Access Control

• Hardware single channel RF radio
• Nodes contend for the same wireless channel
• Traffic is highly correlated
• Periodic nature of sensor networks applications
• detection of common events
• Collision detection mechanism is not available
• Channel capacity 25 packet/s
• High amount of traffic due to
• High node density
• High transmission rate of each node

44
Options

• Application attempts to schedule communication to
  avoid contention
• limit rate
• stagger initiation time
• Communication stack provides adaptation and
  feedback

45
Factors

• Application cannot tell if communication was
  successful without additional msgs
• Radio component is listening (efficiently) by
  default
• natural to introduce CSMA within that component
• Place MAC at RADIO_BYTE level in between
  application layer and RFM radio component
• not just backoff and retry, because listening
  costs
• If channel is very busy, backpressure to
  application to slow transmission rate

revealed subtle jitter bug
(16 bit LFB SR)
46
CSMA Evaluation
Channel Utilization 70
Throughput per node is fair
47
Multihop Bandwidth Management

• Should self-organize into fair, dynamic multihop
  net
• Hidden nodes between each pair of levels
• CSMA is not enough
• Pmsg-to-base drops with each hop
• Investment in packet increases with distance
• need to optimize for low-power fairness!
• RTS/CTS costly (power BW)
• Local rate control to approx. fairness
• Priority to forwarding, adjust own data rate
• Additive increase, multiplicative decrease
• Listen for retransmission as ack
• Simulation ½ of packets get through 4 levels out

48
Example Multihop Adaptive Transmission Control
Max rate 4 samples/sec - rate 4p Channel BW
20 p/s - cannot expect more than 1/3 thru
parent Monitor number of children (n) a(n) 1/n
b ½ p p a(n) on success (echo) p
p b without rate control, success drops ½
per hop
49
Proximity / Location detection

• Signal strength sensing
• Circuit works, falls off cleanly in good
  environment
• Incredibly sensitive to obstructions!
• Error rates a useful proximity metric
• Bit errors vs. packet errors
• signal strength Kalman filter provides good
  position detection

50
Signal Strength
Digital Pot
inc
TX
Proc
dir
50k
sel
51
Near Signal Strength at low-power
52
Thoughts on robust Algorithms

• Active Dynamic Route Determination
• When hear a new route beacon, record parent,
  retransmit from SELF, ignore additional messages
  for epoch
• Radio cell structure very unpredictable
• Builds and maintains good breadth-first forest
• Each node maintains O(1) state
• Fundamental operation is pruning retransmission
• Monotonic variables
• Message signature caches
• Takes energy to retain structure

53
Authentication / Security

• RC-5 shared key crypto in 1.7 kb
• Modified Tesla protocol for confidential
  authenticated base broadcast
• Easy to compromise a node, but hard to get many

54
Application-Specific Virtual Machine

• Small byte-code interpreter component
• Code, static data, stack
• Accepts clock-event capsules
• Other events too
• Hides split-phase operations below interpreter
• HW collection of components defines space of
  applications
• Allows very efficient coding within this space
• Capsules define specific query / logic

55
Larger Challenges

• Programming support for systems of generalized
  state machines
• highly-concurrent, event-driven programming
• language, debugging, verification, composition
• Programming the unstructured aggregate
• SPMD, Data Parallel, Query Processing, Tuples
• Resilient Aggregators
• MAX vs 90
• Understanding how an extreme system is behaving
  and what is its envelope
• adversarial simulation
• Self-configuring, self-correcting systems

56
Expanding the Computing Spectrum

• Open Internet Services

• Internet Services

• Servers
• Workstations
• Personal Computers

57
Common issues at the Extremes

• Concurrency intensive
• data streams and real-time events, not
  command-response
• Communications-centric
• Limited resources (relative to load)
• Huge variation in load
• population usage physical stimuli
• robustness
• Hands-off (no UI)
• Dynamic configuration, discovery
• Self-organized and reactive control
• Similar execution model
• event driven,
• components
• Complimentary roles
• tiny semi-autonomous devices empowered by
  infrastructure
• infrastructure services connected to the real
  world
• Huge space of open problems...