SensIT Collaborative Signal Processing Candidate Tracking Benchmarks v0'5

1
SensIT Collaborative Signal ProcessingCandidate
Tracking Benchmarksv0.5

• Jim Reich, Xerox PARC
• February 22, 2001

2
Current State of the Document

• Currently, sensor fields, trajectories shown are
  examples, do not reflect new candidate node
  types, scenario parameters, etc.
• Drawings not to scale
• Parameter values are not yet filled in
• Dynamics do not include gear shifting
• Operational scenarios may use different querying
  and may compose these scenarios in interesting
  ways
• Not all benchmarks are expected to be demoable
  during the current program
• Some benchmarks may only be testable in
  simulation due to vehicle/node availability
• Some may be too difficult

3
Assumptions

• Single vehicle unless otherwise shown
• Unless otherwise stated, vehicles attempt to
  maintain constant speed and do not shift gears
• Unless otherwise stated, vehicles start beyond
  the sensor field and arrive in sequence
• Complications to be handled as we get better
• Variation of acoustic signature with aspect
• Acceleration, braking, and gear shifts
• Changes the acoustic signals
• Spatio-temporally varying propagation models
• Node unreliability

4
Generic Scenario Parameters

• Field Geometry
• Field Size (1D length (m), 2D length (m) x
  width (m)
• Node Density (1D Nodes/m, 2D Nodes/m2)
• Node mix ( of each type)
• Number, type of vehicles
• Vehicle trajectories
• Vehicle initial states, maneuvers
• Query
• Query source location
• Desired output update rate (Hz) Latency (sec)
• Desired position accuracy, precision, resolution
  (m)
• Network Performance Latency (ms), packet loss ,
  aggregate throughput, neighbor-to-neighbor link
  throughput (bps)
• Node reliability ( sample loss)
• Node energy storage (J)
• Node energy usage (J/FLOP, J/bit_at_X m)
• RF Fading exponent
• Vehicle trajectory knowledge (road, road network,
  off-road)

5
Vehicles Dynamics Assumptions
6
Sensor Assumptions

• Sound
• Exponent, isotropy
• Target Aspect dependence
• Noise model
• Seismic
• Exponent, isotropy, speed of sound variation
• Noise model
• PIR
• Wedge angle, max. target distance
• Target aspect dependence
• Velocity measurement (speed, or just /-?)
• Noise model
• Magnetometer
• Measurement range (m)
• Target aspect dependence
• Velocity measurement?
• Noise model

7
Benchmark Node Types

• Omni Microphone Cluster
• 4 Omnidirectional Microphones (tetra)
• Directional Microphone Cluster
• 4 Fixed Dipoles at 90 to each other
• PIR Cluster
• 4 PIR Sensors at 90 to each other
• Mix A
• 2 Omnidirectional Microphones
• 1 PIR Sensor
• 1 Vertical Seismic
• Mix B
• 1 Omnidirectional Microphone
• 3 PIR sensors at 120 to each other
• Mix C
• 2 Omnidirectional Microphones
• 1 Vertical Seismic
• 1 Magnetometer
• Imager Node (imager only)

M
M
IR
A
B
C
Im
8
1 Track Single Target

• Task
• Estimate target position vs. time

• Challenges
• Localize target
• Maintain accurate estimate in large gaps between
  sparse sensors
• Fuse data from multiple sensor types

9
2 Track Single Maneuvering Target

• Task
• Estimate target position vs. time

• Challenges
• No road, hence no prior knowledge of vehicle
  trajectory
• Constant direction dynamics models no longer
  adequate
• Many sensors making simultaneous observations

10
3 Track Accelerating/Decelerating Target
A

• Task
• Estimate target position vs. time

• Challenges
• Vehicle signature time-varying
• Constant velocity dynamics models no longer
  adequate
• Gear shift requires maintaining internal discrete
  state (curr. gear)

B
Vehicle begins stationary and idling at point
A Accelerates, maintains constant
velocity Decelerates and stops and point
B Extra credit Handle gear shifts
11
4 Count Stationary (idling) Targets

• Task
• Count number of targets
• Locate targets

• Challenges
• Multiple vehicles
• Unknown number of vehicles
• Cannot depend on peak-finding (CPA) of acoustic
  signal

• Task-Specific Benchmarks
• Accuracy of count

12
4 Count Stationary (idling) Targets

• Special Parameters
• Dynamic range of acoustic outputs for ensemble of
  vehicles (dB SPL)

13
5 Two-way traffic

• Task
• Track target positions
• Estimate target crossing time

• Challenges
• Vehicles in close proximity, need to use dynamics
  to keep identities separate

• Task-Specific Benchmarks
• Accuracy of crossing time estimate
• Accuracy of identity tracking

14
5 Two-way traffic

• Special Parameters
• Desired accuracy of crossing time estimate

15
6 Convoy on a Road

• Task
• Count number of vehicles of each type
• Determine order of vehicles

• Challenges
• Multiple vehicles
• Classification and state information must follow
  vehicle along full length of road

• Task-Specific Benchmarks
• Accuracy of count order
• vs. vehicle spacing convoy velocity

Vary inter-vehicle spacing to vary problem
difficulty
16
6 Convoy on a Road

• Special Parameters
• Average Inter-vehicle spacing (m)

17
7 Perimeter Violation Sensing

• Task
• Alert on violation of perimeter
• Ignore activity outside of perimeter
  (distractors)
• Identify intruder type and track location

• Challenges
• Filter out distractor
• Respond quickly while minimizing quiescent
  activity

• Task-Specific Benchmarks
• Detection delay
• Power usage during periods of no violation
• Frequency of false positives

18
7 Perimeter Violation Sensing

• Special Parameters
• Desired notification latency for violation
• Distractor-to-Intruder SPL ratio
• Closest approach, farthest distance of distractor
  from perimeter (m)
• Time elapsed between distractor arrival and
  perimeter violation
• Min. angular separation (from centroid of node
  array) of intruder and distractor at time of
  violation

19
8 Tracking in an Obstacle Field

• Task
• Track vehicle position

• Challenges
• Obstacles cause individual sensors to lose lock
  on the target
• Different sensing modalities are blocked
  differently by obstacles (blocks acoustic
  partially, PIR completely, seismic unaffected)

20
8 Tracking in an Obstacle Field

• Special Parameters
• Obstacle Configuration
• Obstacle Density
• Obstacle Opacity to sensing modalities

21
9 Identity Tracking Mutual Exclusion using
One-Time sensor

• Task
• Track targets
• Maintain target identities
• Re-establish identity of both targets when
  right-hand magnetometer is crossed

• Challenges
• Need to conserve number and type of targets as
  they pass through tunnel.
• Need to reason about targets Seeing blue at top
  right mag. guarantees red at bottom.

• Task-Specific Benchmarks
• Time to propagate data from RHS magnetometer to
  red car in the lower RHS

Targets can only be distinguished from each other
by magnetometers (shown), which give one-time
red/blue output when the vehicle passes over
them.
22
9 Identity Tracking Mutual Exclusion using
One-Time sensor

• Special Parameters
• Desired time for propagation of exclusion
  constraint after magnetometer sees a vehicle
  after split

23
10. Cluster Behavior

• Task
• Track cluster centroid
• Keep count of vehicles in cluster adjusting as
  some leave and join

• Challenges
• Large number of targets
• Coalescing many similar targets, limiting
  exponential hypothesis blowup
• Measuring global properties of cluster (centroid,
  count) rather than properties of single target

• Task-Specific Benchmarks
• Maximum number of targets which can be handled
  simultaneously
• Centroid accuracy
• Miscount

24
10 Cluster Behavior

• Special Parameters
• Cluster density (vehicles/m2)
• Vehicle arrival rate (vehicles/s)
• Vehicle departure rate (vehicles/s)

25
11. Am I Surrounded?

• Task
• Vehicle move to surround
• At any moment, given the spheres of influence of
  each vehicle, is the query source completely
  encircled?

• Challenges
• Large number of targets
• Dynamically calculating global predicate over
  entire field

• Task-Specific Benchmarks
• Latency between completing/breaking encirclement
  and notification of query source

26
11 Am I Surrounded?

• Special Parameters
• Maximum latency for detection of
  encirclement/breaking of encirclement

27
Benchmarking

• Generic Benchmarks
• Energy Consumption
• Total, per node (max, avg)
• f(computation, communication)
• Detection Accuracy
• Frequency of false positives, negatives
• Detection Latency
• Mean, max vs. query source location
• Tracking accuracy
• Mean, max, std.
• f(desired output frequency)
• Tracking Latency
• Mean, max vs. query source location
• Task-specific Benchmarks (see scenarios)

28
Operational/Benchmark Scenario Mapping Matrix