Initialization of Camera Sensor Networks

1
Initialization of Camera Sensor Networks

• IT 620 Topics in Sensor Networks

2
Camera Sensor Networks

• Wireless network of tetherless imaging sensors
• Directional camera sensors
• Fixed region of coverage
• Applications
• Ad-hoc Surveillance
• Environmental and Habitat monitoring
• Virtual tours
• Tasks
• Object detection, recognition, tracking

3
Camera Initialization

• Extrinsic location, orientation
• Intrinsic focal length, skew, principal point
• Set of neighbors
• Degree of overlap
• Pre-requisite for application tasks
• Localization, requires camera location,
  orientation
• Duty-cycling, requires overlap information
• Tracking, requires locationing overlap
  information
• Geographic Routing

4
Factors Effecting Initialization

• Computation Capability
• Infrastructure Support
• Range Estimation
• Landmarks
• Localization

Cricket Mote
sync
range estimation pulse
5
Technology Trends
Stargate

• Sensors/platforms span a large spectrum
• Enable heterogeneous camera networks

6
Multi-Level Initialization
Camera Location Orientation
Location estimation
Range estimation
Relative Locations
Infrastructure Support
No support
Approximate Initialization
Mini-ITX
Mote
Stargate
Computation Capability
Need multiple initialization techniques for
camera sensor networks with varied
resource-constraints
7
Camera Initialization

• Vision-based techniques not well-suited
• Horn86,Tsai 86, Tsai 87, Zhang 00
• Availability of landmarks
• Abundant computation resources
• High-fidelity images
• Calibrate few cameras
• Assumptions not always true in CSNs
• Ad-hoc
• Resource-constrained
• Landmarks not readily available
• Low-fidelity image
• Several cameras

Need automated initialization techniques, which
are quick and cost-effective
8
Application-driven Camera Initialization

• Accurate Calibration
• Sophisticated computation capability
• Infrastructure support for localization
• Snapshot
• Approximate Calibration
• Limited computation capability
• No or minimal infrastructure support
• ACL
• Goal Automated initialization to support
  application tasks
• Object localization and tracking
• Coverage estimation and duty-cycling

9
Snapshot A Self-Calibration Protocol

• No onboard self-localization hardware
• GPS Extra hardware/cost, Outdoor
• Focal length of camera known
• Parameters to estimate
• Location (x,y,z)
• Orientation (pan, tilt, roll)

10
Snapshot Key Idea
R2
R3
Camera
R1
R4
Location/Orientation unknown
Known locations

• Self-Calibration
• Set of reference points with known locations
• Camera captures picture of reference points
• Calibrate using known locations and image
  projection
• Snapshot is automated

11
Unknown Camera Location Estimation
Camera location
R2 (x2,y2,z2)
Camera lens
?1
?
R2
R1
Camera center at (x,y,z)
R1(x1,y1,z1)
2D
3D

• Given 2 reference points with known locations
• T angle formed at camera location
• Possible camera locations
• 2D points on a chord
• 3D points on a surface

12
Narrowing Possible Camera Locations

• More reference points decrease possible locations
• 3 reference points
• Locations on intersection of three curves
• 4 reference points
• Locations on intersection of six curves
• Further constrains set of solutions
• Yields better solution than 3 reference points
• With more than 4 reference points
• Use combinations to improve results
• Median Filter
• Mean Filter

13
Snapshot Location Estimation
R2 (x2,y2,z2)
Image plane
focal length f
v2
u1
?1
P1
?1
v1
u2
R1(x1,y1,z1)
Lens
Camera center at (x,y,z)
P2

• P1 and P2 Projection on image plane

Quadratic Equation with 3 unknowns
14
Camera Orientation Estimation

• a pan, ß tilt, ? roll
• R rotation matrix

15
Camera Orientation Estimation

• Given a reference point at (xi,yi,zi) and camera
  at (x,y,z)
• Location in camera coordinates (xi,yi,zi)

Di Distance of object from camera
center DpDistance of projection from
camera center (pxi, pzi) projection
coordinates On camera
and

• Given 3 reference points
• Solve for R and estimate orientation angles

16
Snapshot Protocol
Cricket Beacon Nodes Priyantha 00
Cricket calibration device
Snap
R1 x1 y1 z1 px1 py1 pz1 R2 x2 y2 z2 px2
py2 pz2 R3 x3 y3 z3 px3 py3 pz3 R4 x4 y4
z4 px4 py4 pz4 . . .
Field-of- view
Ri Reference points
17
Object Locationing and Tracking
Object
Image plane
Closest point of approach
O (Object location)
f
P1
Center of lens
P2
C1 (x1,y1,z1)
C2 (x2,y2,z2)

• Continuous Localization
• 3D Localization
• Calculate object vectors
• Transform coordinate space
• Calculate Closest Point of Approach

18
Snapshot Implementation
Snapshot Position Sensor
CMUcam Stargate Camera sensor

• Position Sensor Cricket Mote Sensor
• Camera Sensors
• CMUcam (176x255)
• Sony MotionEye Webcam (640x480)
• Connected to Intel Stargates

19
Camera Location Estimation
CMUcam
Webcam
Use of Cricket position sensor for location
estimation introduces very small error 5cm
20
Camera Orientation Estimation
Exact
Cricket

• CMUcam orientation
• Median errors very similar and less than 3
  degrees
• 95th percentile less than 5 degrees

21
Tracking using Self-calibrated Cameras
Webcam CMUcam

• Median localization error in few tens of
  centimeters

22
Snapshot Summary

• Automated process using Cricket introduces very
  small error 5cm
• Localization error in tens of centimeters
• Protocol scalable to tens of cameras in few
  minutes
• Calibration parameters very sensitive to random
  errors in reference points

23
Need for Approximate Initializtion

• Vision-based calibration
• Accurate parameters
• Requires landmarks and lots of computation
• Locationing systems
• Infrastructure and hardware

• Ad-hoc Camera Sensor Networks
• Landmarks hard to find or destroyed
• Resource constrained
• Estimation of accurate parameters not possible

24
Problem Statement

• Given a CSN with,
• Limited resources
• computation, power
• No/minimal infrastructure support
• is it possible to initialize cameras to
  enable applications?
• Proposed solution Approximate Initialization
• Estimate relative relationships between cameras
• Use only picture taking capability and local
  processing of camera

25
Approximate Initialization

• Degree of Overlap
• Fraction of viewing region that overlaps with
  neighboring cameras
• k-overlap fraction of viewing region overlapping
  by k cameras
• Approximates level of sensing redundancy with
  neighboring cameras

26
Approximate Initialization

• Region of Overlap
• Spatial volume within viewing region that
  overlaps with another camera
• Degree of overlap does not estimate which portion
  overlaps with neighbors
• Approximates location of neighbors and spatial
  region of overlap

Approximate estimates can support application
requirements
27
Estimating k-overlap
reference points viewed at camera i
reference points viewed by k cameras

• k-overlap ratio of randomly placed reference
  objects viewed simultaneously by k cameras
• cameras take pictures
• determine if object can be viewed simultaneously
  by other cameras

28
Skewed Distributions
Estimated
Exact
Camera 1
Camera 3
Camera 2

• Fraction of points does not represent fraction of
  overlap
• Points in sparse region actually represent larger
  region
• Error in estimation due to non-uniform
  distribution

29
Handling Skewed Distributions

• Assign area of each polygon as weight to
  corresponding reference point
• Weight in proportion to density of neighbors

Total weight of reference points viewed at
camera i
Total weight of reference points viewed by k
cameras
30
Approximate 3D Voronoi Tessellation

• Accurate 3D tessellation
• Compute intensive
• Approximation
• Discretize volume into cubes
• Calculate closest reference point
• Add volume to closest
• Points in spare regions will have higher weights

31
Determining Region of Overlap

• where the overlap exists between cameras
• region of overlap is the union of cells
  containing all simultaneously visible points

32
Estimating Reference Point Location

• Assumption Size/Dimensions of reference known
• Estimate dr using object size, image size, focal
  length
• Extend idea to 3D setting to estimate location R

33
Application 1 Duty-Cycling

• Operate in ON-OFF cycles
• dduty-cycling parameter (ON fraction)
• Oik k-overlap of camera
• Parameter in proportion to degree of overlap
  (extent of redundant coverage)

34
Application 2 Triggered Wakeup

• Wakeup scenarios
• Object tracking
• Reliable detection
• Region of overlap can determine potential cameras

Object
C1
C2
C3
35
Best Camera Selection

• Determine best camera
• Projection line
• Object along this line
• Reference points within distance threshold
• Extent of overlap determines best camera

36
Experimental Evaluation

• Simulation
• 150 x 150 x 150
• Two scenarios
• 4 cameras
• 12 cameras
• Non-uniform distribution
• Fraction of objects restricted area

37
Experimental Evaluation

• Implementation
• 8 Cyclops camera sensors
• Crossbow Micaz nodes
• 8ft x 6ft x 17ft

38
Weighted Approximation of k-overlap

• Demonstrates non-weighted scheme shortcoming
• Performs 4-6 times worse than weighted

39
Effect of Skew

• Weighted scheme can correct for skew better
• Non-weighted scheme worse by a factor of 6

40
Region of overlap

• Error decreases with reference points
• 22 with 12 pts/camera
• 10 with 37 pts/camera
• Error 10 in region of overlap estimation

41
Applications
Triggered Wakeup
Duty-Cycling

• Duty-cycling
• Weighted scheme outperforms non-weighted
• Triggered wakeup
• 80 positive wakeups with 10 pts/camera with 2
  triggers

42
Implementation Results

• k-overlap estimation error 2-9
• Region of overlap error 1-11
• Approximate techniques feasible in real
  deployments (10 error)

43
(No Transcript)