Depth from Structured Light II: Error Analysis

1
Depth from Structured Light IIError Analysis

• Michael H. Rosenthal
• April 19th, 2000

2
Outline

• Review of Structured Light Projection
• Error in Ideal Model
• Lighting and Surface Property Errors
• Stripe Assignment Errors
• Motion Errors
• Structured Light in Practice

3
Structured Light Projection

• Camera and projector are calibrated
• Light stripes are projected onto scene
• Depth for each pixel is measured by intersecting
  pixel ray with stripe plane

4
Binary Stripe Encoding

• Pixels are labeled using binary encoding light
  1, dark 0
• High-order bits encode broad regions, low-order
  bits encode fine location

5
Error in Ideal Model
6
Error in Ideal Model

• Pixels and stripes have finite size
• All visible surfaces within the pixel/stripe
  intersection volume are sampled and lumped
  together

7
Error in Ideal Model

• Size of sampled volume depends upon stripe width,
  pixel width and projector orientation
• Corollary depth resolution is controlled by the
  same factors

8
Error from Lighting and Reflection
9
Lighting and Reflection

• Prior assumptions
• Bright Lambertian surface
• Path from projector to surface to camera
• Real world
• Dark or shiny materials (apples, metal, cloth)
• Things in your way

Specular highlights interfere with structured
light patterns
10
Lighting and Reflection

• Preventive Methods
• Capture images with projector fully on and fully
  off
• Use difference to measure intensity range for
  each pixel
• Threshold difference to find to find trouble spots

Images from structured light projected into a
simple scene
11
Lighting and Reflection

• Minimum intensity image may have bright spots -
  specular highlights
• Maximum intensity image may have dark spots -
  shadows or dark objects

12
Lighting and Reflection

• Difference image shows range of intensity between
  on and off
• Narrow range corresponds to non-ideal behavior
• Threshold to identify trouble spots
• Exclude trouble spots from final depth image

13
Lighting and Reflection

• Other practical solutions
• Lambertian coatings and sprays (talc)
• High dynamic range cameras
• Variable exposure times

14
Error in Stripe Assignment
15
Stripe Assignment Errors

• Stripe encoding requires binary decision for each
  pixel
• What happens if a pixel is misassigned to the
  wrong stripe (i.e. gets a 0 rather than a 1 when
  illuminated)?

Images from structured light projected into a
simple scene
16
Stripe Assignment Errors

• Caused by random noise, low contrast, motion,
  edge uncertainty, etc.
• Magnitude depends upon significance of bit!
• 1st bit yields error N/2, 2nd bit yields error
  N/4, kth bit yields error N/2k

Binary stripe encoding (phase shifted method)
17
Stripe Assignment Errors

• Assume that probability of misassignment is p
• Expected error from kth bit is pN/2k
• Total expected error E pN/2 p N/4 p
  N/2k pN/2logN
• E pN(1- 1/2logN)
• For large N, E pN
• For lt 1 expected error, we must have lt 1 chance
  of misassignment

18
Error from Motion
19
Motion Errors

• What errors can motion cause in a structured
  light system?
• Two components of motion
• Along a pixel ray
• Pixel to pixel

Moving object in a structured light system
20
Pixel to Pixel Motion

• Similar to stripe assignment error - random
  bits are flipped in affected pixels

Moving object in a structured light system
21
Motion on a Pixel Ray

• Motion along a pixel ray causes different stripes
  to be reflected over time
• Final pixel code is composed of bits from
  different positions

Motion along a pixel ray traverses projected
stripes over time
22
Motion on a Pixel Ray

• Magnitude of error depends upon initial position
  and velocity
• Very large errors near edges in broad stripes
  (high order bits)

23
Motion on a Pixel Ray

• Absolute error relative to final object position
  - note the chaotic dependence upon position and
  velocity

24
Structured Light in Practice

• Office of the Future uses it for surface
  measurement
• Augmented Reality Biopsy project has tested it
  for skin surface measurement and for internal
  surgery
• Videos!

25
Conclusions

• Four major classes of error in structured light
• Error intrinsic to ideal system - affects spatial
  resolution
• Lighting and reflection error - some regions of
  the scene will be unmeasurable
• Stripe assignment error - need reliable methods
  for binary categorization
• Motion error - yields large, unpredictable errors
• Despite these problems, structured light is
  useful and widely available