Fast Illuminationinvariant Background Subtraction using Two Views: Error

1
Fast Illumination-invariant Background
Subtraction using Two Views Error Analysis,
Sensor Placement and Applications
Ser-Nam Lim, Anurag Mittal, Larry S. Davis and
Nikos Paragios
Problem Description
Eliminating False Detections
Robustness to Near-BG Object
Under weak perspective
Consider a two-cameras placement
Single-camera background subtraction

• Can be shown that
• ? is the proportion of correct detection, Im
  ?-1 It ,
• ? is the ground plane homography from reference
  to
• second view.
• Homogeneous and background pixel on ground plane
• assumptions not necessary since Im can be
• independently determined using ? and It.

Typical disparity-based background
subtraction faces problem with near-background
objects

• Baseline orthogonal to ground plane.
• Lower camera used as reference.

• Shadows.
• Illumination changes.
• Specularities.

• Our algorithm needs only detect top portion,
  follow by
• Base-finding operations.

Stereo-based background subtraction
Additional Advantages

• Can overcome many of these problems, but
• Slow and
• Inaccurate online matches.

Very fast and stereo matches of background model
can be established offline, much more accurate.
Under perspective

• A. Criminisi, I. Reid, A.Zisserman, Single View
  Metrology, 7th IEEE
• International Conference on Computer Vision,
  Kerkya, Greece,
• September 1999.
• Based on Criminisi et. al., we can show that in
• reference view,
• ?ref is unknown scale factor, h is the height of
  It,
• is the normalized vertical vanishing
  line of the
• ground plane, vref is the vertical vanishing
  point.
• Equation also applies to the second camera,
• equating them can be used to determine Ib.
• Base point in second camera is just ? Ib.

Experiments
Project Goals

• Develop a fast two camera background
• subtraction algorithm that doesnt require
• solving the correspondence problem
• online.
• 2.Analyze advantages of various camera
• configurations with respect to robustness
• of background subtraction

• Dealing with illumination changes using our
  sensor placement.
• Dealing with specularities (day raining scene).
• Dealing with specularities (night scene).
• Near-background object detection.
• Indoor scene (requiring perspective model).

Reducing Missed Detections
Initial detection free of false detections

• And the missed detections form a
• component adjacent to the ground plane.

• We assume objects to be detected
• move on a known ground plane.

For a detected pixel It along each epipolar line
in an initial foreground blob
Fast Illumination-Invariant Multi-Camera Approach

• Compute conjugate pixel It (constrained stereo).
• Determine base point Ib.
• If It Ib thres, increment It and repeat
  step 1.
• Mark It as the lowermost pixel.

Comparisons

• Weak perspective model much simpler, ease of
• implementation.
• When objects close to camera, weak perspective
• model can be violated (e.g., indoor scenes).
• Perspective model, much less stable, sensitive to
• calibration errors.

A clever idea
Extension to objects not moving on ground
possible.

• Yuri A. Ivanov, Aaron F. Bobick and John Liu,
  Fast Lighting
• Independent Background Subtraction, IEEE
  Workshop on
• Visual Surveillance, ICCV'98, Bombay, India,
  January 1998.

Base Point
Background model
Proposition 1 In 3D space, the missed
proportion of a homogeneous object with
negligible front-to- back depth is
independent of object position.
Equivalently, the proportion that is correctly
detected remains constant.

• Established conjugate pixels offline.
• Color dissimilarity measure between conjugate
  pixels.

Robustness to Illumination Changes
What are the problems?

• False and missed detections, caused by
• homogeneous objects.

Geometrically, the algorithm is unaffected by

• Lighting changes.
• Shadows.

Proof Extent of missed detection being the
length of the baseline. Thus,
proportion of missed detections
.
Robustness to Specularities
After morphological operation, two possibilities

• Specularities in a single blob, or
• Specularities in a different blob.

Case 1 - Specularities in the same blob

• Virtual image lies below the ground plane.
• Eliminated by base-finding operations.

Detection Errors
Case 2 Specularities in different blob
Given a conjugate pair (p, p)
False detections,

• Hard to find a good stereo match.
• Lambertian Specular at point of reflection.
• Even if matched, typically causes Im above It.

• p is occluded by a foreground object, and
• p is visible in the reference view.

Missed detections,

• p and p are occluded by a foreground object.