Image Primitives and Correspondence

1
Image Primitives and Correspondence
Stefano Soatto Computer Science
Department University of California at Los Angeles
2
Image Primitives and Correspondence
Given an image point in left image, what is the
(corresponding) point in the right image, which
is the projection of the same 3-D point
3
Matching - Correspondence
Lambertian assumption
Rigid body motion
Correspondence
4
Local Deformation Models

• Translational model
• Affine model
• Transformation of the intensity values and
  occlusions

5
Feature Tracking and Optical Flow

• Translational model

• Small baseline

• RHS approx. by first two terms of Taylor series

• Brightness constancy constraint

6
Aperture Problem

• Normal flow

7
Optical Flow

• Integrate over image patch

• Solve

8
Optical Flow, Feature Tracking
Conceptually
rank(G) 0 blank wall problem rank(G) 1
aperture problem rank(G) 2 enough texture
good feature candidates
In reality choice of threshold is involved
9
Optical Flow

• Previous method - assumption locally constant
  flow

• Alternative regularization techniques (locally
  smooth flow fields,
• integration along contours)

• Qualitative properties of the motion fields

10
Feature Tracking
11
3D Reconstruction - Preview
12
Point Feature Extraction

• Compute eigenvalues of G
• If smallest eigenvalue ? of G is bigger than ? -
  mark pixel as candidate
• feature point

• Alternatively feature quality function (Harris
  Corner Detector)

13
Harris Corner Detector - Example
14
Wide Baseline Matching
15
Region based Similarity Metric

• Sum of squared differences

• Normalize cross-correlation

• Sum of absolute differences

16
Edge Detection
original image
gradient magnitude
Canny edge detector

• Compute image derivatives
• if gradient magnitude gt ? and the value is a
  local maximum along gradient
• direction pixel is an edge candidate

17
Line fitting
Non-max suppressed gradient magnitude

• Edge detection, non-maximum suppression
• (traditionally Hough Transform issues of
  resolution, threshold
• selection and search for peaks in Hough
  space)
• Connected components on edge pixels with
  similar orientation
• - group pixels with common orientation

18
Line Fitting
second moment matrix associated with
each connected component

• Line fitting Lines determined from eigenvalues
  and eigenvectors of A
• Candidate line segments - associated line
  quality

19
Take home messages

• Correspondence is easy/difficult/impossible
  depending on the imaging constraints
• Correspondence and reconstruction are tightly
  coupled problems, can be solved simultaneously
  Jin et al., CVPR 2004
• For most scenes simple descriptors suffice to
  establish a few (50-500) corresponding
  points/lines
• From now on just geometry