An Algorithm for Associating the Features of Two Images / G. L. Scott, H. C. Longuet-Higgins A direct method for stereo correspondence based on singular value decomposition / M. Pilu

1
An Algorithm for Associating the Features of Two
Images / G. L. Scott, H. C. Longuet-Higgins A
direct method for stereo correspondence based on
singular value decomposition / M. Pilu

• CSE 291 Seminar Presentation
• Andrew Cosand
• ECE CVRR

2
Outline

• Correspondence Problem
• Examples
• Discrepancy
• SL-H Solution
• Distance Measure
• Singular Value Decomposition
• Relation to Kernel Trick
• Pilus Contribution

3
Correspondence Problem

• Which features in image A correspond to features
  in image B?

4
Correspondence Problem

• This task is trivial for humans, but difficult
  for computers.

5
Correspondence Problem

• Used for stereo image pairs motion images.
• Feature correspondence should exhibit Similarity,
  Proximity and Exclusivity.
• Complexity is combinatorial with number of
  features to compare.

6
Stereo Imaging

• Trinocular camera captures 3 images, horizontally
  and vertically offset.

7
Stereo Imaging

• Feature correspondence is used to extract depth
  information from stereo images
• Distances between cameras are known.
• Distances between the same feature in different
  images is determined.
• Distance from cameras to actual object can be
  calculated.

8
Motion Tracking

• Corresponding features are tracked through
  sequential images to determine object or camera
  motion.

Compound Motion
Object Motion Only
9
Local vs. Global
10
Discrepancy

• Small scale discrepancy constrains corresponding
  features to be close together.
• Slow object movement, slight camera motion,
  narrow baseline stereo
• Large scale discrepancy allows widely separated
  features.
• Fast object movement, large camera motion, wide
  baseline stereo

11
Ternus
12
Ternus
13
Ternus
14
Achieving Good Global Correspondence

• Requires relationships between points
• The inner product of x,y coordinates yields a
  deficient feature space. (Also location biased)
• Gaussian weighted distance better captures the
  spatial relationships between points (location
  and proximity).
• SLH provides superior sphered (decorrelated)
  relationship.
• Pilu adds similarity relationship.

15
(No Transcript)
16
Scott Longuet-Higgins

• Define a distance metric between features
• Gije(-rij2/2?2)
• Create matrix of relationships for all possible
  feature pairs

G11 Gij
17
SLH Distance Measure

• Gaussian Weighted
• ? scales distance weighting (discrepancy)
• Analytic with respect to distance, coordinates
• Decreases monotonically with distance
• Positive Definite for identical images

18
Positive Definite Matrices

• Comparing identical feature sets yields a
  symmetric positive definite matrix.
• Symmetric gets us real eigenvalues.
• Positive definite has positive eigenvalues (which
  means real square roots).
• G U?UT QQT gt Q U?1/2

Matrix Factors
Real
Inner Product
19
Singular Value Decomposition

• SVD factors a matrix into the product of two
  orthogonal matrices and a diagonal matrix of
  singular values (eigenvalues).
• G TDU, G is m-by-n,
• T is orthogonal, m-by-m
• D is diag(?1, ?2, ?p), m-by-n, pminm,n
• U is orthogonal, n-by-n

20
Scott Longuet-Higgins

• Use Singular Value Decomposition on matrix. G
  TDU

21
Scott Longuet-Higgins

• Set diagonal elements of D to 1, recover
  relationship matrix.
• P TIU TU
• Eliminating singular matrix rescales data in
  feature space, essentially sphereing it.

22
Scott Longuet-Higgins

• Largest feature in row and column indicates
  mutual best match (correspondence)

23
Relation to Kernel Trick

• Gaussian Distance is essentially a kernel
• Relates to a dot product in infinite dimensionial
  space.
• This gives a richer feature space with useful
  relationships between features.
• This is why the SVD works here.

24
Pilus Improvement

• Rogue features dont correspond to anything,
  complicating the process.
• SLH only deals with proximity and exclusivity.
• Similarity constraint can eliminate rogue
  features, which shouldnt be similar to anything.

25
Pilus Improvement

• Modify relationship metric to include gray-level
  correlation.
• Gij (e-(Cij 1)2/2?2) e(-rij2/2?2)
• Gij ((Cij1) /2) e(-rij2/2?2)
• Adds similarity to feature space (kernel
  operation).
• Rogue features can be eliminated because they are
  not similar to anything.

26
Results

• Achieves globally better feature matches rather
  than locally good matches.
• Resistant to rogue points.

27
Summary

• SLH essentially maps input to a rich, high
  dimensional feature space using kernel trick,
  then uses SVD to determine matches.
• Pilu improves kernel to achieve better feature
  space.
• Combination works well.

28
References

• This presentation drew material from the
  following sources
• S. Belonge, Notes on Spectral Correspondence
• M. Pilu, A direct method for stereo
  correspondence based on singular value
  decomposition
• variants
• G. L. Scott, H. C. Longuet-Higgins, An Algorithm
  for Associating the Features of Two Images