Triangulation and Multi-View Geometry Class 9

1
Triangulation and Multi-View GeometryClass 9

• Read notes Section 3.3, 4.3-4.4, 5.1
• (if interested, read Triggss paper on MVG using
  tensor notation, see http//www.unc.edu/courses/20
  04fall/comp/290/089/papers/Triggs-ijcv95.pdf)

2
Automatic computation of F

• Step 1. Extract features
• Step 2. Compute a set of potential matches
• Step 3. do
• Step 3.1 select minimal sample (i.e. 7 matches)
• Step 3.2 compute solution(s) for F
• Step 3.3 determine inliers
• until ?(inliers,samples)lt95

Step 4. Compute F based on all inliers Step 5.
Look for additional matches Step 6. Refine F
based on all correct matches
inliers 90 80 70 60 50
samples 5 13 35 106 382
3
Abort verification early
OIOIIIIOIIIOIOIIIIOOIOIIIIOIOIOIIIIIIII
OOOOOIOOIOOOOOIOOOOOOOIOOOOOIOIOOOOOOOO

• Given n samples and an expected proportion of
  inliers p, how likely is it that I have observed
  less than T inliers?
• abort if Plt0.02 (initial sample most probably
  contained outliers)
• (inspired from Chum and Matas BMVC2002)

(use normal approximation to binomial)
To avoid problems this requires to also verify at
random! (but we already have a random sampler
anyway)
4
Finding more matches
restrict search range to neighborhood of
epipolar line (e.g. ?1.5 pixels) relax
disparity restriction (along epipolar line)
5
Degenerate cases

• Degenerate cases
• Planar scene
• Pure rotation
• No unique solution
• Remaining DOF filled by noise
• Use simpler model (e.g. homography)
• Solution 1 Model selection
• (Torr et al., ICCV98, Kanatani, Akaike)
• Compare H and F according to expected residual
  error (compensate for model complexity)
• Solution 2 RANSAC
• Compare H and F according to inlier count
• (see next slide)

6
RANSAC for (quasi-)degenerate cases
80 in plane 2 out plane 18 outlier

• Full model (8pts, 1D solution)

(accept inliers to solution F)

• Planar model (6pts, 3D solution)
• Accept if large number of remaining inliers

(accept inliers to solution F1,F2F3)

• Planeparallax model (plane2pts)

closest rank-6 of Anx9 for all plane inliers
Sample for out of plane points among outliers
7
More problems

• Absence of sufficient features (no texture)
• Repeated structure ambiguity

• Robust matcher also finds
• support for wrong hypothesis
• solution detect repetition

(Schaffalitzky and Zisserman, BMVC98)
8
RANSAC for ambiguous matching

• Include multiple candidate matches in set of
  potential matches
• Select according to matching probability (
  matching score)
• Helps for repeated structures or scenes with
  similar features as it avoids an early
  commitment, but also useful in general

(Tordoff and Murray ECCV02)
9
two-view geometry

• geometric relations between two views is fully
• described by recovered 3x3 matrix F

10
Triangulation (finally!)
x1
C1
L1
Triangulation

• calibration
• correspondences

11
Triangulation

• Backprojection

• Triangulation

Iterative least-squares

• Maximum Likelihood Triangulation

12
Optimal 3D point in epipolar plane

• Given an epipolar plane, find best 3D point for
  (m1,m2)

Select closest points (m1,m2) on epipolar
lines Obtain 3D point through exact
triangulation Guarantees minimal reprojection
error (given this epipolar plane)
13
Non-iterative optimal solution

• Reconstruct matches in projective frame by
  minimizing the reprojection error
• Non-iterative method
• Determine the epipolar plane for reconstruction
• Reconstruct optimal point from selected epipolar
  plane
• Note only works for two views

3DOF
(Hartley and Sturm, CVIU97)
(polynomial of degree 6)
1DOF
14
Backprojection

• Represent point as intersection of row and column

• Condition for solution?

Useful presentation for deriving and
understanding multiple view geometry (notice 3D
planes are linear in 2D point coordinates)
15
Multi-view geometry
(intersection constraint)
(multi-linearity of determinants)
( epipolar constraint!)
(counting argument 11x2-157)
16
Multi-view geometry
(multi-linearity of determinants)
(3x3x327 coefficients)
( trifocal constraint!)
(counting argument 11x3-1518)
17
Multi-view geometry
(multi-linearity of determinants)
(3x3x3x381 coefficients)
( quadrifocal constraint!)
(counting argument 11x4-1529)
18
Next class rectification and stereo
image I(x,y)
image I(x,y)
Disparity map D(x,y)
(x,y)(xD(x,y),y)