Diapositiva 1

1
(No Transcript)
2
Two-view geometry
Epipolar geometry F-matrix comp. 3D
reconstruction Structure comp.
3
Epipolar geometry
Underlying structure in set of matches for rigid
scenes

1. Computable from corresponding points
2. Simplifies matching
3. Allows to detect wrong matches
4. Related to calibration

4
The projective reconstruction theorem
If a set of point correspondences in two views
determine the fundamental matrix uniquely, then
the scene and cameras may be reconstructed from
these correspondences alone, and any two such
reconstructions from these correspondences are
projectively equivalent
allows reconstruction from pair of uncalibrated
images!
5

• Objective
• Given two uncalibrated images compute
  (PM,PM,XMi)
• (i.e. within similarity of original scene and
  cameras)
• Algorithm
• Compute projective reconstruction (P,P,Xi)
• Compute F from xi?xi
• Compute P,P from F
• Triangulate Xi from xi?xi
• Rectify reconstruction from projective to metric
• Direct method compute H from control points
• Stratified method
• Affine reconstruction compute p8
• Metric reconstruction compute IAC w

6
Image information provided View relations and projective objects 3-space objects reconstruction ambiguity
point correspondences F projective
point correspondences including vanishing points F,H8 p8 affine
Points correspondences and internal camera calibration F,H8 w,w p8 W8 metric
7
Epipolar geometry basic equation
separate known from unknown
(data)
(unknowns)
(linear)
8
the singularity constraint
SVD from linearly computed F matrix (rank 3)
Compute closest rank-2 approximation
9
(No Transcript)
10
the minimum case 7 point correspondences
one parameter family of solutions
but F1lF2 not automatically rank 2
11
the minimum case impose rank 2
(obtain 1 or 3 solutions)
(cubic equation)
Compute possible l as eigenvalues of (only real
solutions are potential solutions)
12
the NOT normalized 8-point algorithm
13
the normalized 8-point algorithm

• Transform image to -1,1x-1,1

Least squares yields good results (Hartley,
PAMI97)
14
algebraic minimization
possible to iteratively minimize algebraic
distance subject to det F0 (see book if
interested)
15
Geometric distance
Gold standard Sampson error Symmetric epipolar
distance
16
Gold standard
Maximum Likelihood Estimation
( least-squares for Gaussian noise)
Initialize normalized 8-point, (P,P) from F,
reconstruct Xi
Parameterize
(overparametrized)
Minimize cost using Levenberg-Marquardt (preferabl
y sparse LM, see book)
17
Gold standard
Alternative, minimal parametrization (with a1)
(note (x,y,1) and (x,y,1) are epipoles)

• problems
• a0

? pick largest of a,b,c,d to fix

• epipole at infinity

? pick largest of x,y,w and of x,y,w
4x3x336 parametrizations!
reparametrize at every iteration, to be sure
18
ZhangLoops approach CVIU01
19
First-order geometric error (Sampson error)
(one eq./point ?JJT scalar)
(problem if some x is located at epipole)
advantage no subsidiary variables required
20
Symmetric epipolar error
21
Some experiments
22
Some experiments
23
Some experiments
24
Some experiments
Residual error
(for all points!)
25
Recommendations

1. Do not use unnormalized algorithms

• Quick and easy to implement 8-point normalized

• Better enforce rank-2 constraint during
  minimization

• Best Maximum Likelihood Estimation (minimal
  parameterization, sparse implementation)

26
Special case
Enforce constraints for optimal results Pure
translation (2dof), Planar motion (6dof),
Calibrated case (5dof)
27
The envelope of epipolar lines
What happens to an epipolar line if there is
noise?
Monte Carlo
n50
n25
n15
n10
28
Other entities?
Lines give no constraint for two view
geometry (but will for three and more
views) Curves and surfaces yield some
constraints related to tangency
29
Automatic computation of F

• Interest points
• Putative correspondences
• RANSAC
• (iv) Non-linear re-estimation of F
• Guided matching
• (repeat (iv) and (v) until stable)

30
Feature points

• Extract feature points to relate images
• Required properties
• Well-defined
• (i.e. neigboring points should all be
  different)
• Stable across views

(i.e. same 3D point should be extracted as
feature for neighboring viewpoints)
31
Feature points
(e.g.HarrisStephens88 ShiTomasi94)
Find points that differ as much as possible from
all neighboring points
homogeneous
edge
corner
M should have large eigenvalues
Feature local maxima (subpixel) of F(?1, ? 2)
32
Feature points

• Select strongest features (e.g. 1000/image)

33
Feature matching

• Evaluate NCC for all features with
• similar coordinates

Keep mutual best matches Still many wrong matches!
34
Feature example
0.96 -0.40 -0.16 -0.39 0.19
-0.05 0.75 -0.47 0.51 0.72
-0.18 -0.39 0.73 0.15 -0.75
-0.27 0.49 0.16 0.79 0.21
0.08 0.50 -0.45 0.28 0.99
Gives satisfying results for small image motions
35
Wide-baseline matching

• Requirement to cope with larger variations
  between images
• Translation, rotation, scaling
• Foreshortening
• Non-diffuse reflections
• Illumination

geometric transformations
photometric changes
36
Wide-baseline matching
(Tuytelaars and Van Gool BMVC 2000)

• Wide baseline matching for two different region
  types

37
RANSAC

• Step 1. Extract features
• Step 2. Compute a set of potential matches
• Step 3. do
• Step 3.1 select minimal sample (i.e. 7 matches)
• Step 3.2 compute solution(s) for F
• Step 3.3 determine inliers
• until ?(inliers,samples)lt95

Step 4. Compute F based on all inliers Step 5.
Look for additional matches Step 6. Refine F
based on all correct matches
inliers 90 80 70 60 50
samples 5 13 35 106 382
38
Finding more matches
restrict search range to neighborhood of
epipolar line (?1.5 pixels) relax disparity
restriction (along epipolar line)
39
Degenerate cases

• Degenerate cases
• Planar scene
• Pure rotation
• No unique solution
• Remaining DOF filled by noise
• Use simpler model (e.g. homography)
• Model selection (Torr et al., ICCV98, Kanatani,
  Akaike)
• Compare H and F according to expected residual
  error (compensate for model complexity)

40
More problems

• Absence of sufficient features (no texture)
• Repeated structure ambiguity

• Robust matcher also finds
• support for wrong hypothesis
• solution detect repetition

(Schaffalitzky and Zisserman, BMVC98)
41
two-view geometry

• geometric relations between two views is fully
• described by recovered 3x3 matrix F

42
Next class image pair rectificationreconstructi
ng points and lines