Multiple View Geometry

1
Multiple View Geometry

• Marc Pollefeys
• COMP 256

2
Last class
Gaussian pyramid
Laplacian pyramid
Gabor filters
Fourier transform
Texture synthesis
3
Not last class
4
Shape-from-texture
5
Tentative class schedule
Aug 26/28 - Introduction
Sep 2/4 Cameras Radiometry
Sep 9/11 Sources Shadows Color
Sep 16/18 Linear filters edges (Isabel hurricane)
Sep 23/25 Pyramids Texture Multi-View Geometry
Sep30/Oct2 Stereo Project proposals
Oct 7/9 - Optical flow
Oct 14/16 Tracking -
Oct 21/23 Silhouettes/carving Structure from motion
Oct 28/30 - Camera calibration
Nov 4/6 Project update Segmentation
Nov 11/13 Fitting Probabilistic segm.fit.
Nov 18/20 Matching templates Matching relations
Nov 25/27 Range data (Thanksgiving)
Dec 2/4 Final project Final project
6
THE GEOMETRY OF MULTIPLE VIEWS

• Epipolar Geometry
• The Essential Matrix
• The Fundamental Matrix
• The Trifocal Tensor
• The Quadrifocal Tensor

Reading Chapter 10.
7
Epipolar Geometry

• Epipolar Plane

• Baseline

• Epipoles

• Epipolar Lines

8
Epipolar Constraint

• Potential matches for p have to lie on the
  corresponding
• epipolar line l.

• Potential matches for p have to lie on the
  corresponding
• epipolar line l.

9
Epipolar Constraint Calibrated Case
Essential Matrix (Longuet-Higgins, 1981)
10
Properties of the Essential Matrix
T

• E p is the epipolar line associated with p.
• ETp is the epipolar line associated with p.
• E e0 and ETe0.
• E is singular.
• E has two equal non-zero singular values
• (Huang and Faugeras, 1989).

T
11
Epipolar Constraint Small Motions
To First-Order
Pure translation Focus of Expansion
12
Epipolar Constraint Uncalibrated Case
Fundamental Matrix (Faugeras and Luong, 1992)
13
Properties of the Fundamental Matrix

• F p is the epipolar line associated with p.
• FT p is the epipolar line associated with p.
• F e0 and FT e0.
• F is singular.

T
T
14
The Eight-Point Algorithm (Longuet-Higgins, 1981)
15
Non-Linear Least-Squares Approach (Luong et al.,
1993)
Minimize
with respect to the coefficients of F , using an
appropriate rank-2 parameterization.
16
Problem with eight-point algorithm
linear least-squares unit norm vector F
yielding smallest residual What happens when
there is noise?
17
The Normalized Eight-Point Algorithm (Hartley,
1995)

• Center the image data at the origin, and scale
  it so the
• mean squared distance between the origin and the
  data
• points is 2 pixels q T p , q T p.
• Use the eight-point algorithm to compute F from
  the
• points q and q .
• Enforce the rank-2 constraint.
• Output T F T.

i
i
i
i
i
i
T
18
Epipolar geometry example
19
Example converging cameras
courtesy of Andrew Zisserman
20
Example motion parallel with image plane
(simple for stereo ? rectification)
courtesy of Andrew Zisserman
21
Example forward motion
e
e
courtesy of Andrew Zisserman
22
Fundamental matrix for pure translation
auto-epipolar
courtesy of Andrew Zisserman
23
Fundamental matrix for pure translation
courtesy of Andrew Zisserman
24
Trinocular Epipolar Constraints
These constraints are not independent!
25
Trinocular Epipolar Constraints Transfer
Given p and p , p can be computed as the
solution of linear equations.
1
2
3
26
Trinocular Epipolar Constraints Transfer

• problem for epipolar transfer in trifocal plane!

There must be more to trifocal geometry
image from Hartley and Zisserman
27
Trifocal Constraints
28
Trifocal Constraints
Calibrated Case
All 3x3 minors must be zero!
Trifocal Tensor
29
Trifocal Constraints
Uncalibrated Case
Trifocal Tensor
30
Trifocal Constraints 3 Points
Pick any two lines l and l through p and p .
2
3
2
3
Do it again.
31
Properties of the Trifocal Tensor
T
i

• For any matching epipolar lines, l G l
  0.
• The matrices G are singular.
• They satisfy 8 independent constraints in the
• uncalibrated case (Faugeras and Mourrain, 1995).

2
1
3
i
1
Estimating the Trifocal Tensor

• Ignore the non-linear constraints and use linear
  least-squares
• Impose the constraints a posteriori.

32
T
i
For any matching epipolar lines, l G l
0.
2
1
3
The backprojections of the two lines do not
define a line!
33
Trifocal Tensor Example
108 putative matches
18 outliers
(26 samples)
95 final inliers
88 inliers
(0.19)
(0.43) (0.23)
courtesy of Andrew Zisserman
34
Trifocal Tensor Example
additional line matches
images courtesy of Andrew Zisserman
35
Transfer trifocal transfer
(using tensor notation)
doesnt work if lepipolar line
image courtesy of Hartley and Zisserman
36
Image warping using T(1,2,N)
(Avidan and Shashua 97)
37
Multiple Views (Faugeras and Mourrain, 1995)
38
Two Views
Epipolar Constraint
39
Three Views
Trifocal Constraint
40
Four Views
Quadrifocal Constraint (Triggs, 1995)
41
Geometrically, the four rays must intersect in P..
42
Quadrifocal Tensor and Lines
43
Quadrifocal tensor

• determinant is multilinear
• thus linear in coefficients of lines
  !
• There must exist a tensor with 81 coefficients
  containing all possible combination of x,y,w
  coefficients for all 4 images the quadrifocal
  tensor

44
Scale-Restraint Condition from Photogrammetry
45
Next classStereo
image I(x,y)
image I(x,y)
Disparity map D(x,y)
(x,y)(xD(x,y),y)
FP Chapter 11