Title: ECE 549CS 543: COMPUTER VISON LECTURE 20
1ECE 549/CS 543 COMPUTER VISON LECTURE
20 PROJECTIVE STRUCTURE FROM MOTION I
From Affine to Euclidean SFM The Projective SFM
Problem The Projective Ambiguity of Projective
SFM Projective SFM from two Pictures Projective
SFM from Multiple Pictures
- Reading Chapters 12 and 13
- A list of potential projects is at
- http//www-cvr.ai.uiuc.edu/ponce/fall04/project
s.pdf - Homework Affine SFM due Th. Nov. 11.
- http//www-cvr.ai.uiuc.edu/ponce/fall04/hw4/hw4
.pdf
2Suppose we observe a scene with m fixed cameras..
u11 u12 u1n v11 v12 v1n um1 um2
umn vm1 vm2 vmn
A1 A2 Am
P1 P2 Pn
3What if we could factorize D? (Tomasi and
Kanade, 1992)
Affine SFM is solved!
Singular Value Decomposition
We can take
4Relative reconstruction error 2.8
Mean reprojection error 2.4pixel
5From uncalibrated to calibrated cameras
Weak-perspective camera
Calibrated camera
Problem what is Q ?
Note Absolute scale cannot be recovered. The
Euclidean shape (defined up to an arbitrary
similitude) is recovered.
6Relative reconstruction error 3.0
Mean reprojection error 2.4pixel
7Reconstruction Results (Tomasi and Kanade, 1992)
Reprinted from Factoring Image Sequences into
Shape and Motion, by C. Tomasi and T. Kanade,
Proc. IEEE Workshop on Visual Motion (1991). ?
1991 IEEE.
8The Projective Structure-from-Motion Problem
Given m perspective images of n fixed points P
we can write
j
2mn equations in 11m3n unknowns
Overconstrained problem, that can be solved using
(non-linear) least squares!
9The Projective Ambiguity of Projective SFM
When the intrinsic and extrinsic parameters are
unknown
and Q is an arbitrary non-singular 4x4 matrix.
Q is a projective transformation.
10Motion estimation from fundamental matrices
Q
Once M and M are known, P can be computed with
LLS.
Facts
b can be found using LLS.
11Relative reconstruction error 1.2
Mean reprojection error 0.8pixel
12Projective Structure from Motion and Factorization
Factorization??
- Algorithm (Sturm and Triggs, 1996)
- Guess the depths
- Factorize D
- Iterate.
Does it converge? (Mahamud and Hebert, 2000)
13Relative reconstruction error 0.2
Mean reprojection error 0.9pixel
14Bundle adjustment (Photogrammetry)
Minimize
with respect to the matrices Mi and the point
positions Pj .
15Relative reconstruction error 0.2
Mean reprojection error 0.8pixel
16From Projective to Euclidean Images
If z , P , R and t are solutions, so are l z
, l P , R and l t .
Absolute scale cannot be recovered! The Euclidean
shape (defined up to an arbitrary similitude) is
the best that can be recovered.
17From uncalibrated to calibrated cameras
Perspective camera
Calibrated camera
Problem what is Q ?
18From uncalibrated to calibrated cameras II
Perspective camera
Calibrated camera
Problem what is Q ?
Example known image center
19Relative reconstruction error 1.2
Mean reprojection error 0.9pixel
20From uncalibrated to calibrated cameras II
Perspective camera
Calibrated camera
Problem what is Q ?
Example known image center
21Relative reconstruction error 1.5
Mean reprojection error 0.9pixel
22(Pollefeys, Koch and Van Gool, 1999)
Reprinted from Self-Calibration and Metric 3D
Reconstruction from Uncalibrated
Image Sequences, by M. Pollefeys, PhD Thesis,
Katholieke Universiteit, Leuven (1999).