Multiple View Geometry

1
Multiple View Geometry

• Marc Pollefeys
• University of North Carolina at Chapel Hill

Modified by Philippos Mordohai
2
Outline

• Computation of camera matrix P
• Introduction to epipolar geometry estimation
• Chapters 6 and 8 of Multiple View Geometry in
  Computer Vision by Hartley and Zisserman

3
Pinhole camera
4
Camera anatomy
Camera center Principal point Principal ray
5
Camera calibration
6
Resectioning
Resectioning correspondence between 3D and image
entities
7
Basic equations
Similar to last weeks H estimation
8
Basic equations
minimal solution
P has 11 dof, 2 independent eq./points

• 5½ correspondences needed (say 6)

Over-determined solution
n ? 6 points
9
Degenerate configurations

• Camera and points on a twisted cubic
• Points lie on plane or single line passing
• through projection center

10
Data normalization
Centroid at origin
Scaling
11
Line correspondences
Extend DLT to lines
(back-project line)
(2 independent eq.)
12
Geometric error
Assume 3D points are more accurately known
13
Gold Standard algorithm

• Objective
• Given n6 3D to 2D point correspondences
  Xi?xi, determine the Maximum Likelihood
  Estimation of P
• Algorithm
• Linear solution
• Normalization
• DLT
• Minimization of geometric error using the
  linear estimate as a starting point minimize the
  geometric error
• Denormalization

14
Calibration example

• Canny edge detection
• Straight line fitting to the detected edges
• Intersecting the lines to obtain the images
  corners
• typically precision lt1/10
• (HZ rule of thumb 5n constraints for n unknowns)

15
Errors in the world
Errors in the image and in the world
16
Restricted camera estimation

• Find best fit that satisfies
• skew s is zero
• pixels are square
• principal point is known
• complete camera matrix K is known

• Minimize geometric error
• impose constraint through parameterization
• Image only ?9 ? ?2n, otherwise ?3n9 ? ?5n

• Minimize algebraic error
• assume map from param q ? PKR-RC, i.e. pg(q)
• minimize Ag(q) (9 instead of 12 parameters)

17
Reduced measurement matrix

• One only has to work with 12x12 matrix, not 2nx12
• Optimization cost is independent of the number of
  correspondences

18
Restricted camera estimation

• Initialization
• Use general DLT
• Clamp values to desired values, e.g. s0, ?x ?y
• Note can sometimes cause big jump in error
• Alternative initialization
• Use general DLT
• Impose soft constraints
• gradually increase weights

19
Exterior orientation (hand-eye coordination)
Calibrated camera, position and orientation
unknown ? Pose estimation 6 dof ? 3 points
minimal (4 solutions in general)
20
Experimental evaluation
Algebraic method minimizes 12 errors instead of
2n396
21
Covariance estimation
ML residual error
d number of parameters
22
Covariance for estimated camera
Compute Jacobian of measured points in terms of
camera parameters at ML solution, then
(variance per parameter can be found on diagonal)
Confidence ellipsoid for camera center
cumulative-1
(chi-square distribution distribution of sum of
squares)
23
(No Transcript)
24
Radial distortion
short and long focal length
25
(No Transcript)
26
(No Transcript)
27
Correction of distortion
Choice of the distortion function and center

• Computing the parameters of the distortion
  function
• Minimize with additional unknowns
• Straighten lines

28
Placing virtual models in video
unmodelled radial distortion
Bundle adjustment needed to avoid drift of
virtual object throughout sequence
modelled radial distortion
(Sony TRV900 miniDV)
29
Some typical calibration algorithms
Tsai calibration
Zhang calibration
http//research.microsoft.com/zhang/calib/
Z. Zhang. A flexible new technique for camera
calibration. IEEE Transactions on Pattern
Analysis and Machine Intelligence,
22(11)1330-1334, 2000. Z. Zhang. Flexible Camera
Calibration By Viewing a Plane From Unknown
Orientations. International Conference on
Computer Vision (ICCV'99), Corfu, Greece, pages
666-673, September 1999.
30
Related Topics

• Calibration from vanishing points
• Calibration from the absolute conic

31
Outline

• Computation of camera matrix P
• Introduction to epipolar geometry estimation

32
Three questions

• Correspondence geometry Given an image point x
  in the first image, how does this constrain the
  position of the corresponding point x in the
  second image?

• (ii) Camera geometry (motion) Given a set of
  corresponding image points xi ?xi, i1,,n,
  what are the cameras P and P for the two views?

• (iii) Scene geometry (structure) Given
  corresponding image points xi ?xi and cameras
  P, P, what is the position of (their pre-image)
  X in space?

33
The epipolar geometry
C,C,x,x and X are coplanar
34
The epipolar geometry
What if only C,C,x are known?
35
The epipolar geometry
All points on p project on l and l
36
The epipolar geometry
Family of planes p and lines l and l
Intersection in e and e
37
The epipolar geometry
epipoles e,e intersection of baseline with
image plane projection of projection center in
other image vanishing point of camera motion
direction
an epipolar plane plane containing baseline
(1-D family)
an epipolar line intersection of epipolar plane
with image (always come in corresponding pairs)
38
Example converging cameras
39
Example motion parallel with image plane
(simple for stereo ? rectification)
40
Example forward motion
e
e