Calibration Issues

1
Calibration Issues

• Linear Models
• Homography estimation H
• Epipolar geometry F, E
• Interior camera parameters K
• Exterior camera parameters R,t
• Camera pose R,t
• Interest Point Detection Description
• Algorithms
• Overdetermined systems of linear equations ?
  Error Minimization
• Direct Linear Transform DLT
• Normalization
• Nonlinearities ? iterative error minimization,
  Levenberg-Marquardt
• Outliers ? Robustness, RANSAC

2
Pinhole Camera
Z
X
p(x,y,1)
Ycam
R,t
Y
principal point (x0,y0)
p(x,y)
Xcam
Zcam
optical axis
Ccam
f
y
P(Xcam,Ycam,Zcam)
P(Xcam,Ycam,Zcam)
x
P(X,Y,Z,1)
P(X,Y,Z,1)
image plane pi (x,y) Zcam -f
focal length f

• real camera

• Pinhole C center of projection

• interior camera parameters
• x0, y0, f,

• 2D projection ? 3D scene

• exterior parameters
• camera pose
• R, t

• p(x,y) ? line of sight viewing direction

3
Pinhole Camera
Z
X
R,t
Ycam
Y
principal point (x0,y0)
p(x,y,1)
Xcam
Zcam
optical axis
Ccam
f
y
P(X,Y,Z,1)
x
P(X,Y,Z,1)
image plane pi (x,y) Zcam -f
P 3 x 4 matrix camera projection
matrix Pollefeys p.24, eq. (3.8)
4
The Basic Pinhole Model
inhomog. coord.
homog. coord.

• ? Note Figures taken from, notation following
  Hartley,Zisserman

5
The Basic Pinhole Model
6
Principal Point Offset
7
Principal Point Offset
camera calibration matrix interior/internal
parameters interior/internal orientation
8
Camera Rotation and Translation
3 x 4 P2
P3
4 x 4 P3
P3
9
Camera Rotation and Translation
3 x 4 projection matrix P 9 degrees of freedom
3 internal parameters in K 3 rotation angles in
R 3 translations in C

10
Camera Rotation and Translation
Simplified notation avoid explicit modeling of C
11
From Pinhole ? Real Cameras K
3 x R, 3 x t

• Pinhole
• 3 parameters in K
• CCD
• 4 parameters
• Finite projective camera
• 5 parameters
• skew s

9
10
my
mx
11
12
Projective Camera
But We model real cameras as finite projective
cameras ( lens distortion)

• Finite projective camera
• K is an upper triangular matrix
• KR is non-singular
• General projective camera
• P is an arbitrary 3 x 4 matrix of rank 3
• P has also 11 degrees of freedom

13
Camera Calibration in Practice (1)

• Take
• 1 picture of a 3D calibration target,
• or several pictures of a planar calibration
  target
• (take care so that all parameters can be
  recovered !)
• Establish point correspondences
• Calculate P
• set of linear equations
• Decompose P

14
3D Targets
Hartley Zisserman
Heikkilä

• Many ways to build
• Corners vs. circles (center of gravity)
• Precision of building, attaching,
• CNC measured points
• EMT coordinate measurement machine

Photogrammetry Godding / Jähne
15
2D vs. 3D Targets
f 28mm, z 300mm
f 50mm, z 470mm
f 84mm, z 720mm
16
2D Targets
f 28mm, z 280mm
f 50mm, z 470mm
f 84mm, z 720mm

• arbitrary scaling !
• z/f const.
• closeup of toy car vs. real car at a distance
• but subtle differences in image quality !

17
Image Quality (1)
f 28mm, z 280mm
f 50mm, z 470mm
? lens distortion !
18
Image Quality (2)
f 28mm, z 280mm
f 50mm, z 470mm
? chromatic aberration
19
Lens Distortion Model

• Several ways to model
• Most common
• Radial lens distortion ki
• Tangential lens distortion tj
• Radial gtgt tangential
• Polynomial approximation up to varying order

r
(x0,y0)
y
x
20
Camera Calibration in Practice (2)

• Take
• 1 picture of a 3D calibration target,
• or several pictures of a planar calibration
  target
• Establish point correspondences
• Calculate P
• set of linear equations
• Decompose P

A first estimate for linear Interior parameters
(K)

• Add nonlinear relationships (model ki, tj)
• Perform iterative optimization (w.r.t. some
  error)
• Enforce constraints (such as structure of K and
  R)

21
More Matrices

• Homography H
• Projection P (K, R, t )
• Multiple views
• Epipolar geometry
• Uncalibrated stereo Fundamental matrix F
• Calibrated stereo Essential matrix E
• Stereo rig
• Camera motion ? many views ? AR tracking

22
Epipolar Geometry (1)

• Figures from Hartley Zisserman
• C, C, x, x, X are co-planar (lie in the
  epipolar plane p)

23
Epipolar Geometry (2)

• Assume that only C, C, and x are known

24
Epipolar Geometry (3)
C
C

• p projects on epipolar lines l and l
• baseline connects C, C
• epipoles e, e

25
Epipolar Geometry (4)
C
C

• When 3D position of X varies, p rotates about
  the baseline
• Family of planes epipolar pencil
  Ebenenbüschel

26
Epipolar Geometry Example 1Converging Cameras
HartleyZisserman
27
Epipolar Geometry Example 2Forward
Translation HartleyZisserman, Pollefeys
e
e
28
The Fundamental Matrix F (1)
We had an example Homography H
29
The Fundamental Matrix F (2)
skew-symmetric matrix

• Transfer xi via Xi in p to xi
• 2D homography Hp maps each xi to xi

30
The Fundamental Matrix F (3)

• F relates x in one image with its corresponding
  epipolar line l in the other image (all X in R3
  !)
• The corresponding point x must lie on l
• This relates to
• How to estimate F?

? Point correspondences
31
Calibration Issues

• Linear Models
• Homography estimation H
• Epipolar geometry F, E
• Interior camera parameters K
• Exterior camera parameters R,t
• Camera pose R,t
• Interest Point Detection Description
• Algorithms
• Overdetermined systems of linear equations ?
  Error Minimization
• Direct Linear Transform DLT
• Normalization
• Nonlinearities ? iterative error minimization,
  Levenberg-Marquardt
• Outliers ? Robustness, RANSAC

32
A final word on E

• Essential matrix E
• Similar to F
• Relates calibrated stereo rig
• Internal matrices K and K are known

R, t
normalized coordinates