Calibration

1
Calibration

• Dorit Moshe

2
In todays show

• How positions in the image relate to 3D positions
  in the world?
• We will use analytical geometry to quantify more
  precisely the relationship between a camera, the
  objects it observes, and the pictures of these
  objects
• We start by briefly recalling elementary notions
  of analytical Euclidean geometry.
• We then introduce the various physical
  parameters that relate the world and camera
  coordinate frames, and present as an application
  various methods for estimating these parameters,
  a process known as geometric camera calibration.
• We also present along the way a linear
  least-squares technique for parameter estimation

3
Motivation

• How positions in the image relate to 3D positions
  in the world?
• The reconstruction of 3D image is not trivial. We
  have to reconstruct the third coordinate!

4
Example

• Rabbit or Man?

• The information lays within the 3rd coordinate
• Markus Raetz, Metamorphose II, 1991-92

5

• 2D projections are not the same as the real
  object as we usually see everyday!

6

• But

7
Introduction

• Camera calibration estimation of the unknown
  values in a camera model.
• Intrinsic parameters - Link the frame coordinates
  of an image point with its corresponding camera
  coordinates
• Extrinsic parameters - define the location and
  orientation of the camera coordinate system with
  respect to the world coordinate system

8
Euclidean Geometry - reminder

• Orthonormal coordinate frame (F) is defined by
• a point O in E3 and three unit vectors i, j and
  k orthogonal to each other

9
Transformations

• FP - the coordinate vector of the point P in the
  frame F
• Lets consider two frames A and B
• (A) (OA, iA, jA, kA)
• (B) (OB, iB, jB, kB)
• How can we express BP as a function of AP?
• Let us suppose that the basis vectors of both
  coordinate systems are parallel to each other,
  i.e., iA iB, jA jB and kA kB, but the
  origins OA and OB are distinct
• We say that two coordinate systems are separated
  by a pure translation,

10
Pure translation

• BP AP BOA

11
Pure rotation

• When the origins of the two frames coincide,
  i.e.,OA OB O, we say that the frames are
  separated by a pure rotation.

12

• We define the rotation matrix

13

• It means that for pure rotation

BP
BP
14

• The inverse of a rotation matrix is equal to its
  transpose
• Its determinant is equal to 1 the transform
  preserves the volume.
• Not every transformation that preserves the
  volume keeps the sign. For example - reflection
• This ortho normal transform preserves length and
  angles.

15
Example (Pure rotation)

• kAkBk
• The vector iB is obtained by applying to the
  vector iA a counterclockwise rotation of angle ?
  about k.

16
Translation and rotation- rigid transformation
17

• As a single matrix equation

AP
BP
Rotation
Transformation
18
Homogenous coordinates

• Add an extra coordinate and use an equivalence
  relation
• For 3D, equivalence relation k(X,Y,Z,T) is the
  same as (X,Y,Z,T)
• Motivation it will be possible to write the
  action of a perspective camera as a matrix

19
Homogenous/Non-Homogenous transformation for 3D
point

• Non-homogenous to homogenous add 1 as the 4th
  coordinate

• Homogenous to non- homogenous devide 1st 3
  coordinates by the 4th

20
Homogenous/Non-Homogenous transformation for 2D
point

• Non-homogenous to homogenous add 1 as the 3rd
  coordinate

• Homogenous to non- homogenous devide 1st 2
  coordinates by the 3rd

21
Camera calibration

• Use the camera to tell you things about the world
• Relationship between coordinates in the world and
  coordinates in the image geometric calibration
• (We will not discuss here the relationship
  between intensities in the world and intensities
  in the image photometric camera calibration.)

22
Three coordinate systems involved

• Camera perspective projection.
• Image intrinsic/internal camera parameters
• World extrinsic/external camera parameters

23
The camera perspective equation

• The coordinates (x, y, z) of a scene point P
  observed by a pinhole camera are related to its
  image coordinates (x, y) by the perspective
  equation

• We have by similar triangles (x,y,z)-gt (f x/z, f
  y/z, -f ).
• Ignoring the third coordinate, we get (x,y,z)-gt
  (f x/z, f y/z)

P
P
24
Intrinsic parameters

• Relate the cameras coordinate system to the
  idealized coordinate system
• We can associate with a camera two different
  image planes the first one is a normalized plane
  located at a unit distance from the pinhole. We
  attach to this plane its own coordinate system
  with an origin located at the point where
  the optical axis pierces it. According to the
  perspective eq
• Perspective projection

25
Intrinsic parameters(cont)
f
1
26
Intrinsic parameters(cont)

• The second is the physical retina. It is located
  at a distance f ?1 from the pinhole, and the
  image coordinates (u,v) are usually expressed in
  pixel units.
• Pixels are usually rectangular, so the camera has
  two additional scale parameters k and l, and

f is a distance in meters
Define
27
Intrinsic parameters(cont)

• The actual origin of the camera coordinate system
  is at a corner C of the retina, and not at its
  center. It adds two parameters u0 and v0 that
  define the position (in pixel units) of C0 in the
  retinal coordinate system.

28
Intrinsic parameters(cont)

• The camera coordinate system may also be skewed,
  due to some manufacturing error, so the angle ?
  between the two image axes is not equal to 90.

29
Intrinsic parameters(cont)

• Using homogenous coordinates

3x4 matrix
30
Intrinsic parameters(cont)

• The physical size of the pixels and the skew are
  always fixed for a given camera, and they can in
  principle be measured during manufacturing

31
Extrinsic parameters

• Relate the cameras coordinate system to a fixed
  world coordinate system and specify its position
  and orientation in space.
• We consider the case where the camera frame (C)
  is distinct from the world frame (W).

Non-homogenous coordinates
Homogenous coordinates
32
Extrinsic parameters(cont)
33
Combining extrinsic and intrinsic calibration
parameters

• M can be defined with 11 free coefficients
• 5 are intrinsic parameters a,ß,u0,v0,?
• 6 are extrinsic the 3 angles defining R, 3
  coordinates of t

3 coords of t
3 raws of R
M is only defined up to scale in this setting!!
34
Rewriting the equation
World coordinates
Pixel coordinates
35

• Z is in the camera coordinate system, but we can
  solve it, cause

And we get
Relation between image positions, u,v to points
at 3D positions in P (homogenous coordinates)
36
Calibration methods

• Techniques for estimating the intrinsic and
  extrinsic parameters of a camera
• Suppose that a camera observes n geometric
  features such as points or lines with known
  positions in some fixed world coordinate system.
• We will
• Compute the perspective projection matrix M
  associated with the camera in this coordinate
  system
• Compute the intrinsic and extrinsic parameters of
  the camera from this matrix

Which features should we choose?
37
A linear approach to camera calibration

• For each feature point, i, we have

• For n features, we will get 2n equations

38
A linear approach to camera calibration(cont)
0
P
m
39
A linear approach to camera calibration(cont)

• When ngt6, the system is over-constrained, i.e.
  there is no non-zero vector m in R12 that
  satisfies exactly these equations.
• On the other hand, a zero vector is always a
  solution.
• According to the linear least-squares methods, we
  want to compute the value of the unit vector m
  that minimizes Pm2.
• In particular, estimating the vector m reduces to
  computing the eigenvectors and eigenvalues of the
  12x12 matrix PTP

40
Linear least squares methods

• Let us consider a system of n equations, p
  unknowns

• A is n x p matrix with coefficients aij, x
  (x1,..,xp)T
• There is no single solution if np. The non
  trivial solution exists only if A is non
  singular.
• We will try to find vector x that minimize E (the
  error measure)

41
Linear least squares methods(cont)

• Need to impose a contraint on x, since x0 yields
  the minimum.
• Since E(?x) ?2E(x), we will use the contraint
  x21.
• ExT(ATA)x , where ATA (pxp) matrix is positive
  smmetric matrix
• It can be diagonalized in an orthonormal basis of
  eigenvectors ei (i1,..,p) associated with
  eigenvalues 0 ?1 ?1 ?p
• we can write x as xµ1e1 µnen, that (µ12
  µp2 )1

? e1 minimizes the error E. It is the eigenvector
associated with the minimum eigenvalue of ATA (?1
)

42
Recovering the intrinsic and extrinsic parameters

• Once we have the M matrix, we can recover the
  intrinsic and extrinsic parameters in a simple
  mathematical process, described in ForsythPonce,
  section 6.3.1

43
Camera Calibration with a Single Image

• Sometimes more than one view of the same picture
  is used to estimate calibration parameters (For
  example, Stereo)
• Most Camera parameters can be estimated from the
  measurements of a single image when sufficient
  geometric object knowledge is available.
• The object knowledge used in the approach
  described here consists of parallelism and
  perpendicularity assumptions of straight object
  edges.
• In buildings parallel and perpendicular edges are
  usually abundant. Therefore, this method is often
  applicable for historic imagery of possibly
  demolished buildings taken with an unknown
  camera.

44
Targets for camera calibration
45
Projection of each point gives us two equations
and there are 11 unknowns. 6 points in general
position are sufficient for calibration.
46

• The 6 anchor points clicked by the user are
  represented in green. If the user had clicked
  more accurately, they should lie exactly at the
  corners of the small white squares. Using these 6
  points and the corresponding 3D anchor points,
  the program computes an initial estimate of the
  projection matrix.

http//www-sop.inria.fr/robotvis/personnel/lucr/de
tecproj.html
47
We take as input a set of at least 6 non-coplanar
3D anchor points, and their 2D images. The 2D
coordinates do not need to be very accurate, they
are typically obtained manually by a user who
clicks their approximate position.
48
Summary

• We saw the goal of calibration
• We mentioned Euclidean Geometry
• We learned about internal/external camera
  parameters
• We learned how to compute them from a given set
  of points
• We saw an example of calibration by one picture
  only
• We will see (stereo lecture) computing 3D
  coordinates from more than one picture (More than
  one view).