Last 4 lectures

1
Last 4 lectures
Camera Structure
HDR
Image Transform
Image Filtering
2
Today
Camera Projection Camera Calibration
3
Pinhole camera
4
Pinhole camera model
(X,Y,Z)
P
origin
p
(x,y)
principal point
(optical center)

• The coordinate system
• We will use the pin-hole model as an
  approximation
• Put the optical center (Center Of Projection) at
  the origin
• Put the image plane (Projection Plane) in front
  of the COP (Why?)

5
Pinhole camera model
principal point
6
Pinhole camera model
(X,Y,Z)
P
origin
p
(x,y)
principal point
(optical center)
7
Pinhole camera model
(X,Y,Z)
P
origin
p
(x,y)
principal point
(optical center)
8
Intrinsic matrix
Is this form of K good enough?

• non-square pixels (digital video)

9
Intrinsic matrix
Is this form of K good enough?

• non-square pixels (digital video)
• skew

10
Intrinsic matrix
Is this form of K good enough?

• non-square pixels (digital video)
• skew
• radial distortion

11
Distortion

• Radial distortion of the image
• Caused by imperfect lenses
• Deviations are most noticeable for rays that pass
  through the edge of the lens

12
Barrel Distortion
No distortion
Wide Angle Lens
Barrel
13
Pin Cushion Distortion
No distortion
Telephoto lens
Pin cushion
14
Modeling distortion
Distortion-Free
With Distortion
1. Project (X, Y, Z)to normalized image
coordinates
2. Apply radial distortion
3. Apply focal length translate image center

• To model lens distortion
• Use above projection operation instead of
  standard projection matrix multiplication

15
Camera rotation and translation
extrinsic matrix
16
Two kinds of parameters

• internal or intrinsic parameters focal length,
  optical center, skew
• external or extrinsic (pose) rotation and
  translation

17
Other projection models
18
Orthographic projection

• Special case of perspective projection
• Distance from the COP to the PP is infinite

• Also called parallel projection (x, y, z) ?
  (x, y)

19
Other types of projections

• Scaled orthographic
• Also called weak perspective
• Affine projection
• Also called paraperspective

20
Fun with perspective
21
Perspective cues
22
Perspective cues
23
Fun with perspective
Ames room
24
Forced perspective in LOTR
Elijah Wood 5' 6" (1.68 m)
Ian McKellen 5' 11" (1.80 m)
25
Camera calibration
26
Camera calibration

• Estimate both intrinsic and extrinsic parameters
• Mainly, two categories
• Using objects with known geometry as reference
• Self calibration (structure from motion)

27
Camera calibration approaches

• Directly estimate 11 unknowns in the M matrix
  using known 3D points (Xi,Yi,Zi) and measured
  feature positions (ui,vi)

28
Linear regression
29
Linear regression
30
Linear regression
Solve for Projection Matrix M using least-square
techniques
31
Normal equation (Geometric Interpretation)

• Given an overdetermined system

the normal equation is that which minimizes the
sum of the square differences between left and
right sides
32
Normal equation (Differential Interpretation)
nxm, n equations, m variables
33
Normal equation
Carl Friedrich Gauss
Who invented Least Square?
34
Nonlinear optimization

• A probabilistic view of least square
• Feature measurement equations

• Likelihood of M given (ui,vi)

35
Optimal estimation

• Log likelihood of M given (ui,vi)

• It is a least square problem (but not necessarily
  linear least square)
• How do we minimize C?

36
Nonlinear least square methods
37
Least square fitting
number of data points
number of parameters
38
Nonlinear least square fitting
39
Function minimization
Least square is related to function minimization.

• It is very hard to solve in general. Here, we
  only consider a simpler problem of finding local
  minimum.

40
Function minimization
41
Quadratic functions
Approximate the function with a quadratic
function within a small neighborhood
42
Function minimization
43
Computing gradient and Hessian
Gradient
Hessian
44
Computing gradient and Hessian
Gradient
Hessian
45
Computing gradient and Hessian
Gradient
Hessian
46
Computing gradient and Hessian
Gradient
Hessian
47
Computing gradient and Hessian
Gradient
Hessian
48
Searching for update h
Gradient
Hessian
Idea 1 Steepest Descent
49
Steepest descent method
isocontour
gradient
50
Steepest descent method

• It has good performance in the initial stage of
  the iterative process. Converge very slow with a
  linear rate.

51
Searching for update h
Gradient
Hessian
Idea 2 minimizing the quadric directly
Converge faster but needs to solve the linear
system
52
Recap Calibration

• Directly estimate 11 unknowns in the M matrix
  using known 3D points (Xi,Yi,Zi) and measured
  feature positions (ui,vi)

Camera Model
53
Recap Calibration

• Directly estimate 11 unknowns in the M matrix
  using known 3D points (Xi,Yi,Zi) and measured
  feature positions (ui,vi)

Linear Approach
54
Recap Calibration

• Directly estimate 11 unknowns in the M matrix
  using known 3D points (Xi,Yi,Zi) and measured
  feature positions (ui,vi)

NonLinear Approach
55
Practical Issue
is hard to make and the 3D feature positions are
difficult to measure!
56
A popular calibration tool
57
Multi-plane calibration

Images courtesy Jean-Yves Bouguet, Intel Corp.

• Advantage
• Only requires a plane
• Dont have to know positions/orientations
• Good code available online!
• Intels OpenCV library http//www.intel.com/rese
  arch/mrl/research/opencv/
• Matlab version by Jean-Yves Bouget
  http//www.vision.caltech.edu/bouguetj/calib_doc/i
  ndex.html
• Zhengyou Zhangs web site http//research.micros
  oft.com/zhang/Calib/

58
Step 1 data acquisition
59
Step 2 specify corner order
60
Step 3 corner extraction
61
Step 3 corner extraction
62
Step 4 minimize projection error
63
Step 4 camera calibration
64
Step 4 camera calibration
65
Step 5 refinement