CS 44957495 Computer Vision

1
CS 4495/7495Computer Vision

• Cameras, Geometric Cameras Models,
• and Geometric Camera Calibration
• Frank Dellaert
• Some Slides by Forsyth Ponce, Jim Rehg

2
Outline

• Pinhole Cameras
• Cameras with Lenses
• Homogeneous Coordinates
• Joining Points, Lines, and Planes
• Geometric Camera Model
• Intrinsic Parameters
• Extrinsic Parameters
• Projective Cameras
• Geometric Camera Calibration

3
Cameras

• First photograph due to Niepce
• Basic abstraction is the pinhole camera
• lenses required to ensure image is not too dark
• various other abstractions can be applied

4
Pinhole cameras

• Abstract camera model - box with a small hole in
  it

• Pinhole cameras work in practice

5
Distant objects are smaller
C
B
6
Parallel lines meet
7
Vanishing points

• each set of parallel lines (direction) meets at
  a different point
• The vanishing point for this direction
• Sets of parallel lines on the same plane lead to
  collinear vanishing points.
• The line is called the horizon for that plane

• Good ways to spot faked images
• scale and perspective dont work
• vanishing points behave badly
• supermarket tabloids are a great source.

8
(No Transcript)
9
The equation of projection
Weird and difficult diagram in the book
10
Pinhole Camera

• The better way to do cameras

k
P
p
i
C
Q
f
q
O
j
11
Geometric properties of projection

• Points go to points
• Lines go to lines
• Planes go to whole image
• Polygons go to polygons
• Degenerate cases
• line through focal point to point
• plane through focal point to line

12
Weak perspective

• perspective effects, but not over the scale of
  individual objects
• collect points into a group at about the same
  depth, then divide each point by the depth of its
  group
• easy
• - wrong

13
Orthographic projection
14

• Cameras with Lenses

15
Pinhole too big - many directions are
averaged, blurring the image Pinhole too
small- diffraction effects blur the
image Generally, pinhole cameras are dark,
because a very small set of rays from a
particular point hits the screen.
16
The reason for lenses
17
The thin lens
18
Focussing
http//www.theimagingsource.com
19
Spherical aberration
20
Vignetting
21
Other annoying phenomena

• Chromatic aberration
• Light at different wavelengths follows different
  paths hence, some wavelengths are defocussed
• Machines coat the lens
• Humans live with it
• Geometric phenomena
• Radial Distortion
• Barrel distortion

22
Other stuff in Chapter 1

• Background reading
• Human eye
• Sensing
• CCD Cameras
• Sensor Models

23

• Homogeneous Coordinates

24
2D Coordinate Frames Points

• coordinates x and y

j
p (x,y)T
i
o
25
2D Lines

• Line l axbyc

j
p(x,y)T
c
(a,b)T
i
26
Homogeneous Coordinates

• Uniform treatment of points and lines
• Line-point incidence lTp0

j
stay the same when scaled
p(x,y,1)T(kx,kx,k)T
c
l(a,b,c)T(ka,kb,kc)T
(a,b)T
i
27
Join cross product !

• Join of two lines is a pointpl1xl2
• Join of two points is a linelp1xp2

28
Joining two parallel lines ?

• (a,b,c)

(a,b,c)
(a,b,d)
29
Points at Infinity !
(-b,a,0)T
Line at infinity linf(0,0,1)T
j
l(a,b,c)T
i
(-b,a,0)T
30
In 3D Same Story

• 3D points (x,y,z,w)T
• 3D planes (a,b,c,d)T
• join of three points plane
• join of three planes point
• plane at infinity (0,0,0,1)T

31

• Joining Points, Lines, and Planes

32
Method 1 Determinants

• 2D Line between two 2D points
• Point x on l must be linear comb of p1 p2

33
Lines
cameras01.m
34
Join of 3D points
Determinant minors (analog of cross product)
35
Method 2 Homogenous Equations

• we want lTp0 for all lines !

Live Demo! cameras02.m
36
cameras03.m
37
Solving Homogeneous Equations

• Much nicer than in book

38
SVD Figure
AU S VT
4x3
4x4
3x3
R3P2
39
Applied to line fitting
cameras04.m
Caveat Normalization matters !
40
Plane fitting
cameras05.m
41

• Geometric Camera Model

42
Perspective Camera Model
43
We can see infinity !
Railroad parallel lines
44

• Intrinsic Parameters

45
Normalized Image coordinates
1
O
uX/Z dimensionless !
P
46
Pixel units
Pixels are on a grid of a certain dimension
f
O
uk f X/Z in pixels ! f m (in
meters) k pixels/m
P
47
Pixel coordinates
We put the pixel coordinate origin on topleft
f
O
uu0 k f X/Z
P
48
Pixel coordinates in 2D
640
(0.5,0.5)
(u0 k f X/Z, v0 k f Y/Z)
i
(u0,v0)
480
(640.5,480.5)
j
49
Important MATLAB Convention
(1,1) !
Just as good as any other convention !
50
Summary Intrinsic Calibration
5 Degrees of Freedom !
51

• Extrinsic Parameters

52
Camera Pose
In order to apply the camera model, objects in
the scene must be expressed in camera coordinates.
Camera Coordinates
World Coordinates
Calibration target looks tilted from
camera viewpoint. This can be explained as
a difference in coordinate systems.
53
Rigid Body Transformations

• Need a way to specify the six degrees-of-freedom
  of a rigid body.
• Why are their 6 DOF?

A rigid body is a collection of points whose
positions relative to each other cant change
Fix one point, three DOF
Fix second point, two more DOF (must
maintain distance constraint)
Third point adds one more DOF, for
rotation around line
54
Two kinds of transformations

• Rotation and Translation of body
• Two views to describe
• Coordinate frame does not change, but the
  coordinates change
• A more useful view is look upon it as change of
  coordinate frames.
• Shape of body doesn't matter.

55
Notations

• Superscript is referencing coordinate frame
• AP is coordinates of P in frame A
• BP is coordinates of P in frame B
• The transformation in the example is a pure
  translation.

56
Translation

• Using homogeneous coordinates, translation can
  be expressed as a matrix multiplication.
• Translation is communicative

57
Rotation
means describing frame A in The coordinate system
of frame B
58
Rotation
Orthogonal matrix!
59
Example Rotation about z axis
What is the rotation matrix?
60
Rotation in homogeneous coordinates

• Using homogeneous coordinates, rotation can be
  expressed as a matrix multiplication.
• Rotation is communicative

61
Rigid transformations
62
Rigid transformations (cont)

• Unified treatment using homogeneous coordinates.

63

• Projective Cameras

64
Projective Camera Matrix
56 DOF 11 !
65

• Geometric Camera Calibration

66
Affine Calibration
67
Affine Calibration
68
Inserting Synthetic Objects
pMP
69
Projective Camera Matrix
56 DOF 11 !
70
Columns Rows of M
m2P0
O
71
Linear Approach
72
Example 6 points on a cube
cameras06.m
73
Inserting synthetic objects
cameras07.m
74
Recipe

• Form homogeneous eq. Am0
• Do SVD on AUSVT
• Pick eigenvector w smallest eigenvalue
• Then, from M, recover K,R, and t (ugly, see
  page 46, not tested)

75
Non-linear Method

• Write non-linear least-squares equations
• Optimize for 11 unknowns
• Levenberg-Marquardt optimization
• Maximum Likelihood solution
• - Can get stuck in local minima
• Initialize
• using linear method
• or just guess