CS 39549525: Spring 2003

1
CS 395/495-25 Spring 2003

• IBMR Week 7A
• The Camera MatrixContinued...
• Jack Tumblin
• jet_at_cs.northwestern.edu

2
IBMR-Related Seminars

• 3D Scanning for Cultural Heritage Applications
  Holly Rushmeier, IBM TJ Watson
• Friday May 16 300pm, Rm 381, CS Dept.
• no title yet ...ltBRDF, BRSSDF capture? Optics of
  Hair? Inverse Rendering?gt
• Steve Marschner, Cornell University Friday May
  23 300pm, Rm 381, CS Dept.

3
Reminders

• ProjA graded Good Job! 90,95, 110
• ProjB graded Good! minor H confusions...
• MidTerm graded
• ProjC posted tonight Due next Friday, May 15
• Start Watsons Late Policy Grade -(3n)
  points n of class meetings late
• ProjD coming Thurs May 15, due Thurs May 29
• Take-Home Final Exam Thurs June 5, due June 11

4
Basic Cameras Revisited

• Plenty of Terminology
• Image Plane or Focal Plane
• Focal Distance f
• Camera Center C
• Principal Point p
• Principal Axis zc
• Principal Plane (?!?! DOESNT touch principle
  point!?!?)
• Camera Coords (xc,yc,zc)
• Image Coords (x,y)

yc
C
p
f
zc
y'
x'
xc
5
Recall Basic Camera P0

• Basic Camera as a 3x4 matrix
• To translate image origin (x,y) away from zc
  axisShift principal point p from (0,0) to
  (px,py)
• does NOT use homogeneous 1 term in X
• is NOT obvious scales zc (think in P2 zcx3
  1,...)

yc
xc yc zc 1
x y z
f 0 px 0 0 f py 00 0 1 0

y
y
p
C
z
f
zc
y'
P0 X x
x'
xc
6
Basic Camera P0 P3?P2 (or camera R3)

• Basic Camera P0 is a 3x4 matrix
• Non-square pixels? change scaling (?x, ?y)
• Parallelogram pixels? set nonzero skew s
• K matrix (internal) camera calib. matrix

yc
xc yc zc 1
x y z
?xf s px 0 0 ?yf py 00 0
1 0

y
y
p
C
z
K (3x3 submatrix)
f
zc
y'
P0 X x
x'
xc
K 0 P0
7
Complete Camera Matrix P

• K matrix internal camera calib. matrix
• RT matrix external camera calib. matrix
• T matrix Translate world origin to cam. origin,
• R matrix then Rotate world to fit cam. axes
• Combine write
• (P0RT)X x
• as
• PX x

Output x P2 Camera Image
Input X P3 World Space
x(camera space)
8
The Pieces of Camera Matrix P

xw yw zw tw

xc yc zc
p1
p3
p4
P
p2
P
PX x, or

• Columns of P matrix image of world-space axes
• p1,p2,p3 image of x,y,z axis vanishing
  points
• Direction D 1 0 0 0T point on P3s x1 axis,
  at inifinity
• PD 1st column of P P1. Repeat for y and z
  axes.
• p4 image of the world-space origin pt.
• Proof let D 0 0 0 1T world origin
• PD 4th column of P image of origin pt.

9
The Pieces of Camera Matrix P
xw yw zw tw
P1T


xc yc zc
P
PX x, or

P
P2T
P3T

• Rows of P matrix camera planes in world space
• row 1 P1T image x-axis plane
• row 2 P2T image y-axis plane
• row 1 P3T cameras principal plane

10
The Pieces of Camera Matrix P
xw yw zw tw
P1T


xc yc zc
P
PX x, or

P
P2T
P3T

• Rows of P matrix planes in world space
• row 1 P1 image x-axis plane
• row 2 P2 image y-axis plane
• row 1 P3 cameras principal plane

yc
p
C
f
zc
xc
11
The Pieces of Camera Matrix P
xw yw zw tw
P1T


xc yc zc
P
PX x, or

P
P2T
P3T

• Rows of P matrix planes in world space
• row 1 P1 image x-axis plane
• row 2 P2 image y-axis plane
• row 1 P3 cameras principal plane

yc
p
C
f
zc
xc
12
The Pieces of Camera Matrix P
xw yw zw tw
P1T


xc yc zc
P
PX x, or

P
P2T
P3T

• Rows of P matrix planes in world space
• row 1 P1 image x-axis plane
• row 2 P2 image y-axis plane
• Careful! Shifting theimage origin by px, py
  shifts the x,y axis planes!

13
The Pieces of Camera Matrix P
xw yw zw tw
P1T


xc yc zc
P
PX x, or

P
P2T
P3T

• Rows of P matrix planes in world space
• row 1 P1 image x-axis plane
• row 2 P2 image y-axis plane
• row 3 P3 cameras principal plane

yc
C
p
f
zc
xc
14
The Pieces of Camera Matrix P
xw yw zw tw
P1T


xc yc zc
P
PX x, or

P
P2T
P3T

• Rows of P matrix planes in world space
• row 1 P1 image x-axis plane
• row 2 P2 image y-axis plane
• row 3 P3 cameras principal plane
• princip. plane P3 p31 p32 p33 p34T
• its normal direction p31 p32 p33 0 T
• Why is it normal? Its the world-space P3
  direction of the zc axisBOOK has yet another
  inconsistent notation for this.....

Principal plane p3
yc
zc
C
p
f
xc
15
The Pieces of Camera Matrix P
xw yw zw tw

• Principal Axis Vector (zc) in world space
• Normal of principal plane m3 p31 p32 p33 T
• P3 Scaling ? Ambiguous direction!! /- m3 ?
• Solution use det(M)m3 as front of camera
• Principal Point p in image space
• image of (infinity point on zc axis m3) Mm3
  p x0 (book renames p as x0)

m1T
xc yc zc
P
p4
PX x, or
P

m2T
m3T
M
yc
p
x0
C
f
zc
xc
16
The Pieces of Camera Matrix P
xw yw zw tw


m1T
xc yc zc
P
p4
PX x, or
P

m2T
m3T
M

• Where is the Camera?
• Camera center C is at camera origin(xc,yc,zc)0C
• What point C will the camera P transform to C0
  ?Answer C is the Null Space of P P C C 0
  (solve for C. SVD works, but heres an easier
  way)
• Finite Camera C Affine Camera C

yc

1

0
d
-M-1p4
C
p
f
zc
(where Md0)
xc
17
Uses for Camera Matrix P
xw yw zw tw

• Each world space R3 point d (x,y,z) defines a
  world-space P3 direction D (x,y,z,0).
• What is the image point xd from direction D?
• xd PD M p4 D M d
• (P4 column has no effect, because of Ds zero)
• xd M d or M-1xd d

xc yc zc
P
M
p4
PX x, or
P

yc
xd

d
p
C
f
zc
X (world space)
xc
18
Uses for Camera Matrix P
xw yw zw tw

• Given image point x0 and camera matrix P,Find
  ray X(?) in world space through both
• Slow, Obvious way
• Pseudo-invert P, apply to x0
• Define pseudo-inverse P as PT(PPT)-1 (note
  PP I)
• Find a world-space point on ray X0 P x0
• LIRP with world-space camera point C X(?) C
  (X0-C)?

xc yc zc
P
M
p4
PX x, or
P

X(?)
yc
x0


p


C
f
zc

xc
19
Uses for Camera Matrix P
xw yw zw tw

• Given image point x0 and camera matrix P,Find
  ray X(?) in world space through both
• Better way ray from C to point D0 at infinity
• Point x0 is the image of (unknown) world-space
  direction D0 (x,y,z,0)T. Define a point d0
  (x,y,z)T.
• Recall we can find d0 from the image M-1x0 d0
  
• Recall world-space camera C -M-1p4
• X(?) ? M-1x0 C M-1 (?x0 p4)
• 0 1 1

xc yc zc
P
M
p4
PX x, or
P


X(?)
yc

x0


p
C
f
zc
xc
20
Uses for Camera Matrix P
xw yw zw tw

• Given world-space point X0, camera matrix P,Find
  camera depth z0
• X0 xw, yw, zw, twT seen thru camera P is
• X0P x0 xc, yc, 1Twc
• Then signed depth z0 is
• z0 wc sign(det M) tw m3

m1T
xc yc zc
P
p4
PX x, or
P

m2T
m3T
M
X0
yc
x0
zc
z0
p
C
f
xc
21
Skipped

• P K0RT How can we separate K, R, T?
• Answer K is triangular use QR decomposition
• Cameras at Infinity
• Orthographic or Parallel Projection Cameras
• Transition to Orthographic
• Weak Perspective projection cameras
• the zoom lens (variable f)
• Moving line-scan or pushbroom cameras
• Translation Scan aerial/sattelite cameras
• Cylindrical Scan panoramic cameras
• UNC HiBall Tracker 6 tiny self-locating
  line-scan cameras

22
Chapter 6 In Just One Slide

• Given point correspondence sets (xi?? Xi), How
  do you find camera matrix P ? (full 11 DOF)
• Surprise! You already know how !
• DLT method -rewrite H x x as Hx ? x
  0 -rewrite P X x as PX ? x 0
• -vectorize, stack, solve Ah 0 for h
  vector -vectorize, stack, solve Ap 0 for p
  vector
• -Normalizing step removes origin dependence
• More data ? better results (at least 28 point
  pairs)
• Algebraic Geometric Error, Sampson Error

23
END