image plane focal plane

1
The Pinhole Camera Model
X
m
F
I
y
Z
f
C
c
x
M
Y

• image plane
  focal plane
• The geometric model of pinhole camera consists of
  an image plane I and an eyepoint C on the focal
  plane F.
• The fundamental property of perspective is that
  every image point m is collinear with the C and
  its corresponding world point M. The point C is
  also called optical center, or the focus. The
  line Cc, perpendicular to I and to F, is called
  optical axis, c is called the principal point.

2
Equations for Perspective Projection

• Let (C,X,Y,Z) be the camera coordinate system
  (c.s.) and (c,x,y) be the image c.s.
• Its clear that
• From the geometric viewpoint, there is no
  difference to replace the image plane by a
  virtual image plane located on the other side of
  the focal plane. In this new c.s, an image point
  (x,y) has 3D coordinates (x,y,f).

3
The Pinhole Camera Model
X
I
x
f
C
c
Z
y
Y
m
M
lM
4
Perspective Projection Matrix

• In projective geometry any point along the ray
  going through the optical center projects to the
  same image point.
• So rescaling homogeneous coordinates makes no
  difference
• (X,Y,Z) s(X,Y,Z) (s X,
  s Y, s Z)
• It can be seen from (1)
• Equations (1) can be rewritten linearly (s
  arbitrary)

5
Perspective Projection Matrix and Extrinsic
Parameters

• Given a vector xx,y,T, we use to denote
  augmented vector by adding 1 as the last element.
  The 3x4 matrix P
• is called the camera perspective projection
  matrix. Given a 3D point MX,Y,ZT and its image
  mx,yT (2) can be written in matrix form as
  (with arbitrary scalar s)
• For real image point, s should not be 0. If s0,
  then Z0, and the 3D point is in the focal plane
  and image coordinates x and y are not defined.
  For all points in the focal plane but C

6
Perspective Projection Matrix and Extrinsic
Parameters

• their corresponding points in the image plane
  are at infinity. For the optical center C, we
  have xys0 and XYZ0.
• In practice 3D points can be expressed in
  arbitrary world c.s. (not only the camera c.s.).
  We go from the old c.s. centered at the optical
  center C to the new c.s. centered at point O
  (world c.s.) by a rotation R followed by a
  translation tCO. A relation between coordinates
  of a single point in a camera c.s. Mc and in the
  world c.s. Mw is
• Mc R
  Mw t
• or more compactly
  
• where D is Euclidean transformation of the 3D
  space

7
Perspective Projection Matrix and Extrinsic
Parameters

• The matrix R and the vector t describe the
  orientation and position of the camera with
  respect to the new world c.s. They are called the
  extrinsic parameters of the camera (3 rotations
  3 translations).

X
(R,t)
I
Xw
x
Zw
C
c
Z
y
O
camera c.s.
Yw
world c.s.
Y
m
M
8
Perspective Projection Matrix

• From (3) and (4) we have
• Therefore the new perspective projection matrix
  is
• In real images, the origin of the image c.s. is
  not the principal point and the scaling along
  each image axis is different, so the image
  coordinates undergo a further transformation
  described by some matrix K, and finally we have

9
Intrinsic Parameters of the Camera

• K is independent of the camera position. It
  contains the interior (or intrinsic) parameters
  of the camera. It is represented as an upper
  triangular matrix
• where and stand for the scaling along the
  x and y axes of the image plane, gives the
  skew (non-orthogonality) between the axes, and
  (u0, v0) are the coordinates of the principal
  point.

y
v
m
v0
c
x
q
u0
o
u
10
Intrinsic Parameters of the Camera

• For a given point, let .
  Since we have
  and thus
• Normalized coordinate system of the camera is a
  system where the image plane is located at a unit
  distance from the optical center (i.e. f1). The
  perspective projection matrix P in such c.s. is
  given by

11
Intrinsic Parameters of the Camera

• For a world point its
  coordinates in normalized coordinate system are
• A matrix Pnew defined by (10) can be decomposed
• where
• Matrix A contains only intrinsic parameters, and
  is called camera intrinsic matrix.

12
Intrinsic Parameters of the Camera

• It is thus clear that the normalized image
  coordinates are given by
• Through this transformation from the available
  pixel image coordinates,u,vT, to the imaginary
  normalized image coordinates the
  projection from the space onto the normalized
  image does not depend on the specific cameras.
  This frees us from thinking about characteristics
  of the specific cameras and allows us to think in
  terms of ideal systems

13
The General Form of the Perspective Projection
Matrix

• Camera can be considered as a system with
  intrinsic and extrinsic parameters. Here are 5
  intrinsic parameters
• the coordinates u0,v0 of principal point, and
  the angle between the two image axes. There
  are 6 extrinsic parameters, three for the
  rotation and three for the translation, which
  define the transformation from the world
  coordinate system, to the standard coordinate
  system of the camera. Combining (7) and (13)
  yields the general form of the perspective
  projection matrix of the camera
• The projection of 3D world coordinates
  to 2D pixel coordinates
  is then given by (s is arbitrary scale
  factor)

14
The General Form of the Perspective Projection
Matrix cont.

• Matrix P has 3x412 elements, but has only 11
  degrees of freedom. Why?
• Let be the (i,j) entry of the matrix P.
  Eliminating the scalar s in (17) yields two
  nonlinear equations

15
The General Form of the Perspective Projection
Matrix cont.

• Problem 1. Given the perspective projection
  matrix P find coordinates of the optical center C
  of the camera in the world coordinate system.
• Solution. Decompose the 3x4 matrix P as the
  concatenation of 3x3 matrix B and a 3-vector b,
  i.e. P B b. Assume that the rank of B is 3.
  Under the pinhole model, the optical center
  projects to 0 0 0T (i.e. s0). Therefore, the
  optical center can be obtained by solving
• The solution is

16
The General Form of the Perspective Projection
Matrix cont.

• Problem 2. Given matrix P and an image point m
  find an optical ray going through this point.
• Solution. The optical center C is on the optical
  ray. Any point on this ray is also projected on
  m. Without loss of generality, we can choose the
  point D such that the scale factor s 1, i.e.
• This gives
  A point on the optical ray is thus given by
• Where l varies from 0 to

17
Perspective Approximations

• The perspective projection (2) is a nonlinear
  mapping which makes it difficult to solve many
  vision problems. It also ill-conditioned when
  perspective effects are small.
• There are several linear mappings, approximating
  the perspective projection
• Orthographic Projection. It ignores the depth
  dimension. It can be used if distance and
  position effects can be ignored.

18
Orthographic and Weak Perspective Projection

• Orthographic Projection

X
I
x
C
c
Z
y
Y
19
Orthographic and Weak Perspective Projection

• Orthographic Projection

X
I
C
Z
Y
20
Weak Perspective Projection

• Weak Perspective Projection
• Much more reasonable approximation is Weak
  Perspective Projection. When the object size is
  small enough with respect to the distance from
  the camera to the object, Z can be replaced by a
  common depth Zc . Then the equations (1) become
  linear
• Here we assumed that the focal length f
  is normalized to 1

21
Weak Perspective Projection

• Two step projection
• image plane
  average depth plane

X
I
C
Zc
Z
Y
22
Weak Perspective Projection

• Let
• Equation (12) can be written as

23
Weak Perspective Projection

• Taking into account the intrinsic and extrinsic
  parameters of the camera yields
• where A is the intrinsic matrix (14), and D
  is the rigid transformation (5).