COMM2J Vision

1
Topic 19 Camera calibration stereo viewing
An important aspect of scene interpretation is 3D
model building based on two or more stereo images
of the scene. Localisation of a robot can also be
achieved by measuring the location of known
landmarks in the scene. Both applications require
a calibrated camera.

• Sub topics
• Intrinsic extrinsic camera parameters
• The pinhole camera
• Extrinsic parameters of the pinhole camera
• Intrinsic camera parameters
• Lens defects
• Radial distortion
• Camera calibration
• Stereoscopic viewing
• Landmark navigation

COMM2J Vision Intelligent Robots 2003
2
Topic 19 Camera parameters
A digital camera will transform a point in real
space r (x,y,z) to pixel co-ordinates s
(U,V). This is achieved using a three stage
transformation

• E is the transformation which represents the
  position and orientation of the camera relative
  to the real world co-ordinates.
• P is the projection used.
• I is the transformation which represents the
  properties of the camera. (focal length,
  principal points etc.)

COMM2J Vision Intelligent Robots 2003
The factors forming E are known as the extrinsic
parameters. The factors forming I are known as
the intrinsic parameters.
3
Topic 19 Pinhole camera model
The pinhole camera represents a simple ideal
model of a camera which incorporates only linear
transforms and has only one intrinsic property
the focal length f of the camera.
(U,V,W)
Optic axis
f
COMM2J Vision Intelligent Robots 2003
(x,y,z)
4
Topic 19 Extrinsic parameters for the pinhole
camera
(xc,yc,zc)
To define the camera position we need to specify
its position in space (xc,yc,zc) and its
orientation. You should be aware that this can be
done in a variety of ways.
COMM2J Vision Intelligent Robots 2003
For example we could in addition to the position
define a view direction vector N, an up vector
V which orientates camera with respect to the
view direction and an optional positive x
direction U which defines a left or right handed
co-ordinate system. This is the approach taken by
MatLab and is common in computer graphics viewing
systems. Here we take a variant of this for
illustration
5
Topic 19 Example of an extrinsic parameter
matrix
Imagine the camera is at the origin of world
co-ordinates. We first rotate the camera about
the X axis through an angle ? then about the
Y axis by ? and then about the Z axis by ?.
The camera is now orientated (the angles defined
are known as Euler angles). Finally we locate the
rotated camera at (xc,yc,zc). The matrix E for
this transformation is
COMM2J Vision Intelligent Robots 2003
6
Topic 19 Components of the matrix
COMM2J Vision Intelligent Robots 2003
7
Topic 19 Intrinsic camera parameters
The factor (xi,yi) represents the location of the
image point in the image plane. However we,
finally need to convert this to pixel
co-ordinates (px,py).
Image plane

f
Image
COMM2J Vision Intelligent Robots 2003

Optic axis
Object
Principal point In image plane In pixel units
Principal point
(ppx,ppy,ppz)
(Cx,Cy)
A full set of intrinsic camera parameters allows
for skewed rectangular pixels
A single pixel
8
Topic 19 The camera calibration matrix K
COMM2J Vision Intelligent Robots 2003
Sometimes fx is written ?f and fy as f where ?
is known as the aspect ratio for the pixel. For a
basic calibration the matrix K can be used as the
intrinsic camera matrix I. Initial values for the
parameters in K can be obtained from the camera
handbook.
9
Topic 19 Incorporating factors for lens defects
If high accuracy is required or alternatively the
camera is of low quality (typical of robot
cameras) then lens defects need to be allowed
for. Real camera lenses are quite complex in
order to minimise the various forms of lens
defects
COMM2J Vision Intelligent Robots 2003
10
Topic 19 Types of aberration
The failure of the lens system to bring all light
rays to a single image point may be due to a
variety of causes known as aberrations. Principle
amongst these are astigmatism, chromatic ,
spherical, coma, curvature of field, distortion
aberrations
COMM2J Vision Intelligent Robots 2003
11
Topic 19 Aberrations
Field curvature
Coma
COMM2J Vision Intelligent Robots 2003
Distortion
12
Topic 19 Correcting for radial distortion
(barrel pincushion)
Radial distortion is a radial displacement of
points to or from the centre of an image due to
the fact that objects at different angular
distances from the object axis have effectively
different focal lengths. The effect can be
cancelled out by warping the image. The
undistorted image plane co-ordinates (xi,yi) can
be obtained from the observed image co-ordinates
(xo,yo) by the transform
COMM2J Vision Intelligent Robots 2003
where
K1 and K2 are the first and second radial
distortion parameters It is possible in addition
to radial to have tangential distortion
parameters.
13
Topic 19 Camera calibration background
COMM2J Vision Intelligent Robots 2003
Known distance
Known spacing
(From Matlab calibration toolbox)
14
Topic 19 Calibration paradigm
Obtain calibration images

• Error minimisation is usually a two stage process
  
• Linear least squares
• Minimise image distances
• Minimising image distances gives good results
  where the minimisation uses the following
  evaluation function

Identify image points
COMM2J Vision Intelligent Robots 2003
Obtain camera matrix by error minimisation
Obtain intrinsic parameters from camera matrix
Image distance minimising can get stuck in a
local minimum hence the use of linear least
squares to get an approximate solution.
15
Topic 19 MatLab camera calibration toolbox
An open source MatLab camera calibration toolbox
is available which gives excellent results.
COMM2J Vision Intelligent Robots 2003
The C implementation of this toolbox is included
in the Open Source Computer Vision library
distributed by Intel and freely available online.

16
Topic 19 Using the MatLab calibration toolbox
You need about 20 images taken of a standard
object from many different angles. You do not
need to know the camera object distance. On
each image mark the four interior corners and the
toolbox will find all the interior points to an
accuracy of about 0.1 pixel.
COMM2J Vision Intelligent Robots 2003
17
Topic 19 Typical results from the MatLab camera
calibration
COMM2J Vision Intelligent Robots 2003
18
Topic 19 MatLab camera parameter list
The MatLab calibration provides the following
basic parameters fx, fy, Cx, Cy, ? In addition
the following distortion coefficients are
recorded K1, K2, K3 for radial distortion P1,
P2 for transverse distortion. For many
applications a smaller subset is adequate. With
modern cameras ? is almost always 0 and the
aspect ratio fx/fy is one. The radial distortions
are small and K3 is 0.
COMM2J Vision Intelligent Robots 2003
19
Topic 19 Putting the camera to work
stereoscopic imaging
Triangulation is a process for locating a third
point from two known points by measuring the
baseline between the two known points and the
angles of the the third point at the position of
the known points
COMM2J Vision Intelligent Robots 2003
The method has been the basis of map surveying
for hundreds of years. Sir George Everests
survey of the Himalayas measured the height of
Mount Everest to an accuracy of 20 feet in 29,000
feet despite being unable to get his baseline
closer than 100 miles away.
20
Topic 19 Putting the camera to work
stereoscopic imaging
Using two or more camera positions we can record
a scene rapidly and provided we can find
corresponding points in two or more images we can
triangulate the scene to build a 3D model of the
scene. By hand the problem of correspondence is
trivial but doing it automatically is still a
very active research area. The following 4 slides
which explain the principle have been taken from
a lecture course by Tony Pridmore which you can
find at
COMM2J Vision Intelligent Robots 2003
21
Topic 19 Stereo viewing
COMM2J Vision Intelligent Robots 2003
22
Topic 19 Stereo viewing
COMM2J Vision Intelligent Robots 2003
23
Topic 19 Stereo viewing
COMM2J Vision Intelligent Robots 2003
24
Topic 19 Stereo viewing
COMM2J Vision Intelligent Robots 2003
25
Topic 19 Camera calibration stereo viewing
COMM2J Vision Intelligent Robots 2003