Day 2: Structure from motion diagram

1
Day 2 Structure from motion diagram
Camera geometry
Projective geometry
n-view geometry
Three-view geometry
Two-view geometry
Calibration
2
First topic projective geometry
Camera geometry
Projective geometry HZ 2,3
n-view geometry
Three-view geometry
Two-view geometry
Calibration
3
But a short introduction to the camera first
Camera geometry
Projective geometry HZ 2,3
n-view geometry
Three-view geometry
Two-view geometry
Calibration
4
Introduction to the camera

• Faugeras the camera as projective engine A
  camera is a particular geometric engine which
  constructs planar images of the 3-dimensional
  world through a projection. FL 174
• we want to invert this imaging process to yield a
  model of the 3-dimensional world
• must understand image formation

5
Theory of perspective in history

• pinhole cameras and the image formation process
  were first explored by Renaissance painters,
  where it was called the theory of perspective
• Leonardo da Vinci (quoting John Pecham)
  Perspective ... should be preferred above all
  mans discourses and disciplines. In this
  subject the visual rays are elucidated by means
  of demonstrations which derive their glory not
  only from mathematics but also from physics the
  one is adorned equally with the flowers of the
  other. Kemp, p. 5
• Brunelleschi, a Florentine architect (Baptistery,
  Duomo) developed theory of perspective circa 1413
• theory ? perspective machines ? camera obscura ?
  daguerrotype ? camera
• Faugeras cover FL176 camera obscura FL177
• need a dark room to capture just the light from
  the pinhole
• used by painters as a portable device (see
  Vermeer in The Girl with a Pearl Earring)
• lenses added at pinhole to gather light better
• Daguerre replaced one wall of camera obscura with
  photosensitive film, yielding first photograph
  (daguerrotype)
• et voila, the camera

6
Camera obscura
7
Theory of perspective 2

• behaviour at infinity immediately became of
  interest, as vanishing points were incorporated
  into paintings
• vanishing point in Bosse FL205
• vanishing line in Houckgeest FL206
• renewed interest with the invention of cameras,
  leading to photogrammetry (theory of measurement
  of 3d world from 2d images) FL174 Pollefeys
  CACM02
• projective geometry developed in mathematics a
  tool that can be used to understand the image
  formation process

8
Pinhole camera model

• geometrically very simple, encoding perspective
  projection
• camera center optical center C
• image plane retinal plane P
• image x of X intersection of XC with P
• practical setup
• two screens image plane and focal plane
• pinhole in focal plane

9
Camera terminology

• camera center, image plane
• confusion HZ (153) equate the image and focal
  planes but for Faugeras (F35), the focal plane
  passes through the optical center
• focal length f dist (C, P)
• focal plane plane through C parallel to P
• principal axis line through C normal to P
  optical axis
• principal point intersection of principal axis
  and P

10
Perspective projection

• not a linear map in Euclidean geometry!
• suppose
• camera center origin
• principal axis z-axis
• (X,Y,Z) ? f/Z (X,Y) in image plane Zf
• this is not linear

11
Projective geometry simplifies

• perspective projection will be a linear map in
  projective geometry
• what are other motivations for using projective
  geometry rather than Euclidean?
• projective geometry is a simpler generalization
  of Euclidean geometry
• lifting up a dimension simplifies, just as
  lifting from flatland to 3-space simplifies
  interpretation of flatland (e.g., sphere passing
  through flatland)
• so
• (a) less special cases
• e.g., translation vs. rotation/scaling
• (b) algebra is often simplified (homogeneous
  coordinates)
• can reason about geometry at infinity
• infinity is one of those special cases for
  Euclidean geometry
• points and lines are treated equivalently (dual),
  so they interact well

12
First topic projective geometry
Camera geometry
Projective geometry HZ 2,3
n-view geometry
Three-view geometry
Two-view geometry
Calibration
13
Projective geometry diagram(2-space)
Behaviour at infinity
Primitives pt/line/conic HZ 2.2
DLT alg HZ 4.1
Hierarchy of maps Invariants HZ 2.4
Projective transform HZ 2.3
Rectification HZ 2.7
14
First topic primitives
Behaviour at infinity
Primitives pt/line/conic HZ 2.2
DLT alg HZ 4.1
Hierarchy of maps Invariants HZ 2.4
Projective transform HZ 2.3
Rectification HZ 2.7
15
Point/line diagram
line (a,b,c)
point x y w
point on line P . L 0
L1 L2
line/line intersection L1 x L2
intersection
point/point join P1 x P2
P1 P2
join
16
Point-line representation

• we want a new data structure (or encoding) for
  points and lines
• importance of representation and notation think
  of pre-20th century mathematics notation (e.g.,
  Descartes)
• consider the implicit representation of a line in
  2-space axbyc0
• the lines homogeneous representation is (a,b,c)
• coefficients of the lines implicit equation
• for consistency with tensors later, let us treat
  line as a row vector (although HZ introduce as a
  column) OK to treat lines and points as generic
  vectors at early stages
• consider a point in 2-space (x,y)
• its homogeneous representation is (x,y,1)
• coordinates in the implicit equation
• a column vector
• what are the ramifications of this encoding?
• point/line incidence, intersection and join are
  all simplified
• point P lies on line L iff P.L 0
• intersection of two lines L1 and L2 is L1 x L2
  (recall cross product)
• join of two points P1 and P2 is P1 x P2
• HZ 26-27

17
Proofs

• the point P lies on the line L iff P.L 0
• after all, point satisfies line equation
• dually, L intersects P iff L.P 0
• recall that inner product 0 encodes
  orthogonality
• intersection of two (distinct) lines L1 and L2 is
  L1 x L2
• think geometrically L1 x L2 is normal to L1 and
  L2
• so (L1xL2) . L1 0 and (L1xL2) . L2 0
• so the point L1xL2 lies on L1 and L2
• e.g., what is the intersection of x1 and y1?
• join of two (distinct) points P1 and P2 is P1xP2
• again (P1xP2) . P1 0 and (P1xP2) . P2 0, so
  this line contains these two points
• morals
• inner product encodes incidence
• this is also simple matrix multiplication if we
  represent points as columns and lines as rows
• cross product encodes intersection and join

18
Exercise

• think of inputting lines and computing their
  intersections using these projective results
• points ? lines ? intersections

19
Homogeneous coordinates

• the point and line encodings are vectors in
  projective space, with homogeneous
  representations
• note that axbyc0 is equivalent to kax kby
  kc 0
• (a,b,c) k(a,b,c) for any k \neq 0
• projective space P2 the set of equivalence
  classes of vectors in R3 (0,0,0) under the
  equivalence relation (a,b,c) k(a,b,c) for
  nonzero k
• in projective space, objects are only known up to
  a scalar multiple
• therefore, both the point and line in 2-space
  have two degrees of freedom (dof), not three
• the two dof in a line (a,b,c) are the two
  independent ratios abc
• projective space always uses one extra dimension
• can visualize P2 as lines through the origin in
  3-space
• projective geometry is the study of the geometry
  of projective space

20
Perspective projection is now linear

• translate from homogeneous coordinates in
  projective space to Cartesian coordinates in
  Euclidean space by
• (x,y,w) ? (x/w, y/w)
• after all, (x,y,1) ? (x,y) and (kx,ky,k) ? (x,y)
• P2 ? R2
• this should look familiar perspective projection
• in projective space, perspective projection is a
  linear map
• Euclidean (X,Y,Z) ? f/Z (X,Y)
• (X,Y,Z,1) ? (fX,fY,Z)
• this can be encoded by a matrix
• (X,Y,Z,1) ? (diag(f,f,1) 0) (X,Y,Z,1)
• aside translation in 3-space is now encodable by
  a homogeneous 4x4 matrix

21
Duality

• points and lines are dual under this
  representation
• think of axbycw 0
• point representation and line representation are
  interchangeable in this way
• can exchange points and lines with no
  ramifications
• therefore, all statements about points can be
  replaced by statements about lines
• e.g., P.L 0 P lies on L or L intersects P
• e.g., P1xP2 maps from two points to a line, and
  L1xL2 maps from two lines to a point
• in general, points and hyperplanes in n-space are
  dual