CS 395/495-26: Spring 2004

1
CS 395/495-26 Spring 2004

• IBMR 2-D Projective Geometry
• --Introduction--
• Jack Tumblin
• jet_at_cs.northwestern.edu

2
Recall Scene Image

• Light 3D Scene
• Illumination, shape, movement, surface BRDF,

2D Image Collection of rays through a point
Image Plane I(x,y)
Position(x,y)
Angle(?,?)
GOAL a Reversible Mapping Scene angles?? image
positions
3
View Interpolation How?

• LATER 3D Projection(Hartley,Zisserman Chapter 2)

Find new views of that scene
From a 3D scene
view1
view2
4
View Interpolation How?

• BUT FIRST, the simpler caseHartley, Zisserman
  Chapter 1 2D Projection

Find new views of that image
From a flat 2D image,
view1
view2
5
Answer 2D Homogeneous Coords

• Chapter 1 2D Projection

From a flat 2D image,
Find new views of that image
view1
view2
2D Homogeneous (x1 x2 x3) coordinates in P2
Cartesian (x,y) coordinates in R2
6
Overview

• Chapter 1 any 2D plane, as seen by
  any 3D camera view (texture mapping
  generalized)
• Homogeneous Coordinates are Wonderful
• Everything is a Matrix
• Conics (you can skip for now)
• Aside interpolation of pixels and points
• Transform your Point of View H matrix
• Parts of H useful kinds of Transforms

7
2-D Homogeneous Coordinates

• WHY? makes MUCH cleaner math!
• Unifies lines and points
• Puts perspective projection into matrix form
• No divide-by-zero defines horizon, infinities

But in P2, write same point x as where
in R2, write point x as
x1 x2 x3
x y
x x1 / x3, y x2 / x3, x3 anything non-zero!
(but usually defaults to 1)
(x,y)
8
Useful 3D Graphics Ideas

• Every Homog. point (x1,x2,x3) describes a 3D ray
• Phantom dimension x3 is the z-buffer value
• 3D?2D hard-wired
• (Fig 1.1, pg 8)
• Homogeneous coords allows translation
  matrix (true. But there is
  more!)

Image Plane (x,y,-1)
(xmax, ymax, -1)
(0,0,-1)
An Ideal Point at (0,0,0) Center of Projection
x y 1
1 0 3 0 1 5 0 0 1
x3 y5 1
9
2-D Homogeneous Coordinates

• WHAT?! Why x3? Why default value of 1?
• Look at lines in R2
• line all (x,y) points where
• scale by k ?? no change
• Using x3 for points UNIFIES notation
• line is a 3-vector named l
• now point (x,y) is a 3-vector too, named x

ax by c 0
kax kby kc 0
ax by c 0
a b c
0
x1 x2 x3
xT.l 0
10
2D Homogeneous Coordinates

• Important Properties 1(see book for details)
• 3 coordinates, but only 2 degrees of
  freedom(only 2 ratios (x1 / x3), (x2 / x3) can
  change)
• DUALITY points, lines are interchangeable
• Line Intersections point
• (a 3D cross-product)
• Point Intersections line
• Projective theorem for lines ?? theorem for
  points!

l1 ? l2 x
x1 ? x2 l
11
Homogeneous Coordinates

• Important Properties 2(see book for details)
• Neatly Sidesteps divide-by-zero
• in (x,y) space (R2)
• Remember, we store (x1 ,x2, x3)T, then compute
  (x1 / x3), (x2 / x3) only if OK.
• OK, yeah, sure, but so what? ...

12
Homogeneous Coordinates

• Why is deferring divide-by-zero so important?
• Cleanly Defines all Points at Infinity or
  Ideal Points (x,y,0)T
• outermost pixels (/-?, /-?) of an infinite
  image plane ((x,y), or R2)
• (x,y,0) is an entire plane of points in the
  (semi-bogus) 3D space (x1 ,x2, x3)T space
• but it is a circle around infinity in the
  (legitimate) P2 projective space (x1 /x3, x2
  /x3)T
• Note! Center of Projection is an ideal point.
  (Do outer limits of image plane wrap around to
  that point?!?!)

13
Homogeneous Coordinates

• Why is deferring divide-by-zero so important?
• Cleanly defines one Line at infinity l? ?
  (0,0,1)T, or 0x 0y 10
• All ideal points are on l? proof?
  l?(x1,x2,0)T 0
• Any line l intersects with l? line at an ideal
  point
• Two parallel lines l and l always meet at an
  ideal point Let l (a,b,c)T and l (a,b,c)T
  (page 7)

14
Homogeneous Coordinates

• Important Properties 3(see book for details)
• View Interpolation (Projective Transform)
• Central image plane (x,y,-1)T
• Choose a known point x
• Apply 3x3 Matrix H to (x,y,-1)Tto make point on
  OTHER plane
• Ray through known point x pierces unknown point
  x

3D world plane
x
x
H x x
Image Plane (x,y,-1)
(0,0,-1)
h11 h12 h13 h21 h22 h21 h31 h32 h33

(0,0,0)
15
END
16
Useful 3D Graphics Ideas

• Every xy point describes a 3D ray in (x1,x2,x3)
• Phantom dimension x3 is the (lost)
  distance-to-camera
• Homogeneous coords allows matrix for 2-D
  translation
• (BUT a 3x3 transform in R3 Euclidean Space
  (x,y,z) cant translateonly rotate,scale, skew!)

Central Image Plane (x,y,1)
x y 1
1 0 3 0 1 5 0 0 1
x3 y5 1
x2
An Ideal Point (0,0,0) at Center of Projection
Ray
-x3
y
x1
x
x y z
1 0 3 0 1 5 0 0 1
xz yz z
17
2D Homogeneous Coordinates

• 3 coords, but only 2 degrees of freedom
  (2DOF)(x1 / x3), (x2 / x3)
• Ideal points (x,y,0)
• (every line hits the horizon at an ideal point)
  
• Line Intersections point
• (a 3D cross-product)
• Point Intersections line
• Duality P2 theorem for lines ?? P2 theorem for
  points!

l1 ? l2 x
x1 ? x2 l
18
2D Homogeneous Coordinates
Left-Handed use x3 or (x,y,-1) image ctr, or...

• 3 coords, but only 2 degrees of freedom
  (2DOF)(x1 / x3), (x2 / x3)
• Ideal points (x,y,0)
• Line Intersections point
• (a 3D cross-product)
• Point Intersections line
• Duality P2 theorem for lines ?? P2 theorem for
  points!

l1 ? l2 x
x1 ? x2 l
19
Projective Transform H
2D image (x,y) ?? Homog. coords x,y,1T x

• Apply the 3x3 matrix H x Hx

Homog. coords x x,y,1T ?? 2D image
(x,y)
x
x
x2
(x,y)
y
x
y
y
y
x3
x
x
x
y
(x,y)
x1
x
20
Recall Projective Transform H
Goal Find H for Image Rectification
2D image (x,y) ?? Homog. coords x,y,1T x

• Apply the 3x3 matrix H x Hx

Homog. coords x x,y,1T ?? 2D image
(x,y)
x
x
x2
(x,y)
y
x
y
y
y
x3
x
x
x
y
(x,y)
x1
x
World Plane
View Plane
21
Projective Transform H

H matrix Plane-to-Plane mapping What if H is
unknown, but we have four or more pairs of x,x
points? (see pg 15, Zisserman) x1, x2, x3, x4,
x1, x2, x3, x4
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3
H x x

x
x
Image Plane (x,y,-1)
(0,0,-1)
(0,0,0)
22
Projective Transform H

• Matrix H transforms Points
• Given a point x (x1,x2,x3) (or in R2
  (x1/x3, x2/x3))
• x H x
• And matrix H-T transform Lines
• Given a line l (a,b,c) in R2
  (ax by c 0) in P2 (ax1
  bx2 cx3 0)
• l H-T l
• Recall point x is on line l iff xTl 0

Jargon points transform covariantly lines
transform contravariantly
23
The bits and pieces of H

• H has 8 independent variables (DOF)
• Cartesian (3D) decomposition
• 2D Translation(tx, ty)3D Scale
  (sx,sy,sz)3D Rotation (?x,?y,?z)
• Computer Vision (Projective 2D) decomp.
• Isometry --3DOF (2D translate tx,ty 2D rotate
  ?z )
• Similarity --4DOF (add uniform scale s)
• Affine --6DOF (add orientable scale s?,/s,
  s??/s)
• Projective--8DOF (changes x3 3D-rotation-like)

24
The bits and pieces of H

• H has 8 independent variables (DOF)
• Computer Graphics method (3x3 matrix)
• 2D Translation(tx, ty)3D Scale
  (sx,sy,sz)3D Rotation (?x,?y,?z)
• Computer Vision method(2D projective)
• Isometry --3DOF (2D translate tx,ty 2Drotate
  ?z )
• Similarity --4DOF (add uniform scale s)
• Affine --6DOF (add orientable scale s?,/s,
  s??/s)
• Projective--8DOF (changes x3 3D-rotation-like)

Affects only x1,x2
25
The bits and pieces of H

• H has 8 independent variables (DOF), not 9!
• Decomposable into 3 useful parts (pg 22)
• H HS HA HP
• Similarity HS
• 2D translate, rotate, uniform scale only (4DOF)
• Affine HA
• non-uniform scale (2DOF)
• Projective HP
• Projective coupling for x3 (2DOF)

?
?
?
sR t 0T 1
I 0 vT v
K 0 0T 1
?
?
?
?
?
?
?
?
?
?
?
?
26
The bits and pieces of H

• H has 8 independent variables (DOF), not 9!
• Decomposable into 3 useful parts (pg 22)
• H HS HA HP
• Similarity HS
• 2D translate, rotate, uniform scale only (4DOF)
  sscale, ?translate details
• Affine HA
• non-uniform scale (2DOF)
• Projective HP
• Projective coupling for x3 (2DOF)

s ? cos ? -s sin ? tx s sin ? s ? cos?
ty 0 0 1
1 0 0 0 1 0 v1 v2 vS
1 k2 0 k3 1 0 0 0 1

27
Compare HA and HP

• (recall)H HS HA HP

L
(parallelogram)
HA
m
(K matrix, 2DOF)
2D image space
(square)
L
m
2D world space
HP
(quadrilateral)
( v vector)
2D image space
28
1-D Projective Geometry

• (?Why? Well use it soon)
• A side view of 2D projective geometry
• Convert R1 scalar b to a 2-vector in P1
• As with P2, we can transform points
• (use various H2x2 s to change cyan lines below)

b 1
H2b
H2a
29
1-D Projective Geometry

• What is invariant here?
• all the positions, lengths and ratios change
  with each transform
• Answer Cross Ratio cross(a,b,c,d)
• let a , b
• cross a-bc-d a-cb-d
• cross(a,b,c,d) cross(a,b,c,d)

a
b
c
d
a
b
c
d
30
Image Rectifying Undo parts of H

• x H x where H HS HA HP
• GOAL Put world plane x into view plane
• Affine Rect. (find only HP (2DOF))
• Similarity Rect. (find only HA HP (6DOF))
• Full Rect. (find all of H (8DOF))
• METHODS (mix match?)
• Full 4-point correspondence
• Vanishing Point, Infinity line methods
• Other feature points, conics,

31
Image Rectifying Undo parts of H

• x H x where H HS HA HP
• GOAL Put world plane x into view plane
• Affine Rect. (find only HP (2DOF))
• Similarity Rect. (find only HA HP (6DOF))
• Full Rect. (find all of H (8DOF))
• METHODS (mix match?)
• Full 4-point correspondence
• Vanishing Point, Infinity line methods
• Conics and circular points

Other DOF? make assumptions, or ignore
32
Vanishing-Point Methods 1

• In 2D image, find two pairs of lines that are
  parallel in world space.

H
world space
2D image
33
Vanishing-Point Methods 1

• Find intersection (vanishing points) p1, p2
• (compute line intersections with 3D
  cross-products (see last lecture)
• Horizon line Ih connects p1, p2.

Ih
p1
p2
H
world space
2D image
34
Vanishing-Point Methods 1

• H-T transforms world infinity line l? ? (0,0,1)T
  to Ih
• Answer (see old book) find, copy the lh
  coordinates!
• Hp

1 0 0 0 1 0 lh1 lh2 lh3
Ih
p1
p2
H
world space
2D image
35
Vanishing-Point Methods 1

• Limitations
• Accurate point, line-finding can be tricky
• Can have high error sensitivity twitchy
• Only rectifies Hp-- but what if HA, HS needed?

Ih
p1
p2
H
world space
2D image
36
Vanishing-Point Methods 2

• No parallel lines?
• find 2D image line with a known distance ratio
• Use Cross-Ratio (in P1 along that line) to
• Find vanishing point distance at point d

Ih
p1
p2
d
c
H
b
a
b
c
d(at infinity)
a
2D image
world space
37
Projective Transform Hx x

Finding H from point pairs (correspondences)
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3
x y 1
Input (or output) image is on central plane
x1 x2 x3


x y
Write R2 expressions(for each x,x pair)
x (h31 x h32 y h33) (h11 x h12 y h13)
Rearrange, solve as a matrix problem
y (h31 x h32 y h33) (h11 x h12 y h13)
38
Projective Transform Hx x

Finding H from point pairs (correspondences)
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3
x y 1
Input (or output) image is on central plane
x1 x2 x3


!STOP! Trucco Book Does This Zisserman Book
calls it naïve, gives better method DLT
direct linear xfm.)
x y
Write R2 expressions (for each x,x pair)
x (h31 x h32 y h33) (h11 x h12 y h13)
Rearrange, solve as a matrix problem
y (h31 x h32 y h33) (h11 x h12 y h13)
39
Projective Transform H

• Finding H from point pairs (correspondences)
• We know that Hx x, and
• we know at least 4 point pairs x and x that
  satisfy it
• ATTEMPT 1plug chug make a matrix of x and
  x values

H x x
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3

H
x1
x1
x2
x3
x4
x2
x3
x4

40
END --Please download the starter code, compile
and run. --Then you can get started on Project 1
right away.