CS 395/495-26: Spring 2004

1
CS 395/495-26 Spring 2004

• IBMRProjective 2D DLT
• Jack Tumblin
• jet_at_cs.northwestern.edu

2
Recall Projective Transform H
2D image (x,y) ?? Homog. coords x,y,wT x
w1

• Apply the 3x3 matrix H x Hx

Homog. coords x x,y,wT w1 ?? 2D
image (x,y)
x
x
x2
(x/w,y/w)
y
x
y
y
y
x3
x
x
x
y
x1
(x/w,y/w)
x
3
Recall Projective Transform Hx x

H defines point pairs (x,x) (correspondences)
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3
x y w
Input (or output) image is on central plane
x1 x2 x3


x x1, x2, x3 notation is confusing for
multiple points! Instead, lets use
x,y,w (so we can use subscripts to number the
point pairs x,x)
x y
Write R2 expressions(for each x,x pair)
x (h31 x h32 y h33) (h11 x h12 y h13)
Rearrange, solve as a matrix problem
y (h31 x h32 y h33) (h11 x h12 y h13)
4
Recall Projective Transform Hx x

H defines point pairs (x,x) (correspondences)
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3
x y w
Input (or output) image is on central plane
x1 x2 x3


let w1
x y
Write R2 expressions(for each x,x pair)
x (h31 x h32 y h33) (h11 x h12 y h13)
Rearrange, solve as a matrix problem
y (h31 x h32 y h33) (h11 x h12 y h13)
5
Recall Projective Transform Hx x

Finding H from point pairs (correspondences)
x1 x2 x3
h11 h12 h13 h21 h22 h21 h31 h32 h33
x1 x2 x3
x y 1
Input (or output) image is on central plane
x1 x2 x3


x y
Write R2 expressions(for each x,x pair)
x (h31 x h32 y h33) (h11 x h12 y h13)
Rearrange, solve as a matrix problem
y (h31 x h32 y h33) (h11 x h12 y h13)
6
Naïve Method

• 4 point correspondence
• Book shows
• Rearrange known vector (dot) unknown vector
• stack, solve for null space
• But this assumes x1x, x2y, x3w1,
• There is a better way (DLT)

x (h31 x h32 y h33) (h11 x h12 y h13)
y (h31 x h32 y h33) (h11 x h12 y h13)
x y 1 0 0 0 -xx -xy -x
0 0 0 0 x y 1 -yx -yy -y
7
Direct Linear Tranform (DLT)

• Naive method Solve Hx x 0
• Requires constant scale (x31 always)
• compare
• A BETTER WAYDLT method Solve Hx ? x 0
• Accepts any scale (x3w?1 OK)

h11 h12 h13 h21 h22 h23 h31 h32 h33
0 0 0 x y 1
-yx -yy -y x y
1 0 0 0 -xx -xy
-x
0 0 0 -wx -wy -ww yx
yy yw 0 wx wy ww
0 0 0 -xx -xy -xw
8
Direct Linear Tranform (DLT)

• Naive method Solve Hx x 0
• Requires constant scale (x31 always)
• Compare to DLT
• DLT method Solve Hx ? x 0
• Accepts any scale (x3w?1, x3w?1)

h1 h2 h3
0 0 0 x y 1
-yx -yy -y x y
1 0 0 0 -xx -xy
-x
0 0 0 -wx -wy -ww yx
yy yw 0 wx wy ww
0 0 0 -xx -xy -xw
0T -wi xiT yI xiT wi
xiT 0T -xi xiT
9
Direct Linear Tranform (DLT)

• DLT method Solve H ? x 0
• Accepts any scale, any point (x3w?1 is OK)
• Pure, Compatible uses only P2 terms.
• Much better suited to error measurements.
• Subtlety
• Wont divide-by-zero if w0 or h330 (it
  happens!)
• has a 3rd row it is not degenerate if/when w0
• OK to use it(Solve 8x12)

0T -wi xiT yI xiT wi
xiT 0T -xi xiT 0 yi xiT
-xi xiT 0T
h1 h2 h3
10
STOP Learn about SVDs
11
Matrix Math SVDs

• Matrix multiply is a change of coordinates
• Rows of A coordinate axis vectors
• Ax a dot product for each axis
• input space?output space
• SVD factors matrix A into 3 parts
• U columns orthonormal axes for OUTPUT
• V columns orthonormal axes for INPUT
• S scale factors V?U

svd(A) U S VT
12
Actual Robustness

• Vectorizing (Flatten, Stack, Null Space method)
  works for almost ANY input! (Points, lines,
  planes, , ?, conics, quadrics, cross-ratios,
  vanishing points, twisted cubics
• Key to DLT formulation (Hx ? x) 0
• Rearrange as dot product (known)(unknown) 0
• Be careful to have ENOUGH constraints
• tricky when you mix types points, lines, (pg
  75)

0
measured output
measured inputtransformed
?
13
Deceptions false Robustness

• Suppose we have 4 pt-correspondences
• Use DLT to write 8x9 (or 12x9) matrix A Ah0
• Solve for h null space. ALWAYS gives H matrix
• But what if points are bad / fictional?
• 3 collinear input pts, crooked out IMPOSSIBLE!
• Yet we get an H solution! Why?
• A matrix rank is 7 or 6? rank 1 H result(s)
• Null Space of A may contain gt1 H solution!
• Degenerate H solution of form aH1 bH2
• Answer SVD ranks A reject bad point sets.

14
Most Important SVD Ideas

• SVD solves Null-Space problems Ax 0 (flatten,
  stack, find null space to solve)
• MANY problems can be written this way
• Expandable! --more data points? tall,
  overconstrained A matrix --SVD finds optimal
  least squares solution.
• Robustness bane of all inverse
  methodsAccuracy vs. of measurements (more
  isnt always better! Outliers...) Data-rich
  images quantity easier than quality
  more is easier than better)
• ?What links measurement errors??H errors?
• ?How can more measurements reduce error?

15
How else can we use DLT?

• I know this building consists of MANY parallel
  and perpendicular lines, corners, regularly
  spaced points.
• How many different ways can we find to rectify
  this image to get this one?

16
Direct Linear Tranform (DLT)

• Naïve method Solves Hx x 0
• Uses constant scale (x3w1 in measurement)
• Compare to DLT
• DLT method Solve Hx ? x 0
• (a different identity x ? x 0)
• Accepts any scale (x3w?1 OK, xx3)

h11 h12 h13 h21 h22 h23 h31 h32 h33
0 0 0 x y 1
-yx -yy -y x y
1 0 0 0 -xx -xy
-x
0 0 0 -wx -wy -ww yx
yy yw 0 wx wy ww
0 0 0 -xx -xy -xw
17
Direct Linear Tranform (DLT)

• Naïve method Solve Hx x 0
• Requires constant scale (x31 always)
• compare
• DLT method Solve Hx ? x 0
• Accepts any scale (x3w?1, x3w?1)

h1 h2 h3
0 0 0 x y 1
-yx -yy -y x y
1 0 0 0 -xx -xy
-x
0 0 0 -wx -wy -ww yx
yy yw 0 wx wy ww
0 0 0 -xx -xy -xw
0T -wi xiT yI xiT wi
xiT 0T -xi xiT
18
Direct Linear Tranform (DLT)

• DLT method Solve H ? x 0
• Accepts any scale, any point (x3w?1 is OK)
• Pure and Compatible -- P2 terms only, no R2
• Much better suited to error measurements.
• Subtlety
• Wont divide-by-zero if w0 or h330 (it
  happens!)
• has a 3rd row it is not degenerate if w0
• OK to use it(Solve 8x12)

0T -wi xiT yI xiT wi
xiT 0T -xi xiT 0 -yi xiT
-xi xiT 0T
h1 h2 h3
19
Adding More Measurements

• How can we use gt4 point correspondences?
• Easy
• Add more rows to our 8x9 matrix A Ah 0
• Use SVD to find Null space (Always gives an
  answer!)
• Result Least squares solution errorAh ?
• minimizes A h 2 ?i ?2i where,?i is
  error for i-th pt. correspondence
• ?i Hxi ? xi 2 (2 rows of A)h 2
  algebraic distance
• Algebraic Distance ? No geometric meaning!

20
Adding More Measurements

• 2DAlgebraic Distance ? No geometric meaning!
• 2D Geometric Distance d(a,b)2 is Better
  measurable length in input or output space
• if a (a1 a2 a3) and b (b1 b2 b3), then
  define
• Turns out that

d(a,b)2 dalgebraic(a,b)
a3b3
(Not very surprising)
21
Adding More Measurements

• Iterative Strategy
• Overconstrain the answer H
• Collect extra measurements (gt4 point pairs, etc.
  )
• expect errors call them estimates x
• Compute a 1st solution (probably by SVD)
• Compute error d(Hx, x)2, and use this to
• Tweak answer H and estimates x
• Compute new answer
• Stop when error lt useful threshold

22
Using Estimates

• Simplest one image transfer method
• Assume inputs are a perfect test pattern
• only output pts are in error
• Adjust output estimates x until d(Hx, x)2 ?0
• (note we re-compute H as x changes)
• Better Symmetric transfer method
• Assume BOTH input x and output x has error.
• Adjust BOTH input and output estimates x
  x(note we also re-compute H as x, x change)
• Stop when d(Hx,x)2 d(H-1x, x)2 is nearly 0

23
END