Multilinear Systems and Invariant Theory

1
Multi-linear Systems and Invariant Theory in
the Context of Computer Vision and
Graphics CS329
Amnon Shashua
2
Material We Will Cover Today

• The structure of 3D-gt2D projection matrix

• The homography matrix

• A primer on projective geometry of the plane

3
The Structure of a Projection Matrix
4
The Structure of a Projection Matrix
5
The Structure of a Projection Matrix
6
The Structure of a Projection Matrix
Generally,
is called the principle point
is aspect ratio
is called the skew
7
The Camera Center
such that
has rank3, thus
Why is
the camera center?
8
The Camera Center
Why is
the camera center?
Consider the optical ray
All points along the line
are mapped to the same point
is a ray through the camera center
9
The Epipolar Points
10
Choice of Canonical Frame
is the new world coordinate frame
We have 15 degrees of freedom (16 upto scale)
Choose W such that
11
Choice of Canonical Frame
Let
We are left with 4 degrees of freedom (upto
scale)
12
Choice of Canonical Frame
13
Choice of Canonical Frame
where
are free variables
14
Projection Matrices
Let
be the image of point
at frame number j
where
are free variables
are known
15
Family of Homography Matrices
Stands for the family of 2D projective
transformations
between two fixed images induced by a plane in
space
The remainder of this class is about making the
above statement intelligible
16
Family of Homography Matrices
Recall,
3D-gt2D from Euclidean world frame to image
world frame to first camera frame
Let K,K be the internal parameters of camera 1,2
and choose canonical
frame in which RI and T0 for first camera.
17
Family of Homography Matrices
Recall that 3rd row of K is
18
Family of Homography Matrices
Assume
are on a planar surface
19
Family of Homography Matrices
and
Let
where the matching pair
Image-to-image mapping
are induced by a planar surface.
20
Family of Homography Matrices
when
first camera frame
is the homography matrix induced by the plane at
infinity
21
Family of Homography Matrices
is the epipole in the second image
22
Relationship Between two Homography Matrices
is the projection of
onto the first image
23
Estimating the Homography Matrix
?
How many matching points?
p
p
4 points make a basis for the projective plane
24
Projective Geometry of the Plane
Equation of a line in the 2D plane
The line is represented by the vector
and
Correspondence between lines and vectors are not
1-1 because
represents the same line
The vector
does represent any line.
Two vectors differing by a scale factor are
equivalent. This equivalence class is called
homogenous vector. Any vector
is a representation of the equivalence class.
25
Projective Geometry of the Plane
A point
lies on the line (coincident with) which is
represented
by
iff
But also
represents the point
represents the point
The vector
does represent any point.
Points and lines are dual to each other (only in
the 2D case!).
26
Projective Geometry of the Plane
note
27
Lines and Points at Infinity
Consider lines
with infinitely large coordinates
which represents the point
All meet at the same point
28
Lines and Points at Infinity
The points
lie on a line
is called the line at infinity
The line
The points
are called ideal points.
A line
meets
at
(which is the direction of the line)
29
A Model of the Projective Plane
ideal point
is the plane
Points are represented as lines (rays) through
the origin
Lines are represented as planes through the origin
30
A Model of the Projective Plane
lines through the origin in


1-dim subspaces of
31
Projective Transformations in
The study of properties of the projective plane
that are invariant under a group of
transformations.
Projectivity
that maps lines to lines (i.e. preserves
colinearity)
Any invertible 3x3 matrix is a Projectivity
Let
Colinear points, i.e.
the points
lie on the line
is the dual.
is called homography, colineation
32
Projective Transformations in
perspectivity
A composition of perspectivities from a plane
to other planes
and back to
is a projectivity.
Every projectivity can be represented in this way.
33
Projective Transformations in
Example, a prespectivity in 1D
Lines adjoining matching points are concurrent
Lines adjoining matching points
(a,a),(b,b),(c,c) are not concurrent
34
Projective Transformations in
is not invariant under H
Points on
are
is not necessarily 0
Parallel lines do not remain parallel !
is mapped to
35
Projective Basis
A Simplex in
is a set of n2 points such that no subset
Of n1 of them lie on a hyperplane (linearly
dependent).
a Simplex is 4 points
In
Theorem there is a unique colineation between
any two Simplexes
36
Why do we need 4 points
Invariants are measurements that remain fixed
under colineations
of independent invariants d.o.f of
configuration - d.o.f of trans.
H has 3 d.o.f
Ex 1D case
A point in 1D is represented by 1 parameter.
4 points we have 4-31 invariant (cross ratio)
2D case H has 8 d.o.f, a point has 2 d.o.f thus
5 points induce 2 invariants
37
Why do we need 4 points
The cross-ratio of 4 points
24 permutations of the 4 points forming 6 groups
38
Why do we need 4 points
5 points gives us 10 d.o.f, thus 10-82
invariants which represent 2D
are the 4 basis points (simplex)
are determined uniquely by
Point of intersection is preserved under
projectivity (exercise)
uniquely determined