Multilinear Systems and Invariant Theory

1
Multi-linear Systems and Invariant Theory in
the Context of Computer Vision and
Graphics Class 3 Infinitesimal
Motion CS329 Stanford University
Amnon Shashua
2
Material We Will Cover Today

• Infinitesimal Motion Model

• Infinitesimal Planar Homography (8-parameter
  flow)

• Factorization Principle for Motion/Structure
  Recovery

• Direct Estimation

3
Infinitesimal Motion Model
Rodriguez Formula
4
Infinitesimal Motion Model
5
Reminder
Assume
6
Infinitesimal Motion Model
Let
7
Infinitesimal Motion Model
8
Infinitesimal Planar Motion (the 8-parameter flow)
9
Infinitesimal Planar Motion (the 8-parameter flow)
10
Infinitesimal Planar Motion (the 8-parameter flow)
Note unlike the discrete case, there is no scale
factor
11
Reconstruction of Structure/Motion (factorization
principle)
Note
2 interchanges
1 interchanges
12
Reconstruction of Structure/Motion (factorization
principle)
13
Reconstruction of Structure/Motion (factorization
principle)
Let
be the flow of point i at image j (image 0 is
ref frame)
14
Reconstruction of Structure/Motion (factorization
principle)
Given W, find S,M
(using SVD)
Let
for some
Goal find
such that
using the structural constraints on S
15
Reconstruction of Structure/Motion (factorization
principle)
Goal find
such that
using the structural constraints on S
Columns 1-3 of S are known, thus columns 1-3 of A
can be determined.
Columns 4-6 of A contain 18 unknowns
eliminate Z and one obtains 5 constraints
16
Reconstruction of Structure/Motion (factorization
principle)
Goal find
such that
using the structural constraints on S
Let
because
17
Reconstruction of Structure/Motion (factorization
principle)
because
Each point provides 5 constraints, thus we need 4
points and 7 views
18
Direct Estimation
The grey values of images 1,2
Goal find u,v per pixel
19
Direct Estimation
Assume
We are assuming that (u,v) can be found by
correlation principle (minimizing the sum of
square differences).
20
Direct Estimation
Taylor expansion
21
Direct Estimation
gradient of image 2
image 1 minus image 2
22
Direct Estimation
aperture problem
23
Direct Estimation
Estimating parametric flow
Every pixel contributes one linear equation for
the 8 unknowns
24
Direct Estimation
Estimating 3-frame Motion
Combine with
25
Direct Estimation
Let
26
Direct Estimation
image 1 to image 2
image 1 to image 3
Each pixel contributes a linear equation to the
15 unknown parameters
27
Direct Estimation Factorization
Let
be the flow of point i at image j (image 0 is
ref frame)
28
Direct Estimation Factorization
29
Direct Estimation Factorization
Recall
30
Direct Estimation Factorization
31
Direct Estimation Factorization
Rank6
Rank6
Enforcing rank6 constraint on the measurement
matrix
removes errors in a least-squares sense.
32
Direct Estimation Factorization
Once U,V are recovered, one can solve for S,M as
before.