3D Computational Vision CSc 83029

1
3-D Computational VisionCSc 83029

• Optical Flow Motion
• The Factorization Method

2
Optical Flow Motion

• Finding the movement of scene objects from
  time-varying images.
• Motion Field
• Optical Flow
• Computing Optical Flow

3
(No Transcript)
4
Computing Time-to-Impact t
v
l(t)f L/D(t)
L
l(t)
D0
f
D0-vt
5
Computing Time-to-Impact t
v
l(t)f L/D(t)
L
l(t)
D0
f
D0-vt
l(t) / l(t) t Quantities measured from image
sequence.
6
Sudden change in viewing position/direction Har
d to compute motion field/optical flow.
7
Views from a sequence of spatially close
viewpoints Motion Field/ Optical Flow
8
Sub problems of Motion Analysis

• Correspondence.
• Reconstruction.
• Segmentation.

9
Motion Field
2-D vector field of velocities of image points,
induced by relative motion between the viewing
camera and the observed points.
Image plane
f
v dt
ri
V dt
p
ro
P
Scene Point Velocity Vdro/dt Image Velocity
v dri/dt
10
Motion Field
2-D vector field of velocities of image points,
induced by relative motion between the viewing
camera and the observed points.
Image plane
f
v dt
ri
V dt
p
Perspective projection
ro
P
Velocity of point P as a function of translation
and rotation
Motion field equations
11
Motion Field
2-D vector field of velocities of image points,
induced by relative motion between the viewing
camera and the observed points gt SUM of 2
COMPONENTS
12
Motion Field
2-D vector field of velocities of image points,
induced by relative motion between the viewing
camera and the observed points gt SUM of 2
COMPONENTS

13
Special Case 1 Pure Translation
2-D vector field of velocities of image points,
induced by relative motion between the viewing
camera and the observed points gt SUM of 2
COMPONENTS

14
Pure Translation Radial Motion Field
p0(x0,y0)
15
Pure Translation Radial Motion Field

• Tz lt 0 FOCUS OF EXPANSION.
• Tz gt 0 FOCUS OF CONTRACTION.
• Tz 0 PARALLEL MOTION FIELD.
• Vanishing point (epipole) p0.

p0(x0,y0)
16
Special Case 2 Moving Plane
n
P
Plane moves n,d are functions of time. Motion
field?
17
Special Case 2 Moving Plane
n
P
Plane moves n,d are functions of time. Motion
field?
18
Special Case 2 Moving Plane
n
P
Plane moves n,d are functions of time. Motion
field?
Motion field quadratic polynomial in (x,y,f) at
any time t. The same motion field can be
produced by 2 different planes undergoing 2
different 3-D motions.
19
Motion Parallax
The relative motion field of two instantaneously
coincident points does not depend on the
rotational component of the motion
p0
20
The notion of Optical Flow
Optical Flow Estimation of the motion field from
a sequence of images.
21
Optical Flow
22
Optical Flow
(xudt,yvdt) udx/dt vdy/dt
(x,y)
t
tdt
Image brightness constancy equation E(x,y,t)E(x
udt, yvdt, tdt)
or
23
The Aperture Problem
The component of the motion field in
the direction orthogonal to the spatial image
gradient is not constrained by the image
brightness constancy equation.
Aperture problem
Image brightness constancy equation E(x,y,t)E(x
udt, yvdt, tdt)
1 constraint 2 unknowns
24
Optical Flow
At each point we know dE/dx dE/dy and dE/dt.
How can we obtain dx/dt and dy/dt?
25
Computing Optical Flow
Assumption The motion field is well approximated
by a constant vector field within any small patch
of the image plane.
provides one constraint
Each
Solution is
26
Computing Optical Flow
Assumption The motion field is well approximated
by a constant vector field within any small patch
of the image plane.
Q
Minimize
27
Computing Optical Flow
Assumption The motion field is well approximated
by a constant vector field within any small patch
of the image plane.
Q
Minimize
gtSolve the linear system
28
Computing Optical Flowglobal approach
Schunck and Horn 81
Minimize the error in the image brightness
constancy constraint.
29
Computing Optical Flowglobal approach
Schunck and Horn 81
Minimize the error in the image brightness
constancy constraint.
Minimize the deviation from smoothness of the
motion vectors.
30
Computing Optical Flowglobal approach
Schunck and Horn 81
Minimize the error in the image brightness
constancy constraint.
Minimize the deviation from smoothness of the
motion vectors.
Find solution by minimizing
where lambda weights the smoothness term.
31
Optical Flow
32
Optical Flow
33
Optical Flow
34
Optical Flow
35
Optical Flow
If the scene is planar the motion Is described by
Using
Solve for the 8 unknowns a, b, c, d, e, f, g and
h.
36
Tracking Rigid Bodies
B
Random Sampling Algorithm Step 1 Find
corners Step 2 Search for correspondence Step 3
Randomly choose small set of matches. Step 4
Estimate F matrix Step 5 Find total number of
matches close to epipolar lines Step 6 Go to
step 3 Step 7 Choose F with largest number of
matches
37
(No Transcript)
38
Structure and Motion Recovery from Video
1. Use multiple image stream to compute the
information about camera motion and 3D structure
of the scene 2. Tracking image features over time
Tracked Features
Original sequence
From Jana Kosecka
39
Structure and Motion Recovery from Video
Computed model 3D coordinates of the feature
points
Original picture
From Jana Kosecka
40
Factorization Method
ji
ki
ii
FRAMES i1N
41
Factorization Method
Pj
ji
ki
(xij,yij)
ii
FRAMES i1N
World Points j1n
42
Factorization Method
Pj
Z
ji
ki
(xij,yij)
Ti
ii
FRAMES i1N
World Points j1n
Y
X
World Reference Frame
43
ASSUMPTIONS The camera model is
orthographic! The positions of n image points
have been tracked.
Pj
Z
ji
ki
(xij,yij)
Ti
ii
FRAMES i1N
World Points j1n
Y
X
World Reference Frame
44
Measurement Matrix
2N (Frames)
n points per frame
Registered Measurement Matrix
2N (Frames)
n points per frame
45
Factorization Method
World Points j1n
Pj
3D Centroid
Z
ji
ki
Ti
ii
FRAMES i1N
Y
X
2D Centroid
World Reference Frame
46
Rank Theorem
2N (Frames)
n points per frame
R Rotation Matrix S Shape Matrix
47
Rank Theorem
2N (Frames)
n points per frame
R Rotation Matrix S Shape Matrix Rank of
is 3.
48
The algorithm
2N (Frames)
n points per frame
Decompose into R and S. Is the decomposition
unique? Translation estimation?
49
(No Transcript)