Optical Flow - PowerPoint PPT Presentation

About This Presentation
Title:

Optical Flow

Description:

Definitions. The optical flow is a velocity field in the image which transforms one image into the next image in a sequence [ Horn&Schunck ] The motion field – PowerPoint PPT presentation

Number of Views:147
Avg rating:3.0/5.0
Slides: 47
Provided by: educ5460
Category:

less

Transcript and Presenter's Notes

Title: Optical Flow


1
Optical Flow
2
Optical Flow
  • Distribution of apparent velocities of movement
    of brightness pattern in an image

3
Optical Flow
  • Perspective projection (pinhole camera)
  • Instantaneous 2D velocity

4
Definitions
The optical flow is a velocity field in the image
which transforms one image into the next image in
a sequence HornSchunck


frame 2
frame 1
flow field
The motion field is the projection into the
image of three-dimensional motion vectors
HornSchunck
5
Ambiguity of optical flow
Frame 1
6
Determining Optical Flow
  • Berthold K.P. Horn and Brian G. Rhunck
  • Artificial Intelligence Laboratory, MIT.

7
Optical Flow
  • Optical flow field Estimate the 2D motion field,
    which are the 2D velocities for all visible
    points
  • Two key problems
  • Determine what image property to track
  • Determine how to track it

8
Brightness Constancy
u
frame t1
v
frame t
9
Brightness Constancy
  • Track points of constant brightness, assuming
    that surface radiance is constant over time
  • Brightness constancy is often violated by Mother
    Nature, so the resulting optical flow field is
    sometimes a very poor approximation to the 2D
    motion field

10
Optical Flow
  • The optical flow cannot be computed at a point
    independently of neighboring points without
    introducing additional constraints

11
Brightness Constancy Constraint
  • The brightness of a particular point in the
    pattern is constant
  • Cannot determine the flow velocity
  • locally without additional
    constraints

Constraint line
12
Brightness Constancy Constraint
  • Normal Velocity
  • No constraint when the gradient magnitude is zero

13
Optical Flow Assumptions
Slide from Michael Black, CS143 2003
14
Optical Flow Assumptions
Slide from Michael Black, CS143 2003
15
Smoothness Constraint
  • Constraint Assumption
  • Neighboring points on the objects have similar
    velocities and
  • The velocity field of the brightness patterns in
    the image varies smoothly almost everywhere
  • Discontinuities in flow can be expected where one
    object occludes another
  • Algorithms based on smoothness constraint are
    likely to have difficulties with occluding edges

16
Smoothness Constraint
  • Minimize the square of the gradient magnitude of
    the optical flow velocity
  • Sum of the squares of Laplacians of x- and y-
    components of the flow

17
Estimate the Partial Derivatives
18
Estimate the Laplacian of Flow Velocities
19
Minimization
  • Minimize the sum of errors in rate of change of
    image brightness
  • And measure of departure from smoothness
  • Total error to be minimized

20
Minimization
  • Good (u,v) occurs at

21
Difference of Flow from Local Average
  • (u,v) lies in the direction towards the
    constraint line along a line that perpendicular
    to it
  • a2 plays significant role when Ex, Ey is small
    roughly equal to the expected noise in the
    estimate of

22
Constrained Minimization
  • Minimizing
  • While maintaining
    0
  • The (u,v) here is just the intersection on the
    constraint line

23
Iterative Solution
24
Optical Flow 1D Case
Brightness Constancy Assumption
25
Tracking in the 1D case
26
Tracking in the 1D case
27
Tracking in the 1D case
Iterating helps refining the velocity vector
Converges in about 5 iterations
28
Algorithm for 1D tracking
29
Iterative Solution
  • Very costly to solve these equations
    simultaneously
  • Iterative algorithms, such as Gauss-Seidel method

It is interesting to note that the new estimates
at a particular point do not depend directly on
the previous estimate at the same point
30
Fill in Uniform Region
  • Uniform region brightness gradient is zero
  • No local information to constrain the vecocity
  • Fill in uniform region
  • Given the values on region boudnary
  • Solve the Laplace equation under Dirichlet
    boundary condition

31
Tightness of Constraint
  • In general, the solution is most accurately
    determined in the region where the brightness is
    not too small and varies in direction from point
    to point
  • Constraint information is available in a relative
    small neighborhood

32
Choice of Iterative Scheme
  • How to interlace the iterations with the time
    step?
  • Get stable values before going to next frame
  • Only one iteration per time-step gives good
    initial guess
  • Ability to deal with more images per unit time
  • Better estimation in uniform regions because of
    the noise in measurements of images will be
    independent and tend to cancel out

33
Experiment
  • Very low resolution video, say, 3232
  • A rotating ball with smoothly varying patterns,
    but no shading

34
Results I
  • Simple linear translation of the entire pattern
  • Estimate between two images
  • Showing the velocity
  • Few changes occur after 32 iterations when errors
    10

35
Results I
1 time step
4 time steps
36
Results I
16 time steps
64 time steps
37
Results II
  • Only one iteration per time-step
  • More rapid
  • Few changes after 16 iteration when errors 7

38
Results II
1 time steps
4 time steps
39
Results II
16 time steps
64 time steps
40
Results III
  • Simple rotation and contraction of the brightness
    pattern

41
Result III
initial
32 time steps
42
Result IV
  • Rigid body motion
  • Laplacian for one of the velocity components
    becomes infinite on the occluding bound
  • Reasonable results are still obtained with finite
    velocities

43
Results IV
Rotating cylinder
Rotating sphere
44
Results IV
Exact flow pattern for a rotating cylinder and
rotating sphere
45
Summary
  • A method for computing optical flow from a
    sequence of images
  • Two components of flow velocity
  • Additional constraint smoothness
  • Iterative method

46
Optical Flow Research Timeline
HornSchunck
LucasKanade
1981
1992
1998
now
BenchmarkGalvin et.al.
BenchmarkBarron et.al.
Seminal papers
A slow and not very consistent improvement in
results, but a lot of useful ingredients were
developed
47
http//people.csail.mit.edu/celiu/ECCV2008/zz
Write a Comment
User Comments (0)
About PowerShow.com