Modern Optical Flow Methods

1
Modern Optical Flow Methods

• Doug Gray
• 1/25/06

2
Motivation

• Dense optical flow
• Motion segmentation
• Object Detection
• Tracking
• Sparse optical flow
• Image stabilization
• Mosaicing
• Enhancement
• Super Resolution

3
Data Conservation (Direct Methods)

• Brightness consistency equation
• 1st order Taylor series approximation
• As a minimization problem
• Where u is now a vector and delta I is a local
  gradient vector

4
As a linear system

• As an over determined system we can use a least
  squares solution

5
Other flow models

• Constant
• Translation
• Affine
• Rotation, Translation Scale
• Quadratic
• Moving Plane

6
Generalized aperture problem

• In dense estimation, R, the estimation region
  must be
• Sufficiently large to constrain the solution
• Sufficiently small to detect motion boundaries
• What if the region has no texture?

7
Spatial Coherence

• Filling in effect
• Blurs motion boundaries
• Horn Schunck
• Solution uses the Euler-Lagrange equations

8
Multiple motion examples
9
Multiple Motion solution
10
Robust Statistics
11
Reformulated problem

• Apply robust ?-function to residuals
• How do we minimize?

12
Simultaneous Over-Relaxation

• Iterative solution
• ? is an overrelaxation parameter used to
  overcorrect in the next iteration
• T is an upper bound

13
Robust Method Results
14
Robust Method Results
15
Lucas Kanade

• K is a weighted region over which to estimate
• K is often a Gaussian for local estimation
• Determined system

16
Local/Global Method

• Horn Schunck
• Lucas Kanade

17
Spatial Approach

• Very similar to Horn Schunck
• Simply evaluate terms with data on a
  non-vanishing integration scale

18
Error metric

• Average angular error is computed as follows
• c indicates correct magnitude
• e indicates error magnitude

19
Results
20
Spatiotemporal Approach

• Integral over time and space
• Gaussian convolution is now spatiotemporal

21
Results
22
More Robust statistics

• Quadratic input, non-quadratic output
• Convex

23
Non-quadratic Approach

• More robust to noise and multiple motions
• The solution, once again, is given by
  Euler-LaGrange equations

24
Results
25
Results
26
Results
27
Confidence Measures

• Gradient magnitude has been proposed, but gives
  poor results
• Energy contribution of each pixel to the error
  function might work better
• Position in a cumulative histogram (i.e. a
  percentile)

28
Example/Results

• Top 20 of pixels
• Proposed measure vs. ideal based on average
  angular error

29
References, Thank you

• M. J. Black, and P Anandan, The robust estimation
  of multiple motions Parametric and
  piecewise-smooth flow fields, Computer Vision and
  Image Understanding, CVIU, 63(1), pp. 75-104,
  Jan. 1996.
• A. Bruhn, J. Weickert, C. Schnörr Lucas/Kanade
  meets Horn/Schunck Combining local and global
  optic flow methods. International Journal of
  Computer Vision, 2005