Motion / Optical Flow II

1
Motion / Optical Flow II

• Estimation of Motion Field
• Avneesh Sud

2
Outline

• Motion Field Optical Flow
• Constraints
• Methods of Estimating Motion Field
• Differential Techniques
• Least Squares
• Horn-Schunck Algorithm
• Comments
• Results by Miguel

3
Motion Field

• 2-D projection of velocities of the image points,
  induced by the relative motion between camera and
  scene
• Not directly measurable from an image

4
Optical Flow

• A vector field subject to Image Brightness
  Constancy Equation (IBCE)
• Apparent motion of the image brightness pattern

5
Optical Flow Vs. Motion Field

• Optical flow does not always correspond to motion
  field
• Optical flow is an approximation of the motion
  field. The error is small at points with high
  spatial gradient under some simplifying
  assumptions (Trucco p195)

6
Ambiguity in Local Optical Flow

• Correspondence between points on isobrightness
  contours?
• A constant patch of uniform brightness multiple
  optical flow solutions

• Use additional constraints !

7
IBCE Revisited
Assume the image intensity of each visible scene
point is unchanging over time
Also known as the Horn and Schunck optical flow
constraint equation
8
Aperture Problem

• Constraint corresponds to a line in velocity
  space
• Given local info, can determine component of
  optical flow vector only in direction of
  brightness gradient

9
Estimating Motion Field

• Differential techniques based on spatial
  temporal variations of the image at all pixels
• Matching (feature-based) techniques rely on
  special image points (features) and track them
  through frames

10
Differential Techniques Least Squares

• Optical Flow Algorithm (Trucco, p196)
• For each pixel p
• Must satisfy (?E)v Et 0
• Assumption This equation holds in the
  neighborhood of p with constant v
• Write this equation for a small (typically 5x5)
  patch centered at p
• Then we find least square fit of v - this is the
  calculated optical flow for pixel p

11
Least Squares Assumptions

• Assumed that ICBE holds in the neighborhood of p
  with constant v
• In case of rigid motion, the motion field of a
  moving plane is a quadratic polynomial in the
  coordinates (x, y, f) of the image points.
  (Trucco p 187)
• Therefore, if the object is smooth rigid, we
  can assume the motion field varies smoothly

12
Differential Techniques Horn- Schunck Algorithm

• Optical flow constraint equation gives the
  component in direction of brightness gradient
• Additional Constraint smoothness of optical
  flow! Neighboring surface points of a rigid
  object have approximately same local displacement
  vectors

13
Horn-Schunck Algorithm

• Two criteria
• Optical flow is smooth, Fs(u,v)
• Small error in optical flow constraint equation,
  Fh(u,v)
• Minimize a combined error functional
• Fc(u,v) Fs(u,v) ? Fh(u,v)
• ? is a weighting parameter
• Variation calculus gives a pair of second order
  differential equations that can be solved
  iteratively

14
Horn-Schunck Algorithm Discrete Case

• Derivatives (and error functionals) are
  approximated by difference operators
• Leads to an iterative solution

15
Intuition of the Iterative Scheme
The new value of (u,v) at a point is equal to the
average of surrounding values minus an adjustment
in the direction of the brightness gradient
16
Horn - Schunck Algorithm
17
Horn-Schunck Algorithm
begin for j 1 to N do for I 1 to M do
begin calculate the values Ex(i,j,t), Ey(i,j,t)
and Et(i,j,t) using a selected approx
formula initialize the values u(I,j) and v(i,j)
to zero end for choose a suitable weighting
value ? choose a suitable number n0 ? 1 of
iterations n 1 while n ? n0 do begin for j
1 to N do for i 1 to M do begin compute
u, v, ? update u(i,j), v(i,j) end for n n
1 end while end
18
Comments

• There are reliable methods for estimating optical
  flow.
• Optical flow is a vector field, from which the
  motion field can be estimated under certain
  conditions.
• Horn Schunk Algorithm as presented does not
  handle discontinuities (silhouettes) well.
• Some real-world results!

19
References

• Introductory Techniques for 3-D Computer Vision,
  Emanuele Trucco and Allessandro Verri, Prentice
  Hall, 1998. Chapter 8
• Robot Vision, B.K.P. Horn, MIT Press 1986.
  Chapter 12
• Computer Vision Three-Dimensional Data from
  Images, Reinhard Klette, Karsten Schluns, Andreas
  Koschan, Springer 1998. Topic 5.2

20
References

• MikeTalk A Talking Facial Display Based on
  Morphing Visemes, Tony Ezzat and Tomaso Poggio,
  Proceedings of the Computer Animation Conference
  Philadelphia, PA, June 1998
• Optical Flow - CVonline Motion and Time Sequence
  Analysis (http//www.dai.ed.ac.uk/CVonline/motion.
  htm)