Optical Snow and the Aperture Problem

1
Optical Snow and the Aperture Problem

• Richard Mann
• School of Computer Science
• University of Waterloo

Michael Langer School of Computer
Science McGill University

2
Optical flow
J.J. Gibson, The Senses Considered As Perceptual
Systems, 1966
3
Layered motion
e.g. occlusions, transparency
4
Motion beyond layers
e.g. falling snow
5
Optical snow
6
Optical Snow
Lateral egomotion in a 3D cluttered scene
7
Optical snow
8
Overview of Talk

• background
• - Fourier analysis of optical snow
• - how to estimate direction of optical snow?
• (Langer and Mann, ICCV
  01)

9
Overview of Talk

• background
• - Fourier analysis of optical snow
• - how to estimate direction of optical snow?
• (Langer and Mann, ICCV
  01)
• new stuff
• - aperture problem

10
Fourier analysis of image translation
(Watson Ahumada 85)
f
t
t
f
y
f
x
If image patch is translating with velocity (v
, v ) then all power lies on a plane
x y
v f v f f 0
x x y y t
11
Optical Snow
Image velocities are (a v , a v )

x y
12
Fourier analysis of optical snow
f
t
f
t
bowtie
f
x
a v f a v f f 0
x x y y t
13
Bowtie of falling spheres

f
t
f
T

14
Bowtie of bush
f
t
f

T
15
Q How to compute motion direction ?A rotate
a wedge and measure power
Minimum of power in wedge occurs when wedge is
aligned with the bowtie.
16
Computing the direction of motion
motion direction
minimum of power
The motion direction is perpendicular to the
direction of minimum of power.
17
Aperture Problem
normal direction
Vertically falling cylinders appear to move in
normal direction.
18
Aperture Problem
true motion direction
normal direction (max of power)
19
Aperture problem
?
falling ellipsoids
same power but random phase
20
Summary

• Optical snow a new motion category
• Fourier-based method for detecting direction of
  motion
• Analysis of aperture problem