OSA 98 talk

1
Nonlinear visual coding from an
intrinsic-geometry perspective
E. Barth A. B. Watson NASA Ames Research
Center http//vision.arc.nasa.gov
Supported by DFG grant Ba 1176/4-1 to EB and
NASA grant 199-06-12-39 to ABW
2
Intrinsic dimensionality in 2D

• i0D constant in all directions
• i1D constant in one direction
• i2D no constant direction

3
i0D
FT
4
i1D
FT

• e.g. straight lines and edges, gratings

5
i2D
FT

• e.g. corners, line ends, curved edges and lines

6
i0D
i1D
i2D
7
(No Transcript)
8
i0D
i2D
i1D
9
Intrinsic dimensionality in 3D

• i0D constant in all (space-time) directions
• i1D constant in 2 directions
• i2D constant in one direction
• i3D no constant direction

10
i0D
FT
11
i1D
FT

• e.g. drifting spatial grating

12
i2D
FT

• e.g. drifting corner, flashed grating

13
i3D
FT

• e.g. flow discontinuities, flashed corners

14
Intrinsic dimensionality and motion

• FT of (rigid) motion signal is in a plane

15
The visual input as a hypersurface

• luminance

hypersurface
Visualization of surfaces is easier
16
Geometric view on intrinsic dimensionality
17
Curvature and motion

• (plane more than line but no volume)

18
The Riemann tensor R

• most important property of (hyper)surfaces
• measures the curvature of the (hyper)surface
• has 6 independent components in 3D
• vanishes in 1D.

19
The Riemann tensor components
are nonlinear combinations of derivatives, i.e.,
of linear filters with various spatio-temporal
orientations.
20
R components and speed v
21
R and direction of motion q
Multiple, distributed representation of direction.
22
Sectional curvatures
23
Direction tunings of R components
vertical motion
horizontal motion
24
Barber pole
Wallach, 1935
25
abolished illusion
Kooi, 1993
26
Orthogonal orientation and direction tunings
Analytical predictions based on R components
Typical Type II MT neuron, macaque monkey
Direction tuning
Rodman Albright, 1989
Orientation tuning
27
Multiple motions
Analytical predictions based on R components
Typical MT neuron, macaque monkey
Recanzone, Wurtz, Schwarz, 1997
28
Conclusion

• Hypothesis that a basic (geometric) signal
  property (the intrinsic dimensionality) is
  encoded in early- and mid-level vision explains
• orientation selectivity (derivatives, and R2, R3)
• endstopping
• (all R components are endstopped for
  translations)
• velocity selectivity
• direction selectivity
• some global-motion percepts (by integration)
• some properties reported for MT neurons.

(Reference to 3D world of moving objects is not
needed.)