Parallel Integration of Video Modules

1
Parallel Integration of Video Modules

• T. Poggio, E.B. Gamble, J.J. Little
• 6.899 Paper Presentation
• Presenter
• Brian Whitman

2
Overview

• Different cues make up a reliable map
• Edge
• Stereo
• Color
• Motion
• How can we integrate these cues to find surface
  discontinuities?

3
Architecture
4
Physical Discontinuities

• Depth
• Orientation
• Albedo Edges
• Specular Edges
• Shadow Edges

5
Implementation

• The architecture was not fully implemented
• Results in integrating brightness with
• Hue
• Texture
• Motion
• Stereo
• But separately not together

6
Smoothness

• Physical processes behind cues change slowly over
  time
• Two points adjacent are not vastly different
  depths
• Need a representation to capture this

7
Discontinuities

• Cues are assumed smooth everywhere except on
  discontinuities
• Each module needs to
• assume and interpolate smoothness
• detect edges and changes

8
Dual Lattices

• Circles are smooth, crosses are lines /
  discontinuities

9
Neighborhoods

10
Quickly, MRF (again)

• Prior probability of depth in the lattice is
• Z normalization, T is temperature, U is energy
  (sum of local contributions)
• If we know g (observation) use it

11
Membrane Prior

• Prior energy when surface is smooth

12
Gaussian Process

• If we assume gaussian process generated the noise

13
Line Process

• Where is the smoothness assumption broken?
• l line between i and j?
• Vc varying energies for different line
  configurations

14
Integrated Process

• Extend the energy function to tie together vision
  modules to brightness gradients
• Assumption changes in brightness guide our
  belief of the source of surface discontinuities

15
High Brightness Gradients

• Instead of energy terms based on line
  configuration, use strengths of brightness edges

16
Low-level Modules

• Paper mentions
• Edge detection
• Stereo
• Motion
• Color
• Texture
• But only has short detail on texture color.

17
Texture Module

• Measures level density
• Blobs are taken through a center-surround filter

18
Color Module

• Hue R/(RG)
• Should be independent of illumination
• MRF uses this to segment image into sections of
  constant reflectance

19
Original image brightness edges
20
Stereo data, MRF generated depth
21
Motion data, MRF generated flow
22
Texture data, MRF generated texture regions
23
Hue, MRF hue segmentation
24
Parallelizing

• Many words about specialized architecture
• Small processes better for mass computation
• Specialized experts model

25
More Recent

• Recent Mohan, Papageorgiou, Poggio paper
• Example-Based Object Detection in Images by
  Components
• Train an ACC using different experts

26
(No Transcript)
27
Conclusions

• All extracted surface discontinuities can be used
  in later understanding
• Do brightness edges aid human computation of
  surface discontinuities?
• Parallelizing image analysis