ECE 285

1
ECE 285

• Brain Mechanisms of Vision
• Hubel and Wiesel, 1979
• Vision by Man and Machine
• Poggio, 1984
• Features and Objects in Visual Processing
• Treisman, 1986
• January 15, 2003
• Kim Harlow

2
SummaryVision by Man and Machine Poggio, 1984

• Differences between human vision processing and
  computer vision processing
• Hardware
• Neurons
• Wires
• Hardware organization
• Neuron connections have thousands of inputs, 3D
  outputs
• Wires have limited inputs, more or less 2D output
• Transmission of signals
• Graded electrical signals, chemical messenger
  substances, ion transport
• Binary pulses
• Temporal organization
• Concurrent analysis of millions of channels, no
  clock
• Serial processing, with clock
• Similarity Information processing tasks
  performed

3
SummaryVision by Man and Machine Poggio, 1984

• Levels of Vision Problem
• Computation
• What tasks need to be completed
• Algorithm
• What sequence of steps needed to complete task
• Hardware
• What neurons/electronic circuits are necessary

4
SummaryVision by Man and Machine Poggio, 1984

• Stereopsis
• Matching features from one image to another,
  determining the disparity between the positions
  and calculating their relative depths in the 3D
  world
• Formal Steps
• Select location in space
• Identify same location in other retinal image
• Measure positions
• Use measurement disparity to calculate location
  depth

Blue lower elevation Red higher elevation
5
SummaryVision by Man and Machine Poggio, 1984

• Random-dot stereogram experiment
• Conclusion stereopsis results from binocular
  disparities, without a need for obvious matching
  visual clues

6
SummaryVision by Man and Machine Poggio, 1984

• Stereopsis Algorithm
• Assumptions to constrain the problem
• Uniqueness of location a point has only one 3D
  location at a given time
• Continuity and opacity discontinuities occur
  only at physical object boundaries
• Use of intensity edges as identifiable features

7
SummaryVision by Man and Machine Poggio, 1984

• Edge detection using derivatives
• Problem derivatives only work well on clean,
  sharp changes
• Solution Laplacian of Gaussian
• Derivative and smoothing
• Similarity to center-surround organization of
  retinal ganglion cells
• Problem different scales of intensity changes
• Solution filters of different sizes

8
SummaryVision by Man and Machine Poggio, 1984

• Stereopsis Algorithm
• Assign 1s to all row pixels with matching binary
  values
• Perform weighted sum of neighboring nodes, using
  positive weights for neighboring nodes not along
  the line of sight and negative weights for
  neighboring nodes along the line of sight
• If result gt threshold, node value 1, else 0
• Iterate until network is stable
• Can be performed in parallel
• Can fill in data gaps and allows for sharp
  discontinuities
• Only applicable for random-dot stereograms, not
  natural images

9
SummaryVision by Man and Machine Poggio, 1984

• Different class of stereopsis algorithms
• Matching of positive or negative patches in LoG
  filtered image pairs
• Matching zero-crossings of same sign made by
  filters of 3 or more sizes
• Conclusion Brain can serve as example of how to
  seek solutions to problems such as vision

10
SummaryBrain Mechanisms of VisionHubel and
Wiesel, 1979

• Visual path and cortical organization
• Ganglion cells
• Center-surround configuration
• Lateral geniculate nuclei
• Primary visual cortex
• Layer IV cells circularly symmetrical
• Simple cells respond to oriented line in
  specific position
• Complex cells respond to oriented line in
  varied position
• Experimentation using microelectrodes to measure
  nerve firing according to retinal stimulation by
  varied light patterns
• Receptive field positioning indicates cortexs
  method of visual scene analysis according to
  eccentricity (closeness to center of gaze)
• Cortical cells arranged in independent column
  systems to represent varied optimal stimulus
  orientation of simple and complex cells and eye
  preference

11
SummaryFeatures and Objects in Visual
Processing Treisman, 1986

• Levels of Visual Processing
• preattentive simultaneous and automatic
• Later stage serial and with focused attention
• Preattentive Detection Experiments
• Illusory property exchanges
• Visual-search tasks
• Target differs from distractors in a simple
  property -- target is detected equally fast,
  regardless of number of distractors
• Target is characterized only by a unique
  combination of properties or components -- time
  taken to detect target increases linearly with
  number of distractors
• Line property experiments ? some properties are
  represented as deviations from a zero position
• Prior Knowledge Experiments
• Expectations help to use attention efficiently
• Expectations do not seem to increase illusory
  exchanges to make abnormal items look like
  expectation
• Object perception also based on a continually
  updated representation

12
Relation of Papers

• Hubel and Wiesel provides a detailed
  understanding of cell level vision processing
• Treisman provides a broader understanding of
  vision processing based on human response time to
  an image stimulus
• Poggio provides a specific application for vision
  processing in stereopsis using edge detection

13
Common Thread Preattentive Processing

• Treisman focuses on what types of visual
  detection tasks are within the preattentive range
• Numerous experiments based on difficulty of
  detecting an item among a number of distractor
  items
• Preattentive processes can distinguish a simple
  property easily but a specific combination of
  properties held also by the distractor items
• Color, size, contrast, tilt, curvature and line
  ends
• At the cell level, Hubel and Wiesel illustrate
  the initial path of vision processing from retina
  to primary visual cortex
• Poggios stereopsis algorithms provide examples
  of a preattentive analysis a task that is
  completed without any prior knowledge of the
  image space

14
Common Thread Edge Detection

• Edge detection is the focus of Poggios
  stereopsis example, as features detectable by a
  preattentive algorithm
• Hubel and Wiesels paper provides a biological
  equivalent to the edge detection algorithm in the
  orientation specific cells found in the primary
  visual cortex
• Simple cells are found to respond actively to
  an optimally oriented line in a narrowly defined
  location
• Complex cells are found to respond actively to
  an optimally oriented line in a range of
  locations
• Same as result of the ideal application of the
  edge detection filters in Poggios paper

15
Additional Vision Processing Tasks

• Hubel and Wiesel present the following
  conclusion the cortexs solution to a basic
  problem is the analyze the visual scene in detail
  in the central portion and more crudely in the
  periphery
• Although computers do not have a defined central
  gaze or periphery, this solution is still able to
  be applied to object and event detection
• When detecting a particular event, the central
  gaze of the algorithm can be defined based on
  prior knowledge such as which pixels correspond
  to the area where the event may take place

16
Human and Machine Vision Parallels

• The hierarchical nature of vision is evident in
  both the human processes in Hubel and Wiesel and
  the machine vision processes
• Cortex Cell Hierarchy (human)
• From layer IV cells to simple cells to complex
  cells
• Vision Hierarchy (human and machine)
• From low-level preattentive processes to
  High-level prior knowledge processes
• Poggio also discusses a hierarchy of vision which
  is used in computer solutions such as the
  stereopsis
• Computation, Algorithm, and Hardware