Biologically Motivated Artificial Vision Systems BMAVS

1
Biologically Motivated Artificial Vision Systems
(BMAVS)

• Presented by Pramod Varma
• Research currently under the guidance of
• Dr. David Enke

2
Overview of Artificial Vision Systems (AVS)

• AVS have proven beneficial for a number of
  engineering and manufacturing applications
• Printed circuit board inspection
• Robot place learning and navigation
• Diagnostic of skin cancers
• Military applications of satellite roadway
  identification
• Many more

3
AVS ( Cont..)

• Performance of these systems has been known to
  degrade considerably in the presence of noise and
  become inflexible when encountering various
  levels of image information
• Although advancement have been made to overcome
  most of these difficulties, the visual abilities
  of these artificial vision system are still faint
  in comparison to the human visual system
• Cognitive functions such as feature extraction
  and visual attention at times are ignored in
  neural network architectures currently being used
  in artificial vision systems
• Although advancement in neural network are
  considerable, they are often employed during the
  later stages of image classification and
  recognition after processing and feature
  extraction have been performed

4
Biological Vision System (BVS)

• A wealth of information has been determined
  regarding the components and structure of the
  primate visual system
• The human visual system should be able to offer
  knowledge and structure for adding versatility to
  existing artificial vision systems

5
Neuroscience and other disciplines

• Neuroscience has made major advances towards a
  prototype of the human brain
• For example cure of pigmentosa
• Researchers are moving away from what is called
  pure artificial vision to what is called more
  interactive vision
• Interactive vision would allow for descriptions
  of both feed forward and feedback processes,
  hierarchical based structures that can skip
  levels and move in the opposite direction

6
Research in BMAVS

• Goal
• Not to exactly mimic the HVS, but to gain
  insights that may further advance the field of
  AVS
• To achieve this, the modeling revolves around the
  regions of retina and brain involves image
  acquisition, filtering, edge detection, noise
  reduction, feature detection, and visual attention

7
Topics to be addressed

• Overview of the Human Visual System
• Emphasis on the Human Retina
• Bio-Inspired artificial vision models
• New Biological-Inspired Model of early vision
• Emphasis on the Human Retina
• Simulation results
• Conclusion

8
The Human Eye

• An eye consists of
• Aperture (pin hole, pit, or pupil) to admit light
• Lens that focuses light
• Photoreceptive elements (retina) that transduce
  the light stimulus

Source http//www.nei.nih.gov/nei/vision/vision2
.htm
9
Retina

• Light passes through the pupil and is focused by
  the lens onto the retina at the back of the eye
• Retina consists of three layers
• Ganglion cell layers
• Bipolar cell layers
• Photoreceptors layer
• The ganglion cell layer is the outermost layer
  and photoreceptor layer is the innermost layer

10
Layers of Retina

• ONL Outer Nuclear Layer
• INLInner Nuclear Layer
• H Horizontal cell
• BBipolar cell
• AAmacrine cells
• OPLOuter Plexiform Layer
• IPLInner Plexiform Layer
• GCLGanglion Cell Layer

Sourcehttp//retina.anatomy.upenn.edu/rob/retcha
p2.html
11
Physiology of Retina

• Photoreceptors
• Photoreceptors hyperpolarizes in response to a
  flash of light
• Two types of photo receptors
• Rods 120 million
• Light sensitive
• Found in periphery of retina
• Low activation of threshold
• Cones 6 million
• Are color sensitive
• Found mostly in the fovea

12
Retinal Neurons (Cont..)

• Horizontal cells
• Lateral inhibition is provided by horizontal
  cells in the first layer
• Vital to retina for making contrast an important
  feature

13
Retinal Neurons (Cont..)

• Bipolar cells
• The axonal terminal of the cone transmits its
  signals to the bipolar cells with a chemical
  synapse which increases gain at the cost of
  adding noise and reducing the intensity range
  over which bipolar cell can respond
• To induce spatial low pass filter

14
Retinal Neurons (Cont..)

• Amacrine cells
• Perform subtractive or shunting control functions
• Involved in directional selectivity of ganglion
  cells
• Feedback from Amacrine can temporarily spatially
  filter the bipolar signal

15
Retinal Neurons (Cont..)

• Ganglion cells
• Signals from the ganglion cells are sent to the
  LGN of the thalamus via the optic nerve/tract
• Ganglion cells in the retinal periphery receive
  input from many photoreceptors while ganglion
  cells in the fovea receive input from only one
  photoreceptor

16
Retinal circuitry
Adapted from Dowling, J.E., and Boycott, B.B.
Proceedings of the Royal Society of London, B.,
1966, 166, 80-111.
17
Physiology of Retina (Cont..)

• Receptive field analysis is a powerful method
  studying the function of neural circuits
• Center surround receptive field cells
• Center surround characteristics
• Receptive field sizes
• Filter and contrast enhancement
• Contrast characteristics
• Blurring and filtering

18
Receptive Fields

• The receptive fields of retinal ganglion cells
  are circular with a center field and a surround
  field
• ON-Cell
• Cell exhibits a low baseline firing rate
• Light placed in center ring increases firing rate
• Light placed on surround decreases firing rate
• OFF-Cell
• Light placed in the center ring reduces the
  firing rate
• Light placed in the surround filed increases the
  firing rate

19
Biologically Inspired Models for use in
Artificial Vision Systems

• Grossberg Vision Model
• Boundary and Feature Contour System
• Neocognitron
• Modeling mammalian retina
• Biologically inspired neural network
  connectionist model - 1997

20
Biologically inspired neural Network
connectionist model

• Diagram is divided into two modules
• Module 1
• Retina
• Module 2
• LGN, Area V1, Area V4, IT, Memory
• Area V2, Area V3 are not modeled

21
Module M1

• Most biologically inspired models neglect the
  processing that occurs within the retina
• The main interests in M1 involves
• Contrast enhancement
• Edge detection
• Noise reduction
• Photoreceptor blurring
• Spatial resolution of the resulting signals
  leaving the retina model

22
Module M2

• Involves interaction between isolated components
• Feedback and feed-forward connections
• Provides extra filtering, noise reduction,
  boundary completion - addition of missing edges
  and line segments within the image, visual
  attention

23
Module 1 Processing

• Retina Model consists of two layers of neural
  interactions
• Layer 1 processing
• Layer 2 processing
• Layer one retina cell interactions

24
Shunting Neuron model

• Shunting neuron has proven successful for
  explaining the responses of cells with regards to
  their excitatory and inhibitory inputs
• Developed by Steven Grossberg

Prate of passive decay Qexcitatory saturation
point R inhibitory saturation points x(t)activat
ion of neuron e(t)total excitatory
input i(t)total inhibitory input
25
Layer One Processing

• Since the fovea region is considered, only
  interactions with the cones photoreceptors are
  considered
• Cones photoreceptors were modeled in a generic
  way
• Only grayscale images were considered.
• Photoreceptors pass their information to bipolar
  and horizontal cells
• The gap junctions between the cones are also
  modeled in order to include a blurring effect
• Reduces the negative effects of the
  photoreceptors graded response
• Reduce any uncorrelated noise by acting as a low
  pass filter
• Include blurring and finally act as an
  anti-aliasing filter

26
Layer One Processing (Cont)

• Each bipolar cell will receive a direct
  excitatory inputs from a photoreceptor
• Bipolar cells have a center surround organization
• The direct response from the cone will provide
  the bipolar center response
• Horizontal cell will provide the antagonistic
  surround response

27
Layer One processing (Cont..)

• Horizontal cell will contact roughly around 7-9
  cones
• Difference between the cones direct response and
  horizontal cell provides a high pass version of
  the input
• This again reduces the noise
• On and Off pathways were modeled
• On pathway responds to light spots on dark
  background
• Off pathway responds to dark spots on a light
  background

28
Layer One Processing (Cont..)

• Bipolar cell equations
• Neurons based on shunting dynamics
• Reduces to
• To determine inhibitory contribution of
  horizontal cells

Prate of passive decay Qexcitatory saturation
point R inhibitory saturation points x(t)activat
ion of neuron e(t)total excitatory
input i(t)total inhibitory input
h(x,y)response of the horizontal cell at
(x,y) c(i,j)cone activation at
(I,j) nhneighborhood span
29
Layer 1 Processing (Cont..)

• Thus, bipolar cell response is given as

b(x,y)0 if b(x,y)
30
Simulation results

• An Image of a human face is used for Simulation
  since it contains both low and high spatial
  frequency components and allows effects of
  varying neighborhood span to be visible
• Bipolar On cell and Off cell responses were
  simulated.

31
Bipolar responses
Bipolar On-center Response
Bipolar Off-center Response
32
Simulation results

• Analysis
• Effect of neighborhood span
• As neighborhood span is increased finer details
  are lost, edges are refined making the border to
  appear thick
• As span is decreased bipolar cells responds to
  more detailed, high spatial frequencies of the
  image
• Effect of bipolar threshold
• Increasing bipolar threshold reduces the noise
• There is a possibility of eliminating valuable
  information

33
Layer 1 Processing (Cont..)

• Photoreceptor Blurring
• If the difference between the central cones
  response and the horizontal cells output is large
  enough to activate the bipolar cells beyond its
  threshold then
• a feedback signal is sent back to the cone
• gap junctions are activated
• transmission of signal between the cones is
  allowed
• The new cone activation after blurring is given
  by
• M scaling factor 1/A if b(x,y)gt Threshold else
  0

34
Layer 2 Processing (Cont)

• The retinal neurons in the second layer fire
  action potentials
• There is 1-1 connection between bipolar and
  ganglion cells
• Lateral support is provided by Amacrine cells
• Samples spatial frequency of the layer 1 output
• Diagram of possible layer 2 retina cell
  interactions

35
Layer 2 Processing (Cont)

• Large field Amacrine cells
• Performs local enhancement of the input signal
• Sums the bipolar signals
• After Amacrine cells acts on summed signal, sends
  this signal back to bipolar cells
• Feedback signals will also be filtered to enhance
  the low contrast
• Performs gain control for the retina by limiting
  the range of bipolar cell responses
• Spreads large lateral distance across retina
• Can also connect to small field Amacrine cells

36
Layer 2 Processing (Cont)

• Small field Amacrine cells
• Provides antagonistic surround to ganglion cells
• Sharpens the bipolar cell responses
• Makes the activations of ganglion cells easier
• Acts as a feedback from large field cells

37
Layer 2 Processing (Cont)

• Triad of connections

38
Layer 2 Processing (Cont)

• Sigmoid function is used as an activation
  function
• Equation for large field Amacrine cell
• Large field Amacrine cells would provide the
  filtered and contrast enhanced cell responses
• Large field Amacrine response is given

39
Layer 2 Processing (Cont)

• Equation for Small field Amacrine cell
• Net activation
• Final response
• Effect of changing slope S
• As slope increases it reduces noise but at the
  same time it comes closer to a pure step function

40
Layer 2 Processing (Cont)

• Static shunting network is used to model ganglion
  cell responses
• Inhibitory contribution is given by summation of
  neighboring small field Amacrine net activation
• Excitatory input is given by bipolar cell
  responses
• Net activation is given as

41
Layer 2 Processing (Cont)

• Final response
• Simulation
• Again the same image was used for modelling

42
Amacrine Cell Responses
Small Field Amacrine cell response ( Off-Center)
Large Field Amacrine cell response (Off-Center)
43
Ganglion Cell Response
Ganglion Cell Response (Off-center)
Ganglion Cell Response (On-center)
44
Comments on the Current Model

• Advantages of this model
• Layer one retina model detects image signals and
  eliminates uncorrelated noise
• Blurring processing has the effect of increasing
  the spatial frequency analysis of the retina
  image processing
• Second layer performs contrast enhancement and
  gain control of the first layer output signals
• Drawbacks of this model
• Choice of the proper parameters is very crucial
• Increasing slope, decreases the noise but also
  decreases the edges that were enhanced
• Ganglion cells found to appear somewhat
  ineffective
• This might be just that its influence is minimal
  in the fovea as a result of previous spatial
  frequency filtering that has occurred in first
  layer

45
Conclusion

• Artificial vision systems should consider
  becoming more interactive if they ever hope to
  mimic or even surpass the visual processing of
  humans
• The human standard should not be considered the
  only zenith that artificial vision systems should
  try reach

46
The future

• The future of interactive vision and biological
  models

47
References

• The hand book of Brain Theory and Neural
  Networks, Michael Arbib
• Biologically inspired neural networks
  connectionist models for use in artificial vision
  systems, David L. Enke
• A biophysically realistic simulation of the
  vertebrate retina, matthias H. Hennig, Klaus
  Funke.
• Dowling, J.E., and Boycott, B.B. Proceedings of
  the Royal
• Society of London, B., 1966, 166, 80-111.
• http//www.webvision.med.utah.edu/anatomy.html
• http//retina.anatomy.upenn.edu/rob/retchap2.html
  
• http//wwwpeace.samaug.edu

48
End of Presentation

• Thank You