Jenny C A Read

1
A Bayesian model of the correspondence problem

• Jenny C A Read
• University of Oxford

2
outline

• psychophysical experiments (with Richard Eagle)
• qualitative interpretation
• quantitative modelling
• conclusions

3
stereopsis motion
how far?
how fast?
4
psychophysical experiments
stereo
motion
monitor
mirror
keypad
observer
5
anti-correlated stimuli
black ?? white
6
anti-correlated stimuli
motion
stereo
?
7
(No Transcript)
8
single Fourier component...
contrast-reversed...
...shifted ?/(2cos?)-d to left
...shifted d to right
9
why reversed motion?
correlated
anti-correlated
1st frame
1st frame
2nd frame
2nd frame
motion to right
perceive
motion to left
10
cross-correlation function
left eye first frame
shift, multiply and sum
CCF(x,y)??dxdy ?FL(x-x,y-y)FR(x,y)
right eye second frame
original image
filtered image
I(x,y)
F(x,y)
11
deducing match from CCF
correlated
anti-correlated
vertical shift
horizontal shift
horizontal shift
12
example stimuli
1d
2d
narrow-band broad-band
all orientations
increase spatial frequency bandwidth
increase orientation bandwidth
13
with narrow-band 1d stimuli
MOTION
STEREO
no problem!
100
100
correlated
anti-correlated
clear reversed perceptions
0
0

• Results reflect periodicity of stimulus.

Read Eagle (2000) Vision Research 40 3345-3358
14
with broad-band 1d stimuli
MOTION
STEREO
no problem!
100
100
correlated
just below chance weak reversed perceptions
anti-correlated
40
40

• Stereo and motion results are similar.
• Maybe both systems have a similar way of
  combining information from different spatial
  frequency channels.

Read Eagle (2000) Vision Research 40 3345-3358
15
results with 2d stimuli
MOTION
STEREO
no problem!
100
100
correlated
anti-correlated
50
0
clear reversed motion
chance

• Stereo and motion results are different.
• Maybe a difference in how systems combine
  information from different orientation channels

Read Eagle (2000) Vision Research 40 3345-3358
16
crossed (stereo) right (motion)
uncrossed (stereo) left (motion)
true disparity
disparity (stereo) displacement (motion)
disparities reported by different channels
Different orientation channels agree on
direction, but not on magnitude.
STEREO SYSTEM Disagreement on magnitude prevents
clear perception of depth.
MOTION SYSTEM Agreement on direction enables
clear perception of reversed motion.
17
the results can be understood qualitativelycan
they be modelled quantitatively?
18
what is perception?
physical world
BRAIN
mental representation of the world
19
a bayesian model
prior visual experience P(W)
physical world W
Bayes theorem P(WI) P(IW) ? P(I) / P(W)
BRAIN
visual image I
mental representation of the world
noise
knowledge of imaging system P(IW)
20
the model
After initial image processing by simple and
complex cells.
each channel assesses the probability of each
possible disparity...
finally output from all channels is combined.
to arrive at a single estimate of disparity.
21
retinal noise
original image
noisy image
22
initial image processing
by linear simple cells with Gabor receptive
fields (RFs)
simple cell response
SPos(???(x,y)I(x,y)dxdy)
23
different simple cell RFs
?(x,y)
odd even
colour key
no response ON region OFF
region
24
bandwidth of Gabor functions
single Gabor
many Gabors
4 spatial frequency channels of 1.5 octaves and
6 orientation channels of 30o are enough.
25
retinal array of RF centres
centre of simple cell RF
26
complex cells
S
left eye first frame
S
odd RF
even RF
Cx
S
right eye second frame
S
27
Channels in my model

• Channel population of complex cells with a
  particular ? and ?, tuned to different possible
  matches.
• Simple cells with many different spatial periods
  ? and orientations ?.
• Population of simple cells with RFs centred on an
  array of different positions in the retina.

28
stereo model
motion model
left retina
right retina
first frame
second frame
each channel consists of 93729 complex cells
tuned to horizontal disparities
each channel consists of 946561 complex cells
tuned to displacements in any direction
29
each complex cell is tuned to a single
correspondence
left retina first frame
poor match low firing rate
good match high firing rate
right retina second frame
original image
filtered image
30
basis of probability analysis
S
compare actual predicted response
left retina right retina
Cx
S
S
Cx
S
31
Bayesian analysis

• C(xL,xR) actual complex cell output.
• S(xL) monocular simple cell output.
• (xL?xR) xL is the correct match for xR.
• P(xL?xR) a priori probability of such a match.
• C(xL,xR) predicted complex cell output.
• Calculate P(CC,S, xL?xR).
• Hence calculate local match probability
• P(xL?xR C,S ) P(C C,S,xL?xR) P(xL?xR) /
  P(C,S)

32
synthesis

• we have a probability for each local match (for
  one channel, for one retinal location)
• could deduce disparity map
• psychophysical experiments required judgement of
  global disparity
• which interval had crossed disparity?
• we need a global estimate of disparity

33
stereo model

• assess probability of each possible matching of
  left/right retinal positions
• average the probabilities of all matches with the
  same horizontal disparity ?x
• average over all channels (? and ?)
• hence derive global match probability
• P(?x) prob. that global disparity is ?x
• pick the most probable disparity
• argmaxP(?x)

34
parameter adjustment

• two free parameters
• amplitude of noise
• strength of preference for small disparities

prior probability
disparity ?x (arcmin)
35
results stereo
36
motion model

• assess probability of each possible matching of
  1st/2nd-frame positions
• average probabilities of all matches with a given
  horizontal displacement ?x
• average over all spatial frequency channels
• different P?(?x) for each orientation channel
• for each orientation channel ?, pick the most
  probable horizontal displacement ?x(?).
• take weighted average of ?x(?) over all ?.

37
results motion
38
parameter adjustment

• Same prior works for motion as for stereo.
• But motion results require 10 ? noise for
  stereo.
• Motion system is more tolerant of poor matches.

39
conclusions

• puzzling experimental results...
• increasing orientation bandwidth
• enhances reversed motion
• impairs reversed depth
• ...can be reproduced by this model.
• physiologically-plausible initial processing
• Bayesian analysis within each channel
• stereo motion systems differ in
• how different orientation channels are combined.
• their tolerance of poor-quality matches.

40
(No Transcript)