Investigation of the relative contributions

1
Investigation of the relative contributions of
3-dimensional and 2-dimensional image cues in
texture segmentation
N. Guyader, L. Jingling, A. S. Lewis L.
Zhaoping
University College London
Correspondence to L. Zhaoping, z.li_at_ucl.ac.uk
2
Background and motivation
Li (1999, 2002) hypothesized that V1 (primary
visual cortex) creates a bottom up saliency map.
Texture segmentation is a good way to probe
saliency, since a salient texture border can pop
out.
Supporting the V1s saliency map, against other
views, is the phenomenon of interference in
texture segmentation (Zhaoping and May 2004),
which we will review.
We ask whether V2 (and other higher cortical
areas) contribute to the bottom up saliency in
addition to V1.
3-D perception can probe the question of V2 vs.
V1, since it is known that V2, but not V1, does
stereo matching and processing for surface
(depth) perception (Cumming and Parker 2000,
Bakin et al 2000, von der Heydt et al 2000).
Hence, we investigate interference in texture
segmentation using 2-D and 3-D stimuli
3
Texture segmentation
Easy and fast
The texture border is very salient and pops out
4
However, when
Homogeneous texture
Stimulus
Difficult, texture border not salient

5
ST
IT
Stimulus
Difficult, texture border not salient

V1s response level maps
6

• Interference in texture segmentation
• is predicted by the V1s saliency map (Zhaoping
  and May, 2004, Society for Neuroscience,
  Abstract)
• but contradicts the traditional views of the
  saliency map (Treisman, Wolfe, Koch, Ullman,
  Itti, etc.)

7
Measuring border saliency by segmentation
reaction time (RT)
8
Question What if the interfering texture (IT)
is in a different depth?
IT In front
IT Behind
Motivation for the question V2 but not V1
seems to solve the stereo matching for 3-D vision
(Cumming and Parker 2000, Bakin et al 2000, von
der Heydt et al 2000). Hence, can V2 contribute
to, or change, the saliency, after V1?
9
Question Is the reduction in interference due to
3-D perception?
Behind
In front
10
A schematic illustration
Note
IT shifted
IT not shifted
This pattern seemed easier to segment.
Is it sufficient to cause the reduction in
interference in the stereo images?
In other words, can a stereo blind person, with
intact V1 but unable to stereo match, achieve the
same reduction in interference when presented
with the dichoptic stimuli with shifted IT in
them?
11
To answer this question, we compare RTs in these
four conditions
12
Full interference
3D
2D
13
Note that 3-D stimuli
Behind
In front
can also be made from other dichoptic stimuli
Eye 2
Eye 1
The interfering texture is shifted by twice the
amount.
14
Shifting IT in one eye reduces interference no
less than when shifting in both eyes
3D
This is consistent with the hypothesis that V1
creates a saliency map, since V1 takes its
maximum (rather than summation of) outputs to
create saliency.
15
Summary
2-D image cues, rather than 3-D stereo cues, play
a dominant role to determine saliency in texture
segmentation
This suggests that bottom up saliency is
predominantly calculated by V1
3-D stereo cues do have a minor or secondary role
(not shown in detail here), and is subject
dependent.
The stereo effects may arise from top down
factors and involve higher visual areas.
16
Subject dependence of the secondary, stereo,
effects
3D, IT in Front
3D, IT Behind
2D
RT (ms)
Expert non-naïve subjects
Non-expert, naïve, subjects
17
Note that, to create a given disparity
difference, either shift IT by half the
disparity in both eyes in opposite directions,
or shift IT in a single eye by the whole
disparity
In 3 out of 4 naïve subjects, the larger shift of
IT in 2D stimuli reduces interference more than
the smaller shift.
Half shift of IT
Whole shift of IT
18
Larger shift advantage in 2D transferred to the
3D?
IT in Front
IT Behind
2D
19
Results not accounted for by traditional views
of saliency (Treisman, Wolfe, Koch, Itti, etc.)
Stimulus
C

A
Simple texture (ST)
Predicts C as easy to segment, like A,
contradicts data

B
Predicts difficult segmentation for C, consistent
with data
interfering texture (IT)
20
The new MAX rule is consistent with the
hypothesis that V1 creates a saliency map
(Zhaoping and May 2004)
The hypothesis states The most active V1 cell
responding to a location determines the saliency
of that location
21
Note
IT not shifted
The interfering texture (IT) is shifted
This pattern seemed easier to segment, is it
sufficient to cause the reduction in
interference in the stereo images?