Mid level vision, neglected yet still important

1
Mid level vision, neglected yet still important

• Ken Nakayama
• Harvard University

2
21st C challenge

• Existence and variation of occlusion and variable
  sources of illumination pose unsolved problems
  for vision

3

• Object representation needs an intermediate level
  format
• Low level vision alone is not even explanatory
  for wide range of visual processes (motion,
  stereo, search)
• Missing -- a satisfactory scientific description
  of surface level vision

4
1970s Visual take-over of the whole brain
1950s
5
Macaque monkey brain flattened
Visual regionsshown in color
6
Global division ofthe visual system
dorsal
(where, how)
ventral
what
7
action
objectrecognition
8
action
objectrecognition
9
Kanizsa
Phenomenology, reviving the Gestalt approach
Level surfaces Method phenomenology Practition
er Gaetano Kanizsa
new concepts amodal and modal completion
10
modal competion (in front)
Amodal competion (behind)
11
Inferences, but at what level ?
12
(No Transcript)
13
Suggests that thereis a completion process
within thevisual system
Amodal completion trumps knowledge of horses
14
Amodal completion allows fragments to be grouped
and thus recognized (strongest evidence)
spot the 5 letter Bs
From Bregman, 1990
15
Occlusion and the problem of segmentation for
object recognition
What belongstogether ?
Border ownership issues - for 3-D scenes,
borderscannot be shared. Border dispute needs
resolution
Rule - border belongs to the closest surface
16
Problem of segmentationKaniszas figure
Normal or amputee ?
17
Border ownership dictatedby lines
preventsmodal and amodal completion
18
New sources of evidence
Stereoscopic disparity
Surface in front owns the border. Thus face
on right is broken up, on left is OK
Nakayama et al.Perception 89 - faces easier to
recognize on left
19
Stereoscopic depth also determines
borderownership between regions. Nearer surface
will ownthe border (for opaque surfaces)
Nakayama Shimojo stereo demonstrations
20
Image level cant even explain much lower level
vision

• Deployment of attention, motion perception,
  texture, visual search

21
(No Transcript)
22
Surfaces needed for much lower visual function
23
He and Nakayama search task
Used stereo vision
Nature (1992)
24
(No Transcript)
25
Random dot stereogram
The correspondence problem an image based problem
26
(No Transcript)
27
occluding surfaces
What gives rise to unpaired points?
What would happen if we presented unpaired
points by themselves?
28
DaVinci stereopsis
(Nakayama Shimojo)
29
Scene depth from unpaired gaps
Gillam and Nakayama, 1999
30
Forest vs plane
A plane is a surface Which can occlude, a set of
random sticks cannot
31
Planes vs sticks
Gillam and Nakayama, 200
32
Level of processinghigh or low level inference?
Hypothesis - inferences learned via associative
cortical learning
33
generic view principle
when faced with more than one surface
interpretation of an image, the visual system
assumes it is viewing the scene from a generic,
not accidental, vantage point.

• Nakayama and Shimojo

34
Some counterintuitive observations
folded wings?
LE
RE
Why dont we interpolate depth and see folded
wings and cards?
35
(No Transcript)
36
Accidental vs generic vantage points
37

• accidental view
• generic view

38
(No Transcript)
39
generic view principle

• when faced with more than one surface
  interpretation of an image, the visual system
  assumes it is viewing the scene from a generic,
  not accidental, vantage point.

40
(No Transcript)
41
Some counterintuitive observations
folded wings?
LE
RE
Why dont we interpolate depth and see folded
wings and cards?
42
(No Transcript)
43
(No Transcript)
44
neural mechanisms ofsurface representation ?

• Cells in V2 respond to subjective contours

Strategy vary stimuli in ways that lead
to Appearance and disappearance of subjective
contours
45
Physiological correlates of illusory contours in
singleunit recordings
Recordings from a single cell in area V2 of monkey
46
Bakin, Nakayama, and Gilbert, 2000
47
Cells coding Border ownership?
Edgar Rubin figure and ground
Von der Heydt Et al.
Edge labeling?
contrast polarity vs edge labeling
48
image based response
49
Border ownership based response
50
Border ownership cells
Von der Heydt and colleagues
51
Von der Heydt (1984)
V2
Bakin, Nakayama, Gilbert (2000)
V2
DaVinci stereopsis
Border ownership cells (V2)
Mechanistic account of surface representation? --gt
52
21st C challenge

• Existence and variation of occlusion and variable
  sources of illumination pose unsolved problems
  for vision

53

• Object representation needs an intermediate level
  format
• Low level vision alone is not even explanatory
  for wide range of visual processes (motion,
  stereo, search)
• Missing -- a satisfactory scientific description
  of surface level vision
• -- demos the importance for illumination for
  object recognition

54
importance of shadow processing
Ted Adelson
55
Shadow processing in object recognition
56
Outline is very destructive to seeing regionsas
shaded. Line is interpreted as a
bounding Contour of an object
57
(No Transcript)