PowerPoint Presentation Art and the Language of Vision

1
Tracking motions, describing events

• Tracking mechanisms
• Encoding from a moving selection
• Tracking and motion perception
• Tracking and perceiving actions
• Anatomy
• Vision as a language

2
1. Tracking mechanisms
What is tracked? Object is basic unit of
tracking How many? Capacity not fixed in number
of targets Limits independent in each
hemifield Does tracking use same resource as
central tasks Principle of Exclusion - no overlap
3
Tracking in computer vision
Passive tracking
2D snake
3D shrink wrap
4
What is tracked?
Brian Scholl et al joined items in a tracking
display and drastically reduced ability to track
Basic display
5
What is tracked?
Brian Scholl et al joined items in a tracking
display and drastically reduced ability to track
Joined item display
6
What is tracked?
Attention object based (Duncan, 1984) IOR
Object based (Chou Yeh, 2005) Tracking Objects
based Selection of objects is obligatory
7
How many? Capacity and speed
In a standard tracking task, Alvarez and
Franconeri found capacity decreased steadily as
speed of items increased Suggests maximum is
fixed by a total capacity (items x load) not a
numkber of objects
2
0
r2 0.995
1
5
Speed (deg/sec)
1
0
5
0
1
1
0
Capacity
8
Independent limits in left and right hemifields
Tracking is easier when items are divided between
hemifields
9
Independent limits in left and right hemifields
Harder when all items are in one hemifield
10
Independent limits in left and right hemifields
11
Consequences for switching or grouping mechanisms
If a single focus of attention can switch rapidly
from item to item, it should do so within a
hemifield as well as across If grouping the items
is required to track them, then grouping should
work as well within a hemifield as across If
switching or grouping is part of the basic
mechanism of tracking, it must occur
independently in the two hemifields At least two
independent control systems for tracking
12
Independent limits in left and right hemifields
Some cost of handoff
13
Overlap with central resources
Measure tracking performance with performing a
distracting central task Count backward from a
random number by 3s This extra load caused
dramatic drop in performance in both
hemifields In addition to the hemifield specific
processes, other parts of tracking process
require central resources
14
Can multiple selections overlap?
If there are multiple regions of selection Can
they overlap Or are they mutually exclusive When
two items overlap (Duncan Blaser, Pyslyshyn,
Holcombe) Report two features of one item But not
one feature from each What about two
non-overlapping items?
15
Task tracking rotating target
Example movie
Sine Track
16
Dual targets
Follow one side of triangle and one bar of
cross Both in the same location Or in different
locations Tests in separate hemifields
17
Track both at different locations even at higher
speed At same location, tracking deteriorates
quickly Similar to Valdes-Sosa, Cobo, Pinilla,
2000 Exclusion one selection per hemifield
18
2. Encoding from a moving selection
What can be picked up from a moving selection
window? Use integration from moving window to
discriminate preselection from postselection
limits
19
Letter Masking
Example letter masking. When a mask follows the
letter, does it destroy its representation at an
early level or only at a post selection level
20
(No Transcript)
21
Adjacent feature binding
When feature pairs are rapidly alternated, does
it destroy their representation at an early level
or only at a post selection level
Early level timing of features degraded, so
pairing is difficult Postselection features are
selected but identity masked by following item
22
Preselection vs postselection limits
Use multiple examples of the alternating
stimulus Move a selection window across
them Arrange timing so that no alternation within
window
23
ColBarLetterMask

24
3. Tracking and motion perception
Two independent systems One preattentive, one
attention based
25
Low-level Motion
Directionally selective cells in visual
cortex Operate preattentively Explain many
phenomena of motion perception Example motion
aftereffect
26
Seeing motion not detected by low-level detectors
Low-level, directionally selective cells are
mostly driven by luminance Color, texture and
binocular disparity do not activate directionally
selective cells very much But we can still see
these patterns move
27
High-Level Motion as tracking
See object Notice change in position
28
Both levels at once
Dual Motion
29
Tracking and modeling actions
Tracking things as they move Making a dynamic
model of the motion --gtSprites High-level motion
should require attention
30
Tracking in computer vision
Passive tracking
2D snake
3D shrink wrap
Tracked features direct model animation
31
Making a dynamic model of the action
High level motion is more than just a passive
tracking A dynamic representation, sprite that
models an objects motion Including
characteristic actions Filling in, not just of
missing static bits of familiar patterns, but of
actions
32
Many dynamic models available Each waits around
for any image action that it can model Like
Selfridges daemons
When its own action comes along it becomes active
to animate the action
It can give a rich and detailed animation based
on sparse and noisy image action
33
Very much like our representation of
melodies Once triggered, our memory of the melody
fills out the sounds we hear These
representations of actions are the units of
high-level motion Rather than the left, right, up
and down, etc of low-level motion Roll, bounce,
slap, break, flutter, glide Do they require
attention?
34
Identifying walking is slow
4 normal Ss
Visual search taskFind rightward walker among
leftward
35
Identifying walking is slow

• In both cases, identifying walker requires
  attention.

Find walker among scrambled figures
36
Apparent Motion
Definition Sequential presentation of separated
objects Can drive low-level motion detectors if
objects close enough together But motion is seen
even for very large separations Visual search
experiments show that attention required even for
simplest case of apparent motion
37
Multistable apparent motion
The movement of a single object can explain the
locations and timing of a large number of local
changes But the object can be assigned different
trajectories at will
38
Quartets
This ambiguous apparent motion stimulus tells us
a lot about the rules of tracking When there is a
choice of trajectory, which is chosen?
39
Quartets basic rule choose shortest path But
once a pairing of items is chosen, stick with it
-- hold on to the model
Quartets
40
When more than one quartet of apparent motion is
present They all appear to do the same thing
If one switches direction from horizontal to
vertical, they all switch
Suggests that you can attend to more than one if
they are all animated by the same model
41
However, you can learn a new model And see
vertical in one quartet With horizontal in the
other But only if they are linked
A model of linked action, like walking but far
simpler
Sprite
42
5. Anatomy
Culham, Cavanagh, Kanwisher, 2001 Culham et
al, 1998
43
Right parietal damageLeft neglectArtists self
portraits as he recovers from neglect
44
Apparent motion test for patients with damage in
parietal lobes
Result even slow apparent motion not seen by
patients
45
Apparent motion test
Slow
Fast
Battelli, Cavanagh, Intrilligator, Tramo, Hénaff,
Michel, Barton, 2001
46
Attentive Tracking
tracked items
47
Results Patients vs Controls
1
0
0
Lesioned hemisphere
7
5
Unlesioned hemisphere
Patient / Control ()
5
0
Bad
Good
2
5
0
Low-level Motion
Selective Attention
Attentive Tracking
Apparent Motion
48
Anatomy
Parietal site for tracking aspect of spatial
attention Contralateral effects of
lesions Bilateral effect of right lesion on
apparent motion Loss of event timing, when
pathway in right parietal
49
6. Vision as a language

• Selection
• Capacity
• Acuity
• Singularity,
• Tracking
• A high-level motion system
• Description
• combining selections
• describe events
• send description to other modules

50
(No Transcript)
51
Are the descriptions part of a language?
52
Why Language

• Run out of single labels
• Cant send pictures
• Language allows combinations of labels with
  specific roles
• Many more descriptions out of same number of
  labels

53
Language of Vision

• Nouns objects
• Verbs actions
• Prepositions
  spatial, temporal relations

54
Nouns

• Vision is knowing what is where by looking (Marr)
• Can be fast, direct from primary, preattentive
  visual system.

55
Verbs Familiar Actions

• Flexible descriptions which embody object
  constraints
• Online animation to fit changing image data
  sprites

Cavanagh, Labianca, Thornton, 2001
56
Spatial and temporal relations between two
selections.Slow, require construction
Prepositions
Out
In
57
Language of Vision

• Any evidence for grammar or syntax?
• What would ungrammatical vision be like?
• Impossible events, magic?

58

• Mistake in object composition
• Bad orthography

59

• Local syntax OK
• Impossible object
• I am writing to you with my sword raised and a
  pistol in each hand. Aaron Burr

60

• Man ?? Dog.
• Phrase structure seems incomplete
• Eventually get right description

61
Common Mechanism?

• Is vision the Ur language?
• Look for similarities in acquisition system and
  development fossils of visual structure

62
Asymmetry in Visual Search
63
Asymmetry in Lexical Marking

• Antonyms bright/dim, long/short, wide/narrow
• Often one is base term, it names the dimension
• Long - short --gt length
• Bright - dim --gt brightness
• Tilted - vertical --gt tilt
• The other is marked short long
• Takes longer to process in speech

64
Conclusions Language

• Attention exports a description of events to the
  mind in a language format
• Language description more efficient

65
Summary

• Early and mid-level features in parallel
• Selection limited in resolution
• No scrutiny within single selection
• Binding across selections
• Tracking capacity flexible, total fixed
• Field independence
• Tracking is contact between attention and input
• Complex models can guide selection
• Output is formatted

66
Glossary
Scene motion the motion paths and actions of
objects in a scene can have low and high-level
components Low-level motion local motion
detected by direction selective
neurons High-level motion tracking object
motions with attention Action representation a
memory of an action pattern that helps recognize
and then animate the perception of a familiar
motion Apparent motion discontinuous motion
stimulus with discrete steps in time and
space Biological motion light spots on a few
locations of a moving person or animal Quartet
an ambiguous, two frame apparent motion that can
be seen either in horizontal or vertical motions
67
(No Transcript)
68
What is low-level motion good for?
Have to know where object is to read out its
low-level motion signals But if you know where
the object is on a moment by moment basis, no
need to read out low-level motion.

69
Tracking
Is tracking just following a target? Or is it
more elaborate? Does it know about characteristic
motions Does it represent a trajectory
70
Maintaining a representation when an object is
hidden
Control
Hidden path