Learning Sensorimotor Contingencies

1
Learning Sensorimotor Contingencies
James J. Clark Centre for Intelligent
Machines McGill University
2
This work is being done in collaboration
with J. Kevin ORegan (CNRS, Univ. Rene
Descartes) and with doctoral students at McGill
University Fatima Drissi-Smaili Ziad
Hafed Muhua Li
3
A mystery Why do we perceive the same
feature value (e.g. Color) when viewing the
feature foveally or peripherally? Why is this a
mystery? The signal provided by retinal
photoreceptors can be quite different when the
image of the feature falls on different places on
the retina. For example the spectral
sensitivity curves of retinal photoreceptors are
shifted towards the blue in peripheral cells as
compared with the foveal cells. .
4
A related mystery (perhaps) Why do neurons
in areas such as V4 and IT, which have large
receptive fields, respond to the same feature
value (e.g. color, orientation, complex shape)
no matter where the feature lies in the
receptive field? The activity of these neurons
is usually reduced when the feature falls in the
periphery of the receptive field as compared with
the center, but the neurons selectivity, or
tuning, is the same everywhere.
5
Perceptual Stability These mysteries can be
more generally considered as related to the
mystery of perceptual stability. Perceptual
stability is the constancy of subjective
experience across self-actions, even though these
self-actions can cause large changes in sensory
inputs.
6
Sensorimotor Contingencies
One theory of perceptual stability, due to
ORegan and Noe, holds that what is perceived is
the sensorimotor contingency associated with a
given physical stimulus. A sensorimotor
contingency is a law or set of laws
that describes the relation between self-actions
and resulting changes in sensory input. Since it
is the presence of a lawful relationship
between sensory input and motor activity that
determines the perception of a physical stimulus,
an appropriate change in sensory input is
necessary for a perception to be stable!
7
Conditioning using Temporal-Difference Learning
We propose that Sensorimotor Contingencies
associated with sensory changes due to eye
movements can be learned using a variety of
learning techniques. We propose the use of the
Temporal-Difference Learning scheme of Sutton
and Barto. This reinforcement learning technique
can be thought of as a form of Conditioning
where the Conditioned Stimulus is the sensory
activity before the eye movement and the
Unconditioned Stimulus is the sensory activity
after the eye movement. After conditioning,
presentation of the conditioned stimulus will
produce the same behaviour as that produced by
the unconditioned stimulus.
8
The Sutton-Barto Temporal-Difference Learning Rule
9
V is a matrix of association strengths between
pre- and post- saccadic stimuli. The pre-motor
stimulus X is held in a short-term
memory generating an eligibilty trace, which will
be used to enhance, in a Hebbian fashion, the
association to the post-motor stimulus. The
reinforcement signal, which is multiplied by the
eligibilty trace to yield the change in the
association matrix, is the difference between 2
different predictions of the foveal response - a
weighted sum of the current and previous foveal
responses, and the action of the current
association matrix on the previous peripheral
stimulus.
10
TRAINING PHASE
Attention selects a peripheral target and
enhances feature detector activity at that
location.
11
TRAINING PHASE
A short-term memory (eligibility trace) of this
feature activity is generated.
12
TRAINING PHASE
An eye movement is made, foveating the target.
13
TRAINING PHASE
Attention shifts to the fovea, enhancing the
feature detector activity there.
14
TRAINING PHASE
The feature detector activity at the fovea is
associated with the feature detector activity
represented in the short-term memory, using an
appropriate learning rule, e.g. the Sutton-Barto
Temporal Difference Rule.
15
RECOGNITION PHASE
Once associations have been built up, the
appearance of an attended-to target in periphery
can produce a response as though the target is
actually foveated. This response can be thought
of as a mental image. This mental image might be
represented by activity in neurons in areas with
large receptive fields (V4, IT) and hence would
be concerned only with feature type, rather than
feature location. This provides an explanation
for the continuity in the quality of the
subjective experience of a stimulus across the
visual field.
16
STEADY-STATE OPERATION
We have divided the processing into two separate
phases, Training and Recognition. In practice,
however, these can co-occur. The learning
mechanism can be continuous, allowing adaptation
to changes in the sensory and motor
systems (e.g. aging of the photoreceptors,
changes in the projective optics of the eye, )
17
Creation of Mental Images
Once the association weights matrix, V, has
been learned, it can be used to generate
predictions, M, of what the foveal image or
feature detector response will look like, based
on the peripheral, responses, P. M
VP It is expected that the association matrix
should map foveal images into themselves,
therefore the eigenvectors of this matrix should
be (linear combinations of) the foveal images.
F kVF
18
AN EXAMPLE STABILITY OF COLOR PERCEPTION
Many factors, including absorption of light by
the lens of the eye, cause a yellowing of the
light falling on the fovea as compared with that
falling on the periphery.
19
(No Transcript)
20
(No Transcript)
21
After training, a presentation of a given color
feature in the fovea is associated with the color
feature that would be observed after the feature
is foveated with an eye movement. This can be
seen in the structure of the association
weights matrix, where peripheral and foveal color
features map to the same color class.
22
(No Transcript)
23
ANOTHER EXAMPLE STABILITY OF STRAIGHT LINE
PERCEPTION
The retina is hemispherical, and this causes
straight lines in space to be projected as 2-D
arcs on the retinal surface, with radii of
curvature that vary with eccentricity
24
Images of Lines Projected onto Receptors
Images of Straight Lines At Various Eccentricities
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
It can be seen that the mental images are all
very close to the foveal images, no matter where
on the periphery the projection of the physical
line falls. The eigenvalues are not equal to
the foveal images, but the foveal images can be
obtained from them through a linear sum.
29
Development of Position Invariance in Neural
Responses
Standard View it is unclear how the development
would proceed without some sort of adaptation
signal coming from the need for constancy of
response across self-actions (e.g. eye movements)
Which feature detectors are connected to the
cell must be learned (and continually adapted)
Feature detectors with differing preferred
stimuli (corresponding to the photoreceptor
responses of a stable physical stimulus as the
eye moves)
30
Development of Position Invariance in Neural
Responses
Alternate View The weightings of the lower level
units are continually updated through the
associative learning mechanism. This mechanism
requires input from the oculomotor system to know
when an eye movement has taken place.
mental image (prediction of foveal response)
Eye movement signal
Association Layer
Feature detectors with differing preferred
stimuli (corresponding to the photoreceptor
responses of a stable physical stimulus as the
eye moves)
31
Conclusions
Perceptual stability and the position invariance
of higher-level cortical neurons may arise from a
learning of sensorimotor contingencies. Such
learning can be accomplished with a
reinforcement learning network, which learns to
generate predictions of lower level visual
feature detector activity which would occur after
foveation of a physical stimulus. In our view, a
projection of a physical stimulus onto
any peripheral retinal location will result in
the same mental image of the feature as
projection onto the fovea.
32
On-going and Future Research
Recurrent Feedback of predictions back down
to low-level feature detectors - will allow
small displacements of foveal image
Interpretation of the Reinforcement Signal -
small signal can be used to drive adaptation
- large signal can be used to indicate
instability of the world or to indicate
that a new class should be created
Psychophysical studies of Pre- and Post-motor
attention shifts Sensorimotor Basis function
representations of the Association weights
matrix.