Title: Magnetoencephalogram (MEG) correlated with perceptual binding of color and motion
1Magnetoencephalogram (MEG) correlated with
perceptual binding of color and motion
- Kaoru Amano1, Shinya Nishida2, and Tsunehiro
Takeda1
1. University of Tokyo 2. NTT Communication
Science Laboratories
2Color motion asynchrony
Perception
Stimulus
Stimulus synchrony (SOA 0 ms)
Perceptual asynchrony (When color reversals and
motion direction reversals are physically in
phase, they are not perceived to be in synchrony.)
250 ms
Stimulus asynchrony (SOA 100 ms)
Perceptual synchrony (When motion directions
reversal precedes color reversals by about 100
ms, they are perceived to be in synchrony)
250 ms
SOA
Moutoussis Zeki (1997)
3Purpose
- To find the neural activities correlated with
perceptual synchrony and binding of color and
motion - Comparison of MEGs between the condition of
perceptual synchrony (SOA100 ms) and that of
perceptual asynchrony (SOA0 ms) - Averaged waveforms
- Wavelet analysis of time-frequency domain
- Psychophysical experiments of the perceptual
synchrony and binding of color and motion - To test psychophysically-proposed models, by
seeing - The difference in latency of motion response and
color response - The effect of alternation rate on MEG responses
- The effect of temporal structure on MEG responses
4Visual stimulus
- Color
- Test (4 s) Color was reversed between red and
green at 2Hz - Pretest (1 s) a green pattern
- Always stationary
- Motion
- Test (4 s) Motion direction was reversed between
expansion and contraction at 2Hz - Pretest (1 s) a stationary pattern
- Always green
- Color Motion (SOA 0 ms)
- Test (4 s) Color and motion direction were
reversed at 2Hz with no physical delay
(perceptual asynchrony) - Pretest (1 s) a stationary green pattern
- Color Motion (SOA 100 ms)
- Test (4 s) Color and motion direction were
reversed at 2Hz with a color delay of 100ms
(perceptual synchrony) - Pretest (1 s) a stationary green pattern
color motion
SOA
stationary
Cont.
Exp.
Exp.
0.5 s
1 s
MEG measurement
5 s
trigger
- Concentric half rings
- SF 1.1 cycle/deg, contrast50
- Red or Green
- Stationary or Expantion/Contraction
- velocity 3 deg/s
5MEG recording and analysis
- Ten male subjects were employed.
- MEG recording
- Whole head MEG system (Yokogawa, PQ244OR) with
230 axial-z sensors and 70 x 3 vector sensors - Sampling rate 625 Hz, Filter 0.3-200 Hz
- The number of average 100 times
- Pre-trigger 1000 ms, post-trigger 4000 ms
- Wavelet analysis
- Sensor output in each trial was convolved with
complex Morlet wavelets and was averaged across
all trials (Tallon-Baudry, 1996). - The time varying energy for each frequency was
corrected by the pre-trigger interval of 500 ms. - The results of right or left occipital channels
were averaged (34 channels for each). - Dipole source localization
- 230 axial-z sensors over the whole head were used
for the estimation. - Analysis was conducted for averaged MEG filtered
at gamma band (20-60 Hz). - Goodness of Fit (GOF) should be larger than 80 .
6Psychophysical experiment
Synchrony task
Binding task
Identify the color presented during expanding
motion
Tell whether color and motion were perceived to
be in phase
Motion fast
Color fast
Motion fast
Color fast
In-phase response ()
Red response ()
Averaged across all subjects (n10)
Color delay SOA (ms)
Color delay SOA (ms)
More successful binding of color and motion for
SOA100 ms than for SOA0 ms.
More synchronous perception of color and motion
for SOA100 ms than for SOA0 ms.
7Interactions of averaged waveforms
Waveform interaction, defined by
ColorMotion-(ColorMotion), averaged
between 1000 and 3750 ms and across occipital 68
sensors and subjects (n10)
Color
Motion
MEG amplitude (fT)
ColorMotion (SOA 0 ms)
MEG amplitudes (fT)
ColorMotion (SOA 100 ms)
Latency (ms)
- Peaked response to each stimulus change (every
250 ms) were found except for motion condition.
- There was no difference in the magnitude of
waveform interaction between the condition of
perceptual asynchrony (SOA0ms) and the condition
of perceptual synchrony (SOA100 ms).
8Interactions in time-frequency domain
Frequency domain interactions, defined by
colormotion-(motioncolor), averaged
between 1000 and 3750 ms and across occipital 34
left sensors and subjects (n10)
Color
Motion
Frequency (Hz)
ColorMotion (SOA 0 ms)
TF Energy
ColorMotion (SOA 100 ms)
20-25
30-35
40-45
50-55
25-30
35-40
45-50
55-60
Latency (ms)
Frequency band (Hz)
- Interactions in 20-40Hz were larger under the
condition of perceptual synchrony (SOA100ms)
than under the condition of perceptual asynchrony
(SOA0ms). - The effect was significant for 30-35 Hz in left
hemisphere (p0.034)
9Dipole source localization
Color,Color Motion (without filter)
Color Motion (20-60Hz)
168 ms
3120 ms
- Dipoles were estimated in the similar visual area
for Color and ColorMotion with both SOAs. - Dipole estimation for motion direction reversal
was not successful because of the low S/N.
- Magnetic field maps of 6/10 subjects indicated
the activities in the occipital areas - Their dipoles were mainly estimated around
calcarine sulcus. - The dipole amplitudes were generally larger under
the condition of perceptual synchrony than under
the condition of perceptual asynchrony.
10Dependency of temporal structure
Enhanced 20-40Hz responses might be correlated
not with perceptual synchrony but with physical
asynchrony
- Perceptual asynchrony depends on the temporal
structure of the stimuli (first-order change
versus second-order change) rather than the
attribute type (color versus motion). - C1P2 Motion delay (color motion asynchrony)
- C1P1 or C2P2 No delay
- C2P1 Color delay
Nishida and Johnston, 2002
Using C1P1 or C2P2 stimulus will make it clear
whether enhanced gamma responses are correlated
with perceptual synchrony or physical asynchrony
11First- and second-order change of color and
position
Nishida and Johnston, 2002
First-order change (transitions)
C1P1 (no delay)
color
Defined by two points in time
Position
C1P2 (motion delay)
C2P1 (color delay)
Second-order change (turning point)
color
Defined by three points in time
C2P2 (no delay)
Position (Motion)
12Psychophysical experiment (C1P1)
Synchrony task
Binding task
Identify the color when the rings were at the
outward position
Tell whether color change and position change
were perceived to be in phase
Motion fast
Color fast
Motion fast
Color fast
Red response ()
In-phase response ()
Color delay SOA (ms)
Color delay SOA (ms)
Averaged across all subjects (n6)
More synchronous perception of color and motion
for SOA0 ms than for SOA100ms.
More successful binding of color and motion for
SOA0ms than for SOA100ms.
13Interactions in waveforms and time-frequency
domain (C1P1)
Waveform interactions, defined by
C1P1-(C1P1), averaged between 1000 and
3750 ms and across occipital 68 sensors and
subjects (n6)
Frequency domain interactions, defined by
C1P1-(C1P1), averaged between 1000 and
3750 ms and across occipital 68 sensors and
subjects (n6)
TF Energy
MEG amplitudes (fT)
20-25
30-35
40-45
50-55
25-30
35-40
45-50
55-60
Frequency band (Hz)
- Interactions of averaged waveforms were very
similar between the two SOA conditions. - Interactions in 20-30Hz were larger under the
condition of perceptual synchrony (SOA 0 ms).
14Psychophysical experiment (C2P2)
Synchrony task
Binding task
Identify the color increasing during expanding
motion
Tell whether color change and motion were
perceived to be in phase
Motion fast
Color fast
Motion fast
Color fast
Green response ()
In-phase response ()
Averaged across all subjects (n6)
Color delay SOA (ms)
Color delay SOA (ms)
More synchronous perception of color and motion
for SOA0 ms than for SOA100ms.
More successful binding of color and motion for
SOA0ms than for SOA100ms.
15Interactions in waveforms and time-frequency
domain (C2P2)
Waveform interactions, defined by
C2P2-(P2C2), averaged between 1000 and
3750 ms and across occipital 68 sensors and
subjects (n6)
Frequency domain interactions, defined by
C2P2-(C2P2), averaged between 1000 and
3750 ms and across occipital 68 sensors and
subjects (n6)
TF Energy
MEG amplitudes (fT)
20-25
30-35
40-45
50-55
25-30
35-40
45-50
55-60
Frequency band (Hz)
- Interactions of averaged waveforms were very
similar between the two SOA conditions. - Interactions in 20-30Hz were larger under the
condition of perceptual synchrony (SOA 0 ms). - The effect was significant for 20-25 Hz in right
hemisphere and 20-25, 25-30 Hz in left hemisphere
(p0.049, p0.032, p0.0065 respectively).
16Discussions
Previous studies on the binding of visual
attributes
20-40 Hz responses were enhanced when C1 and P2
were perceived to be in synchrony (SOA100ms),
while 20-30 Hz responses were enhanced when C1
and P1 or C2 and P2 were perceived to be in
synchrony (SOA0ms).
- Electrophysiological studies on animals
- When two superimposed gratings that differ in
orientation and drift in different directions are
perceived as a single pattern (pattern motion),
synchronization between neurons sensitive to each
motion direction was increased (Castelo-Branco et
al., 2000) - Under the condition of binocular rivalry, cells
driven by the winning eye show a significant
correlation (Fries et al., 1997) - EEG or MEG studies on human
- Gamma band responses were enhanced when subjects
perceived illusory (Kaniza) triangle
(Tallon-Baudry et al., 1996) - Gamma band responses were enhanced when subjects
perceived Dalmatian dog hidden in black blobs
(Tallon-Baudry et al., 1997)
20-30Hz responses, slightly lower than the gamma
band (gt30Hz), might be correlated with perceptual
binding of color and motion
17Models of color motion asynchrony
- Psychophysically-proposed models of color motion
asynchrony - Latency difference model (Moutoussis Zeki,
1997) - Color and motion are processed in different
areas, and the processing time difference between
color and motion leads to perceptual asynchrony - Time marker model (Nishida Johnston, 2002)
- Temporal localisation judgements are made by
using temporal markers assigned to salient
changes - Hybrid model (Clifford et al, 2003 Bedell et
al., 2003) - Separate mechanisms for temporal synchrony and
binding
Give some suggestions on these models by
analyzing MEG amplitudes and latencies
18Time marker model
Nishida Johnston, 2002
- Perceptual motion delay occurs only for rapid
alternations - Reaction times are not different between color
and motion
Latency model can not account for color motion
asynchrony
Temporal localisation judgements are made by
using temporal markers assigned to salient changes
- Accurate temporal order judgement for slow
alternations result from matching between first
order changes (color change) and second order
changes (motion direction reversal) - Color motion asynchrony at rapid alternations
result from matching between first-order changes
(color change and position change)
- Color change, position change (motion)
- First-order change
- Time marker can be allocated even at rapid
alternations - Motion direction reversal
- Second-order change
- Time marker is difficult to be allocated at rapid
alternations
19Verification of the models
- The effect of alternation frequency on the color
change (C1) response and motion direction change
(P2) response. - According to the time marker model, the response
to motion reversal becomes less salient for
higher frequencies while the saliency of color
change does not change largely. - Comparison of MEG amplitudes and latencies
between color change (C1), color direction change
(C2), position change (P1), and motion direction
change (P2) at 2 Hz - Are there any difference in latency between MEG
to P2 and C1 - Does the temporal structure of stimuli affect MEG
amplitudes and/or latency?
20Dependency on alternation rate
NS
0.25 Hz
RMS (fT)
Peak RMS (fT)
2 Hz
0.25Hz
2Hz
0.25Hz
2Hz
RMS (fT)
color
motion
Average of peak RMS values at 250-500, 500-750 ms.
time (ms)
- Increasing alternation rate did not decrease
color response. - Motion response reduced at higher alternation
rate (2Hz).
Consistent with the time marker model
21Dependency on temporal structure Latency
- MEG latency was apparently shorter (but in fact
longer, considering the short stimulus cycles)
for color change (C1) than for motion direction
change (P2). - MEG latency was affected not by stimulus
attribute but by temporal structure
Alternation rate 2Hz
Latency (ms)
- The latency difference between C1 and P2 is
consistent with the latency model, and is also
consistent with the time marker model given MEG
primarily reflects the response to first-order
temporal changes. - Similarity in latency found between C1 and P1,
and between C2 and P2 is consistent with time
marker model, but the latency model cannot
predict these results.
22Dependency on temporal structure Amplitude
- MEG amplitude was smaller for motion direction
change (P2) than for position change (P1), and
for color direction change (C2) than for color
change (C1).
Alternation rate 2Hz
Amplitudes (fT)
Compatible with the hypothesis that MEG primarily
reflects the response to the first-order temporal
change and the first-order change in P2 and C2
were less salient than that in P1 and C1.
23Suggestions on models
- P2 response reduced at higher alternation rate
(2Hz), though increasing alternation rate did not
decrease C1 response. - MEG latency was affected not by stimulus
attribute (color vs. motion) but by temporal
structure (first-order vs. second-order).
The MEG responses support the time marker model
(and hybrid models) more than the latency model.
24Summary
- MEG responses for simultaneous alternations in
color and motion direction were measured. - 20-40 Hz responses were enhanced when C1 and P2
were perceived to be in synchrony (SOA100ms),
while 20-30 Hz responses were enhanced when C1
and P1 or C2 and P2 were perceived to be in
synchrony (SOA0ms). - MEG responses were compatible with the time
marker model of color-motion asynchrony (Nishida
and Johnston, 2002) indicating matching of
salient events.