Lysbilde 1

1
EEG-fMRI Integration (a merry Tale of a Lame, a
Blind and an Ignorant) PASCAL
Workshop Berlin, June 28 2007 Tom Eichele
University of Bergen Norway
Adapted from Kilner
2
(No Transcript)
3
Black Box and Surrogates for Cognitive
Neuroscience
Temporal Resolution Human Cheap Repeatable No
Side-Effects Non-invasive Widely available
Spatial Resolution Human Almost
Cheap Repeatable Side-Effects ? Non-invasive Relat
ively widely available
Temporal Resolution Typically sptially
restricted Either subhuman species or sick
people, small sample Expensive Not
Repeatable Invasive Possible Side-Effects of
Surg./Ana./Inf. Restricted to specialized Centers
4
Surprise vs. Expectancy Oddball AEPs Chronometry
of target processing
Jongsma, Eichele, et al. Clin Neurophysiol. 2006
5
Start at the beginning Pattern Learning
Oddball Switching random and regular
target-to-target intervals (TTI) Induces
predictability of target occurrence. Elicits
amplitude modulation after 2-3 regular targets.
Jongsma, Eichele, et al. Clin Neurophysiol. 2006
6
Start at the beginning Pattern Learning Oddball,
AEPs Switching random and regular
target-to-target intervals (TTI) Induces
predictability of target occurrence. Elicits
amplitude modulation after 2-3 regular targets.
Jongsma, Eichele, et al. Clin Neurophysiol. 2006
7
Pattern Learning Oddball, OEPs Switching random
and regular target-to-target intervals
(TTI) Induces predictability of target
occurrence. Elicits amplitude modulation after
2-3 regular targets.
Jongsma, Eichele, et al. Clin Neurophysiol. 2006
8
Pattern Learning Oddball Switching random and
regular target-to-target intervals (TTI) Induces
predictability of target occurrence. Elicits
amplitude modulation after 2-3 regular targets.
Jongsma, Eichele, et al. Clin Neurophysiol. 2006
9
Where and When does the brain represent Surprise
vs. Expectancy? Single Trial EEG-fMRI
Debener, Ullsperger, Siegel et al. JN 2005 TICS
2006 / Eichele, Specht, Moosmann et al. PNAS 2005
10
Auditory Oddball fMRI STG PL/IPL ACC SMA Central
Precuneus PCC MFG SFG
Eichele, Specht, Moosmann et al. PNAS 2005 see
also Kiehl et al. 2005 NI
11
How to answer a Where-and-When question?

• One solution
• Induce slow across-trial Amplitude Modulations
  measured in ERP MR
• Amplitude Modulation is expressed during specific
  timepoints in the EEG
• Correlate ERP-Amplitude Modulation with fMRI
  signal.

Eichele, Specht, Moosmann et al. PNAS 2005
12
How to answer a Where-and-When question?
Eichele, Specht, Moosmann et al. PNAS 2005
13
How to answer a Where-and-When question?
Eichele, Specht, Moosmann et al. PNAS 2005
14
Eichele, Specht, Moosmann et al. PNAS 2005 (see
supplementary movies)
15
Why didnt the Auditory Onset Response show up?
Were we missing something ?
16
Whats the problem? In fact most
neuroscientists reject EEG and MEG evidence in
the beliefs that recording wave activity is
equivalent to observing an engine with a
stethoscope or a computer with a D'Arsonval
galvanometer. However, one can learn a lot
about a system by listening and watching, if one
knows what to seek and find. Walter J.
Freeman, 2000
17
(No Transcript)
18
How would the brain frame family pictures? How
well does a single voxel represent an EEG
feature?
19
How would the brain frame family pictures? How
well does a single voxel represent an EEG
feature?
20
Whats the problem? How well do we know what
we are looking at? Do the tools we use already
provide good visibility? What can we do if we
dont really know what to look for?
Dont know really.
Dont know really.
Mining / Blind Source Separation!
21
Whats the Situation? Brain activity is
temporally and spatially mixed. i.e. -EEG is
mixed. -fMRI is mixed.
22
Aims
Moosmann, Eichele, Nordby et al, IJP, in press /
Eichele, Calhoun, Moosmann, et al, IJP, in press
23
Single Trial EEG-fMRI revisited
Parallel / Joint
Debener, Ullsperger, Siegel et al. JN 2005 TICS
2006 / Eichele, Specht, Moosmann et al. PNAS 2005
24
Joint ICA Simulation Sources s
Moosmann, Eichele, Nordby et al, IJP, in press
25
Joint ICA Simulation Mixed x As
Moosmann, Eichele, Nordby et al, IJP, in press
26
Joint Model Simulation Unmixed y Wx
Moosmann, Eichele, Nordby et al, IJP, in press
27
2. Acquisition Pre-Processing
3. Reduction Concatenation
5. Integration by prediction
1. Data
4. ICA
Subj. 1, ICf1
Subj. M, ICfN
Subj. 1, s1(v1t1)
Map
1
A1
2
1
N
2
u1(v1t1)
Timecourse
K
fMRI -Realignment -Normalization -Smoothing -
fMRI -spatial ICA x As -back-projection (Gi-1Â)
RiYi
y ßX
fMRI -reduce data via PCA (R) -concatenate Subjec
ts in G
Subj. 2, s2(v2t2)
Convolution Regression
1
1
Single-Trial Modulation
2
A2
2
u2(v2t2)
Stimuli/Task
EEG -Artifact Red. -Filtering -Epoching -
EEG -reduce data via PCA (R) -concatenate Subject
s in G
EEG -temporal ICA xAs -back-projection
K
N
Average
Subj.M,sM(vMtm)
1
1
1
y1(i1)
Be1
x1(j)
y1(j)
Te1 ()
Re1-1
Single Trial Images
1
K
N
K
2
uM(vMtM)
1
1
1
1
1
1
x(j)
y2(i2)
s(j)
Be2
Te2 ()
x2(j)
y2(j)
Âe-1
Ge-1
Ce-1
Re2-1
K
AM
2
K
N
N
K
N
Map
N
1
1
1
yM(iM)
TeM ()
xM(j)
yM(j)
BeM
Re2-1
Subj. 1, ICe1
Subj. M, ICeN
K
K
N
Eichele, Calhoun, Moosmann, et al, IJP, in press
28
Previously lost in translation
z12
z6
z0
z36
z-12
x18
y-57
y57
Eichele, Calhoun, Moosmann, et al, IJP, in press
29
If this is what happens in EEG-fMRI when people
make errors
Is there some brain signal that tells that
people will make errors in the future?
Debener, Ullsperger, Siegel et al. JN 2005
30
5. Component Selection Inference
Map-based criteria
Timecourse-based criteria
Event-Related Response? ? DeCon
-Select replicable gray matter maps that are
present in the population. -Deconvolve the HRF.
-If an event related HRF is present estimate
single trial amplitudes.
Replicability ?

Physiology?
Functional Modulation? ? STA
Population?


Eichele, Debener, Calhoun, et al, submitted
31
-10
28
38
oIFG
mSFG
Amplitude (ß)

IC3
(h)
(i)
(g)
fdr 0.01 t124.83, puncorr.2.1?10-4

24
-60
8
PC
Amplitude (ß)
IC4
(l)
(k)
fdr 0.01 t124.46, puncorr.3.9?10-4
Eichele, Debener, Calhoun, et al, submitted
32
Conclusion Unmixing of EEG and fMRI
improves detection of correspondences
in concurrent data. N1/MMN Recovers known
patterns of responses that are not well
represented by Voxel-by-Voxel prediction. Error
Precursor Extract novel unknown
function-relevant responses that are difficult
to assess/model/see otherwise. Outlook That
being said, we know that we are 10 years behind
the possibly suitable applications from machine
learning -)
33
Thanks ! On behalf of Vince Calhoun Stefan
Debener Matthias Moosmann Karsten Specht Markus
Ullsperger
http//icatb.sourceforge.net http//sccn.ucsd.edu/
eeglab http//fmri.uib.no http//themindinstitute.
org/