Title: Learning fMRI-Based Classifiers for Cognitive States
1Learning fMRI-Based Classifiers for Cognitive
States
- Stefan Niculescu
- Carnegie Mellon University
- April, 2003
Our Group Tom Mitchell, Luis Barrios, Rebecca
Hutchinson, Marcel Just, Francisco Pereira,
Xuerui Wang
2fMRI and Cognitive Modeling
- Have
- First generative models
- Task ? Cognitive state seq. ? average fMRIROI
- Predict subject-independent, gross anatomical
regions - Miss subject-subject variation, trial-trial
variation - Want
- Much greater precision, reverse the prediction
- ltfMRI, behavioral data, stimulusgt of single
subject, single trial ? Cognitive state seq.
3Cognitive task
Cognitive state sequence
4Cognitive task
Cognitive state sequence
Virtual sensors of cognitive state
5- Does fMRI contain enough information?
- Can we devise learning algorithms to construct
such virtual sensors?
Cognitive task
Cognitive state sequence
Virtual sensors of cognitive state
6(No Transcript)
7 8Preliminary Experiments Learning Virtual Sensors
- Machine learning approach train classifiers
- fMRI(t, t d) ? CognitiveState
- Fixed set of possible states
- Trained per subject, per experiment
- Time interval specified
9Approach
- Learn fMRI(t,,tk) ? CognitiveState
- Classifiers
- Gaussian Naïve Bayes, SVM, kNN
- Feature selection/abstraction
- Select subset of voxels (by signal, by anatomy)
- Select subinterval of time
- Average activities over space, time
- Normalize voxel activities
10Study 1 Pictures and Sentences
Xuerui Wang and Stefan Niculescu
- Trial read sentence, view picture, answer
whether sentence describes picture - Picture presented first in half of trials,
sentence first in other half
- Image every 500 msec
- 12 normal subjects
- Three possible objects star, dollar, plus
- Collected by Just et al.
11- It is true that the star is above the plus?
12(No Transcript)
13 ---
14(No Transcript)
15Is Subject Viewing Picture or Sentence?
- Learn fMRI(t, , t15) ? Picture, Sentence
- 40 training trials (40 pictures and 40 sentences)
- 7 ROIs
- Training methods
- K Nearest Neighbor
- Support Vector Machine
- Naïve Bayes
16Is Subject Viewing Picture or Sentence?
- SVMs and GNB worked better than kNN
- Results (leave one out) on picture-then-sentence,
sentence-then-picture data and combined - Random guess 50 accuracy
- SVM using pair of time slices at 5.0,5.5 sec
after stimulus 91 accuracy
17Error for Single-Subject Classifiers
- 95 confidence intervals are 10 - 15 large
- Accuracy of default classifier is 50
18Can We Train Subject-Indep Classifiers?
19Training Cross-Subject Classifiers
- Approach define supervoxels based on
anatomically defined regions of interest - Normalize per voxel activity for each subject
- Each value scaled now in 0,1
- Abstract to seven brain region supervoxels
- 16 snapshots for each supervoxel
- Train on n-1 subjects, test on nth
- Leave one subject out cross validation
20Error for Cross Subject Classifiers
- 95 confidence intervals approximately 5 large
- Accuracy of default classifier is 50
21Study 2 Word Categories
Francisco Pereira
- Family members
- Occupations
- Tools
- Kitchen items
- Dwellings
- Building parts
- 4 legged animals
- Fish
- Trees
- Flowers
- Fruits
- Vegetables
22Word Categories Study
- Ten neurologically normal subjects
- Stimulus
- 12 blocks of words
- Category name (2 sec)
- Word (400 msec), Blank screen (1200 msec) answer
- Word (400 msec), Blank screen (1200 msec) answer
-
- Subject answers whether each word in category
- 32 words per block, nearly all in category
- Category blocks interspersed with 5 fixation
blocks
23Training Classifier for Word Categories
- Learn fMRI(t) ? word-category(t)
- fMRI(t) 8470 to 11,136 voxels, depending on
subject - Training methods
- train ten single-subect classifiers
- kNN (k 1,3,5)
- Gaussian Naïve Bayes ? P(fMRI(t) word-category)
24Study 2 Results
- Classifier outputs ranked list of classes
- Evaluate by the fraction of classes ranked ahead
of true class - 0perfect, 0.5random, 1.0 unbelievably poor
25Study 3 Syntactic Ambiguity
Rebecca Hutchinson
- Is subject reading ambiguous or unambiguous
sentence? - The experienced soldiers warned about the
dangers conducted the midnight raid. - The experienced solders spoke about the dangers
before the midnight raid.
26Study 3 Results
- 10 examples, 4 subjects
- Almost random results if no feature selection
used - With feature selection
- SVM - 77 accuracy
- GNB - 75 accuracy
- 5NN 72 accuracy
27Feature Selection
- Five feature selection methods
- All (all voxels available)
- Active (n most active available voxels according
to a t-test) - RoiActive (n most active voxels in each ROI)
- RoiActiveAvg (average of the n most active
voxels in each ROI) - Disc (n most discriminating voxels according to
a trained classifier) - Active works best
28Feature Selection
29Summary
- Successful training of classifiers for
instantaneous cognitive state in three studies - Cross subject classifiers trained by abstracting
to anatomically defined ROIs - Feature selection and abstraction are essential
30Research Opportunities
- Learning temporal models
- HMMs, Temporal Bayes nets,
- Discovering useful data abstractions
- ICA, PCA, hidden layers,
- Linking cognitive states to cognitive models
- ACT-R, CAPS
- Merging data from multiple sources
- fMRI, ERP, reaction times,
31End of talk