Indexing and Retrieving Dynamic Brain Images

1
Indexing and Retrieving Dynamic Brain Images

• Kickoff Meeting
• March 28, 2003 SCILS, Rutgers
• Paul B. Kantor, PI
• Stephen J. Hanson , Co-PI

2
Overall Model

• A brain event is characterized by an
  activation a(t) which is an (imaginary?)
  pointer that moves through the brain as neurons
  act sequentially in some mental process.
• The set of points t,a(t) characterizes the
  mental process

3
Model (2)

• The set of points can be conceived of as a graph
  in 4-dimensional space with coordinates x,t.
• The graph can be concisely represented by graph
  indexing techniques
• Those indexing techniques can be used to support
  retrieval of related mental processes even if
  the associated circumstances are not apparently
  similar.

4
Experimental-Analytic Paradigm

• Do until funds are exhausted
• Experiments produce data
• Analysts reduce it to graphs
• Study of graphs suggests new experiments
• loop
• Funds presently
• Rutgers ISTC Pilot funds 25K
• NSF 2. Millions/3years
• McDonnell Foundation 500K/
• NIH R01 (planning)

5
Roles of Key Personnel

• J. Cohen - design, conduct experiments
• S. Hanson - co-PI design expts fusion time
  resolution
• P. Kantor - co-PI design indexing and retrieval
• D. Silver - reduce activations to centroids
• S. Dickinson - index and retrieve graphs
• L. Shepp - improve time resolution -- physics
  level
• B. Bly - RUMBA software
• C. Hanson - design, conduct expts manage data

6
Additional Key Personnel

• Graduate Students
• Ulukbek Ibraev -- graph finding and indexing
• Xiaosong Yuan -- correlation and image analysis
• Arnav Sheth -- diffusion of signal around
  activation
• Yaroslav Halchenko --fusion EEG/fMRI
• Adi Zaimi data collection and design,
  simulations, modeling
• Other Personnel
• Donovan Rebbechi--programming, RUMBA architecture
• Mike Edwards--programming, archive maintenance
• Barak Pearlmutter--theory and algorithm
  development
• Toshi Matsuka-- cognitive modeling and neural
  computation

7
Threats to the Undertaking

• A. Time resolution of fMRI is very poor
• B. Hemodynamic response wrt to Neural spike
  firing is poorly understood
• C. Signals may leap across brain on long neurons
• D. All brain activation may be connectionist/distr
  ibuted
• E. All other threats (Tell the Wigner story)
• Todays focus
• Improving time resolution Dynamic Activity
  Modeling

8
Improving time resolution

• New basis functions for selected regions (today)
• Shepp et al
• Fusion (at a later meeting)- Hanson et al
• Ingenious spacing of stimuli and collection
• Kantor

9
Dynamic brain activity modeling (BOLD) Wavelets
(today) --Daubechies Cluster following (at a
later meeting) --Kantor Silver Skeletonization
(at a later meeting) --Dickinson
10
TOOLS and ARCHIVE RUMBA tools (at another
meeting) --Ben Bly ArchiveRUMBA
sharing --Donovan Rebbechi AIR- RUMBA --Donovan
Rebbechi
11
Data Collection Cognitive Theory
Continuous fMRI paradigms Event Perception (at
later meeting) --Catherine Hanson Similarity
based fMRI paradigms Flanker task (at a later
meeting) --Jonathan Cohen Allegra User
Group--(set up by Cohenwww. Fill in)
12
Administrivia

• Although we are all hard at work
• we have just completed the contract
  Rutgers-Princeton
• the NSF would like to know what we are
  accomplishing, to decide whether to give us the
  next part of the funding
• Materials so far submitted are at the web site
  http//scils.rutgers.edu/kantor/SECRET/brain/Annu
  alReport/draft.html
• Please send material to me.
• kantor_at_scils.rutgers.edu

13
Project Details Improved Detection methods
Neural Networks and Object Recognition(Hanson,
Matsuka Haxby)

• Problem Determining the sensitivities of voxels
  in ventral temporal lobe for object recognition.
• Method
• 1) Neural Networks as a nonlinear classifierno
  contrast or baseline reference!
• 2) sensitivity analysis with noisei N(0, SDi)
• Results 
• (a) individual voxels are sensitive in
  recognition of multiple categories.
• (b) patterns of voxels sensitivities are
  somewhat similar for many categories.

14
Object Recognition (cont.)(Hanson, Matsuka
Haxby)
Face House Cat Bottle

• Results (figure)
• Use can be used for detection of complex signals
  and input intodynamic indexing algorithms

Scissor Shoe Chair Random
15
Project DetailsRUMBA tools D. Rebbechi, B.
Bly, S. Hanson-Newark C Library Python
scripting/python environment support Command line
tools Work in progress GUI
16

• Archive
• RUMBA sharingNapster Brain.
• D. Rebbechi, B. Bly, S. Hanson
• Encrypted data archive
• Data sharing requires the downloader to obtain
  the owners consent
• A data sharing agreement is represented in a
  cryptographically signed contract'
• XML repository summarizing data is publicly
  accessible but data is hidden.

17

• AIR- RUMBA (automatic image registraion)
• D. Rebbechi, B. Bly, S. Hanson
• Rigid body motion correction
• Affine inter-modality registration using
  AIR-inspired cost function.
• Polynomial warp transformation for template based
  image registration
• Future directions stochastic gradient method,
  using mutual information

18
FMRI/EEG FUSIONHanson, Halchenklo Zaimi

• Preprocessing
• EEG noise/artifacts(eyeblinks) removal ICA
• fMRI baseline preprocessing
• Initialization
• Merge inverse solutions for EEG or fMRI
• Use LP to get 'worst-ever' approximate solution
• Optimizationto concurrently reconstruct both
  signals F and E while satisfying constraints
  on fused modality S smoothness in time/space.
  Each signal has some temporal (fMRI) or spatial
  (EEG) influence on the other through forward
  equations.

19
(No Transcript)
20
Data Collection Data Flow (C. Hanson, Edwards,
Rebbechi)
21
Data Archive (C. Hanson, Edwards, Rebbechi)
22
Data Sets So far (C. Hanson, Edwards, Rebbechi)

• Oddball task (C Hanson, Rutgers) subject
  responds when an oddball (a nonconforming
  stimulus) appears in a series of identical
  stimuli
• Event perception task (C Hanson, Rutgers)
  subject asked to parse action sequences into
  meaningful units (events)
• Incremental stimulus recognition task (C Hanson,
  Rutgers) subject probed for identity of
  occluded objects that are incrementally revealed
• Noise and motor and auditory tasks( (Bly, C.
  Hanson Rutgers)- subject either at rest or doing
  simple motor task (finger tapping) or listening
  to auditory input
• 1-back task (Haxby, Princeton) subject is
  presented with a series of stimuli and
  periodically asked to decide if current stimulus
  was presented in the previous trial
• Morality task (Cohen, Princeton) subject asked
  to make moral decisions about fictitious
  situations
• Flanker task (Cohen, Princeton) subject is
  asked to report the directionality of an arrow
  when flanking stimuli are consistent or
  inconsistent with target