Title: Generative models for automated brain MRI segmentation
1Generative models for automated brain MRI
segmentation
- Koen Van Leemput
- Athinoula A. Martinos Center for Biomedical
Imaging - Department of Radiology, MGH
- Harvard Medical School, USA
- Computer Science and Artificial Intelligence
Laboratory - Massachusetts Institute of Technology, USA
2MRI of the brain
- Magnetic resonance imaging
- Harmless
- Three dimensional (3-D)
- High soft tissue contrast
- High spatial resolution
- Extremely versatile
- Possibly multi-spectral
voxel
Ideal for studying the living human brain
3Segmentation of brain MRI
- Delineating structures of interest in the images
- Segmentation is important
- Basic neuroscience
- Uncovering disease mechanisms
- Diagnosis, treatment planning, and follow-up
- Clinical drug trials
-
- Automated computational methods are needed
4Overview
- Segmentation basics modeling and inference
- Modeling MRI bias fields
- Mesh-based brain atlases
- Whole-brain segmentation
5Overview
- Segmentation basics modeling and inference
- Modeling MRI bias fields
- Mesh-based brain atlases
- Whole-brain segmentation
6The problem to be solved
MRI image
7The problem to be solved
MRI image
Label image
8One solution generative modeling
- Formulate a statistical model of how an MRI image
is formed - The model depends on some parameters
labeling model
imaging model
Label image
MRI image
9Segmentation inverse problem
MRI image
Label image
10Segmentation inverse problem
MRI image
Label image
- Bayesian inference
- Start from our statistical model of image
formation - Play with the mathematical rules of probability
11Bayesian inference
- Practical approximation
- Involves two optimizations
- First estimate the optimal model parameters
- Then find the optimal segmentation based on those
parameter estimates
12Example Gaussian mixture model
labeling model
imaging model
MRI image
Label image
- The label in each voxel is drawn independently
with a probability for tissue type k - Assume a uniform prior for the
labeling model parameters
13Example Gaussian mixture model
labeling model
imaging model
MRI image
Label image
- The intensity in each voxel is drawn
independently from a Gaussian distribution
associated with its label - The imaging model parameters are the mean
and variance of each Gaussian - Assume a uniform prior
14Example Gaussian mixture model
three labels
Model parameters
are unknown
Mean and variance of each Gaussian
Relative weight of each Gaussian
15Optimization 1 parameter estimation
- Given an MRI image to be segmented, what is the
MAP parameter estimate ? - Parameter optimization with an Expectation
Maximization (EM) algorithm
- Repeatedly maximize a lower bound to the
objective function - Iterative parameter optimizer using only
closed-form parameter updates!
current estimate
16Optimization 1 parameter estimation
17Optimization 1 parameter estimation
18Optimization 2 segmentation
white matter
Upon completion of the parameter estimation
algorithm, assign each voxel to the MAP label
CSF
gray matter
19Overview
- Segmentation basics modeling and inference
- Modeling MRI bias fields
- Mesh-based brain atlases
- Whole-brain segmentation
20MRI bias field artifact
- Intensity inhomogeneities across the image area
- Imaging artifact in MRI
- equipment limitations
- patient-induced electrodynamic interactions
MRI data
after intensity windowing
21MRI bias field artifact
- Causes segmentation errors with our segmentation
procedure so far
22MRI bias field artifact
Causes segmentation errors with our segmentation
procedure so far
23Improved imaging model
labeling model
imaging model
MRI image
Label image
24Improved imaging model
labeling model
imaging model
MRI image
Label image
old model
25Improved imaging model
labeling model
imaging model
MRI image
Label image
polynomial bias field model
old model
26Model parameter estimation
- Polynomial coefficients are part of the model
parameters - Parameter optimization with a Generalized
Expectation Maximization (GEM) algorithm
- Repeatedly improve a lower bound to the objective
function - Iterative parameter optimizer using only
closed-form parameter updates! Van Leemput et
al., IEEE TMI 1999
current estimate
27Example
28Example
MRI data
White matter without bias field model
White matter with bias field model
Estimated bias field
29Example
MRI data
White matter without bias field model
White matter with bias field model
Estimated bias field
30Overview
- Segmentation basics modeling and inference
- Modeling MRI bias fields
- Mesh-based brain atlases
- Whole-brain segmentation
31Improving the labeling model
labeling model
imaging model
MRI image
Label image
- So far our labeling model just expresses the
relative frequency of occurrence of different
labels - Too simplistic for segmenting the brain into 30
subregions
A more realistic labeling model is needed!
32Improving the labeling model
33Improving the labeling model
Try to find the underlying probability
distribution
Manual segmentations in N individuals (training
data)
34Modeling the training data (2-D)
Triangular mesh representation
35Modeling the training data (2-D)
atlas
- Assign label probabilities to each mesh node
- Flat prior
- Label probabilities are linearly interpolated
over triangle areas
36Modeling the training data (2-D)
atlas
Mesh node positions are sampled from a
topology-preserving Markov random field prior
warped atlases
knob that controls the flexibility of the atlas
warp
37Modeling the training data (2-D)
atlas
Example segmentations are sampled according to
the deformed atlases
warped atlases
example segmentations
38Bayesian inference Van Leemput, IEEE TMI 2009
- Given a collection of manual segmentations
- what is the most probable atlas?
- what is the most likely value of the parameter
controlling the flexibility of the deformations? - what is the most likely mesh
- representation?
- Good models explain regularities in the manual
segmentations - Automatically yields sparse representations that
explicitly avoid overfitting to the training data - cf. Minimum Description Length
39Example atlas
Derived from manual segmentations of 36 brain
substructures in 4 individuals
Has average shape
40Overview
- Segmentation basics modeling and inference
- Modeling MRI bias fields
- Mesh-based brain atlases
- Whole-brain segmentation
41Whole-brain segmentation
labeling model
imaging model
MRI image
Label image
- Tetrahedral mesh-based atlas
- The labeling model parameters are the
location of the mesh nodes - The prior is the topology-preserving
MRF model (penalizes deformations)
42Whole-brain segmentation
labeling model
imaging model
MRI image
Label image
polynomial bias field model
Gaussian mixture model
43Whole-brain segmentation
- Model parameter estimation
- Fully automated segmentation procedure
- No need for pre-processing (skull stripping, bias
field corr., ) - Automatically adapts to different scanners and
acquisition sequences! - Fast!
Improve the imaging model parameters
(Generalized Expectation-Maximization closed-for
m expressions) Improve the atlas warp
(registration gradient in analytical form)
44Examples (validation under way)
45Examples (validation under way)
46Examples (validation under way)
47Examples (validation under way)
48Examples (validation under way)
49Examples (validation under way)
50