Title: Stephen Pizer
1Tutorial Statistics of Object Geometry
- Stephen Pizer
- Medical Image Display Analysis Group
- University of North Carolina, USA
- with credit to
- T. Fletcher, A. Thall, S. Joshi, P. Yushkevich,
G. Gerig
10 October 2002
2Uses of Statistical Geometric Characterization
- Medical science determine geometric ways in
which pathological and normal classes differ - Diagnostic determine if particular patients
geometry is in pathological or normal class - Educational communicate anatomic variability in
atlases - Priors for segmentation
- Monte Carlo generation of images
3Object RepresentationObjectives
- Relation to other instances of the shape class
- Representing the real world
- Deformation while staying in shape class
- Discrimination by shape class
- Locality
- Relation to Euclidean space/projective Euclidean
space - Matching image data
4Geometric aspects Invariants and correspondence
- Desire An image space geometric representation
that - is at multiple levels of scale (locality)
- at one level of scale is based on the object
- and at lower levels based on objects figures
- at each level recognizes invariances associated
with shape - provides positional and orientational and metric
correspondence across various instances of the
shape class
5Object Representations
- Atlas voxels with a displacement at each voxel
Dx(x) - Set of distinguished points xi with a
displacement at each - Landmarks
- Boundary points in a mesh
- With normal b (x,n)
- Loci of medial atoms m (x,F,r,q) or end
atom (x,F,r,q,h)
6Continuous M-reps B-splines in (x,y,z,r)
Yushkevich
7Building an Object Representation from Atoms a
- Sampled
- aij
- can have inter-atom mesh (active surface)
- Parametrized
- a(u,v)
- e.g., spherical harmonics, where coefficients
become representation - e.g., quadric or superquadric surfaces
- some atom components are derivatives of others
8Object representation Parametrized Boundaries
- Parametrized boundaries x(u,v)
- n(u,v) is normalized ?x/?u ? ?x/?v
- Coefficients of decompositions
- x(u,v) Si ci f i(u,v)
- Spherical harmonics (u,v) latitude, longitude
- Sampled point positions are linear in
coefficients Axc
9Object representation Parametrized Medial Loci
- Parametrized medial loci m(u,v) x,r(u,v)
- n(u,v) is normalized ?x/?u ? ?x/?v
- ?xr(u,v) -cos(q)b
- gradient per distance on x(u,v)
10Sampled medial shape representation Discrete
M-rep slabs (bars)
- Meshes of medial atoms
- Objects connected as host, subfigures
- Multiple such objects, interrelated
11Interpolating Medial Atoms in a Figure
- Interpolate x, r via B-splines Yushkevich
- Trimming curve via rlt0 at outside control points
- Avoids corner problems of quadmesh
- Yields continuous boundary
- Via modified subdivision surface Thall
- Approximate orthogonality at sail ends
- Interpolated atoms via boundary and distance
- At ends elongation h needs also to be
interpolated - Need to use synthetic medial geometry Damon
Medial sheet
Implied boundary
12End Atoms (x,F,r,q,h)
Medial atom with one more parameter elongation h
13Sampled medial shape representation M-rep tube
figures
- Same atoms as for slabs
- r is radius of tube
- sails are rotated about b
- Chain rather than mesh
x rRb,n(q)b
xrRb,n(-q)b
14For correspondence Object-intrinsic
coordinatesGeometric coordinates from m-reps
- Single figure
- Medial sheet (u,v) (u) in 2D
- t medial side
- t signed r-propl dist from implied boundary
- 3-space (u,v,t, t)
- Implied boundary (u,v,t)
15Sampled medial shape representation Linked m-rep
slabs
- Linked figures
- Hinge atoms known in figural coordinates (u,v,t)
of parent figure - Other atoms known in the medial coordinates of
their neighbors
16Figural Coordinates for Object Made From
Multiple Attached Figures
- Blend in hinge regions
- w(d1/r1 - d2 /r2 )/T
- Blended d/r when w lt1 and u-u0 lt T
- Implicit boundary (u,w, t)
- Or blend by subdivision surface
w
17Figural Coordinates for Multiple Objects
- Inside objects or on boundary
- Per object
- In neighbor objects coordinates
- Interobject space
- In neighbor objects coordinates
- Far outside boundary (u,v,t, t) via distance
(scale) related figural convexification ?? - ??
18Heuristic Medial Correspondence
Original (Spline Parameter)
Arclength
1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
1
Radius
Coordinate Mapping
19Continuous Analytical Features
- Can be sampled arbitrarily.
- Allow functional shape analysis
- Possible at many scales medial, bdry, other
object
Boundary texture scale
Medial Curvature
20Feature-Based Correspondence on Medial Locus by
Statistical Registration of Features
curvature
dr/ds
dr/ds
Also works in 3D
21What is Statistical Geometric Characterization
- Given a population of instances of an object
class - e.g., subcortical regions of normal males of age
30 - Given a geometric representation z of a given
instance - e.g., a set of positions on the boundary of the
object - and thus the description zi of the ith instance
- A statistical characterization of the class is
the probability density p(z) - which is estimated from the instances zi
22Benefits of Probabilistically Describing Anatomic
Region Geometry
- Discrimination among geometric classes, Ck
- Compare probabilities p(z Ck)
- Comprehension of asymmetries or distinctions of
classes - Differences between means
- Difference between variabilities
- Segmentation by deformable models
- Probability of geometry p(z) provides prior
- Provides object-intrinsic coords in which
multiscale image probabilities p(Iz) can be
described - Educational atlas with variability
- Monte Carlo generation of shapes, of diffeo-
morphisms, to produce pseudo-patient test images
23Necessary Analysis Provisions To Achieve Locality
Training Feasibility
- Multiple scales
- Allows few random variables per scale
- At each scale, a level of locality (spatial
extent) associated with its random variable - Positional correspondence
- Across instances
- Between scales
Large scale Smaller scale
24Discussion of Scale
- Spatial aspects of a geometric feature
- Its position
- Its spatial extent
- Region summarized
- Level of detail captured
- Residues from larger scales
- Distances to neighbors with which it has a
statistical relationship - Markov random field
- Cf PDM, spherical harmonics, dense Euclidean
positions, landmarks, m-reps
Large scale Smaller scale
25Scale Situations in Various Statistical Geometric
Analysis Approaches
Level of Detail
Global coef for Multidetail feature, Detail
residues each level of detail, E.g., spher.
harm. E.g., boundary pt. E.g., object hierarchy
Fine
Coarse
Location
Location
Location
26Principles of Object-Intrinsic Coordinates at a
Scale Level
- Coordinates at one scale must relate to parent
coordinates at next larger scale - Coordinates at one scale must be writable in
neighbors coordinate system - Statistically stable features at all scales must
be relatable at various scale levels
27Figurally Relevant Spatial Scale Levels
Primitives and Neighbors
- Multi-object complex
- Individual object
- multiple figures
- in geom. reln to neighbors
- in relation to complex
- Individual figure
- mesh of medial atoms
- subfigs in relation to neighbors
- in relation to object
- Figural section
- multiple figural sections
- each centered at medial atom
- medial atoms in relation to neigbhors
- in relation to figure
- Figural section residue, more finely spaced, ..
- gt multiple boundary sections (vertices)
- Boundary section
- vertices in relation to vertex neighbors
- in relation to figural section
28Multiscale Probability Leads to Trainable
Probabilities
- If the total geometric representation z is at
all scales or smallest scale, it is not stably
trainable with attainable numbers of training
cases, so multiscale - Let zk be the geometric representation at scale
level k - Let zki be the ith geometric primitive at scale
level k - Let N(zki) be the neighbors of zki (at level k)
- Let P(zki) be the parent of zki (at level k-1)
- Probability via Markov random fields
- p(zki P(zki), N(zki) )
- Many trainable probabilities
- If p(zki rel. to P(zki), zki rel. to N(zki) )
- Requires parametrized probabilities
29Multi-Scale-Level Image AnalysisGeometry
Probability
- Multiscale critical for effectiveness with
efficiency - O(number of smallest scale primitives)
- Markov random field probabilistic basis
- Vs. methods working at small scale only or at
global scale small scale only
30Multi-Scale-Level Image Analysisvia M-reps
- Thesis multi-scale-level image analysis is
particularly well supported by representation
built around m-reps - Intuitive, medically relevant scale levels
- Object-based positional and orientational
correspondence - Geometrically well suited to deformation
31Geometric Typicality MetricsStatistical Metrics
32Statistics/Probability Aspects Principal
component analysis
- Any shape, x, can be written as
- x xmean Pb r
- log p(x) f(b1, bt,r2)
b1
33Visualizing Measuring Global Deformation
- Shape Measurement
- Modes of shape variation across patients
- Measurement z amount of each mode
c cmean z1s1p1
c cmean z2s2p2
34Statistics/Probability Aspects Markov random
fields
- Suppose zT (z1 zn)
- p(zi zj, j?i)
- p(zi zk k a neighbour of i)
- (i. e., assume sparse covariance matrix)
- Need only evaluate O(n) terms to optimize p(z) or
p(z image) - Can only evaluate p(zi), i.e., locally
- Interscale within scale by locality
35Multiscale Geometry and Probability
- If z is at all scales or smallest scale, it is
not stably trainable, so multiscale - Let zk be the geometric repn at scale k
- Let zki be the ith geometric primitive at scale k
- Let N(zki) be the neighbors of zki
- Let P(zki) be the parent of zki
- Let C(zki) be the children of zki
- Probability via Markov random fields
- p(zki P(zki), N(zki), C(zki) )
- Many trainable probabilities
- Requires parametrized probabilities for training
36Examples with m-reps components p(zki P(zki),
N(zki), C(zki) )
- z1 (necessarily global) similarity transform for
body section - z2i similarity transform for the ith object
- Neighbors are adjacent (perhaps abutting) objects
- z3i similarity transform for the ith figure of
its object in its parents figural coordinates - Neighbors are adjacent (perhaps abutting) figures
- z4i medial atom transform for the ith medial
atom - Neighbors are adjacent medial atoms
- z5i medial atom transform for the ith medial
atom residue at finer scale (see next slide) - z6i boundary offset along medially implied
normal for the ith boundary vertex - Neighbors are adjacent vertices
- Probability via Markov random fields
- p(zki P(zki), N(zki), C(zki) )
- Many trainable probabilities
- Requires parametrized probabilities for training
37Multiscale Geometry and Probability for a Figure
coarse, global
- Geometrically ? smaller scale
- Interpolate (1st order) finer spacing of atoms
- Residual atom change, i.e., local
- Probability
- At any scale, relates figurally homologous points
- Markov random field relating medial atom with
- its immediate neighbors at that scale
- its parent atom at the next larger scale and the
corresponding position - its children atoms
coarse resampled
fine, local
38Published Methods of Global Statistical Geometric
Characterization in Medicine
- Global variability
- via principal component analysis on
- features globally, e.g., boundary points or
landmarks, or - global features, e.g., spherical harmonic
coefficients for boundary
- Global difference
- via linear (or other) discriminant on features
- globally or on global features
- Globally based diagnosis
- via linear (or other) discriminant on features
- globally or on global features
- Example authors BooksteinGolland Gerig
Joshi Thompson TogaTaylor
39Published Methods of Local Statistical Geometric
Characterization
A
- Local variability
- via principal component analysis on features
globally or on global features, plus display of
local properties of principal component - Local difference
- via linear (or other) discriminant on global
geometric primitives, plus display of local
properties of discriminant direction - On displacement vectors
signed, unsigned re inside/outside - Example authors Gerig Golland Joshi
Taylor Thompson Toga
Displacement significance Schizophrenic vs.
control hippocampus
40Shortcomings of Published Methods of Statistical
Geometric Characterization
- Unintuitive
- Would like terms like bent, twisted, pimpled,
constricted, elongated, extra figure - Frequently nonlocal or local wrt global template
- Depends on getting correspondence to template
correct - Need where the differences are in object
coordinates - Which object, which figure, where in figure,
where on boundary surface - Requires infeasible number of training cases
- Due to too many random variables (features)
41Overcoming Shortcomings of Methods of Statistical
Geometric Characterization
- Intuitive
- Figural (medial) representation provides terms
like bent, twisted, pimpled, constricted,
elongated, extra figure - Local
- Hierarchy by scale level provides appropriate
level of locality - Object figure based hierarchy yields intuitive
locality and good positional correspondences - Which object, which figure, where in figure,
where on boundary surface - Positional correspondences across training cases
scale levels - Trainable by feasible number of cases
- Few features in residue between scale levels
- Relative to description at next larger scale
level - Relative to neighbors at same scale level
42Conclusions re Object Based Image Analysis
- Work at multiple levels of scale
- At each scale use representation appropriate for
that scale - At intermediate scales
- Represent medially
- Sense at (implied) boundary
- Papers at midag.cs.unc.edu/pubs/papers
43Extensions
- Variable topology
- jump diffusion (local shape)
- level set?
- Active Appearance Models
- shape and intensity
- explaining the image
- iterative matching algorithm
44Recommended Readings
- For deformable sampled boundary models T Cootes,
A Hill, CJ Taylor (1994). Use of active shape
models for locating structures in medical images.
Image Vision Computing 12 355-366. - For deformable parametrized boundary models
Kelemen, Gerig, et al - For m-rep based shape Pizer, Fritsch, et al,
IEEE TMI, Oct. 1999 - For 3D deformable m-reps Joshi, Pizer, et al,
IPMI 2001 (Springer LNCS 2082) Pizer, Joshi, et
al, MICCAI 2001 (Springer LNCS 2208)
45Recommended Readings
- For Procrustes, landmark based deformation
(Bookstein), shape space (Kendall) especially
understandable in Dryden Mardia, Statistical
Shape Analysis - For iterative conditional posterior, pixel
primitive based shape Grenander Miller Blake
Christensen et al