Stephen Pizer

1
Tutorial Statistics of Object Geometry

• Stephen Pizer
• Medical Image Display Analysis Group
• University of North Carolina, USA
• with credit to
• T. Fletcher, A. Thall, S. Joshi, P. Yushkevich,
  G. Gerig

10 October 2002
2
Uses of Statistical Geometric Characterization

• Medical science determine geometric ways in
  which pathological and normal classes differ
• Diagnostic determine if particular patients
  geometry is in pathological or normal class
• Educational communicate anatomic variability in
  atlases
• Priors for segmentation
• Monte Carlo generation of images

3
Object RepresentationObjectives

• Relation to other instances of the shape class
• Representing the real world
• Deformation while staying in shape class
• Discrimination by shape class
• Locality
• Relation to Euclidean space/projective Euclidean
  space
• Matching image data

4
Geometric aspects Invariants and correspondence

• Desire An image space geometric representation
  that
• is at multiple levels of scale (locality)
• at one level of scale is based on the object
• and at lower levels based on objects figures
• at each level recognizes invariances associated
  with shape
• provides positional and orientational and metric
  correspondence across various instances of the
  shape class

5
Object Representations

• Atlas voxels with a displacement at each voxel
  Dx(x)
• Set of distinguished points xi with a
  displacement at each
• Landmarks
• Boundary points in a mesh
• With normal b (x,n)
• Loci of medial atoms m (x,F,r,q) or end
  atom (x,F,r,q,h)

6
Continuous M-reps B-splines in (x,y,z,r)
Yushkevich
7
Building an Object Representation from Atoms a

• Sampled
• aij
• can have inter-atom mesh (active surface)
• Parametrized
• a(u,v)
• e.g., spherical harmonics, where coefficients
  become representation
• e.g., quadric or superquadric surfaces
• some atom components are derivatives of others

8
Object representation Parametrized Boundaries

• Parametrized boundaries x(u,v)
• n(u,v) is normalized ?x/?u ? ?x/?v
• Coefficients of decompositions
• x(u,v) Si ci f i(u,v)
• Spherical harmonics (u,v) latitude, longitude
• Sampled point positions are linear in
  coefficients Axc

9
Object representation Parametrized Medial Loci

• Parametrized medial loci m(u,v) x,r(u,v)
• n(u,v) is normalized ?x/?u ? ?x/?v
• ?xr(u,v) -cos(q)b
• gradient per distance on x(u,v)

10
Sampled medial shape representation Discrete
M-rep slabs (bars)

• Meshes of medial atoms
• Objects connected as host, subfigures
• Multiple such objects, interrelated

11
Interpolating Medial Atoms in a Figure

• Interpolate x, r via B-splines Yushkevich
• Trimming curve via rlt0 at outside control points
• Avoids corner problems of quadmesh
• Yields continuous boundary
• Via modified subdivision surface Thall
• Approximate orthogonality at sail ends
• Interpolated atoms via boundary and distance
• At ends elongation h needs also to be
  interpolated
• Need to use synthetic medial geometry Damon

Medial sheet
Implied boundary
12
End Atoms (x,F,r,q,h)
Medial atom with one more parameter elongation h
13
Sampled medial shape representation M-rep tube
figures

• Same atoms as for slabs
• r is radius of tube
• sails are rotated about b
• Chain rather than mesh

x rRb,n(q)b
xrRb,n(-q)b
14
For correspondence Object-intrinsic
coordinatesGeometric coordinates from m-reps

• Single figure
• Medial sheet (u,v) (u) in 2D
• t medial side
• t signed r-propl dist from implied boundary
• 3-space (u,v,t, t)
• Implied boundary (u,v,t)

15
Sampled medial shape representation Linked m-rep
slabs

• Linked figures
• Hinge atoms known in figural coordinates (u,v,t)
  of parent figure
• Other atoms known in the medial coordinates of
  their neighbors

16
Figural Coordinates for Object Made From
Multiple Attached Figures

• Blend in hinge regions
• w(d1/r1 - d2 /r2 )/T
• Blended d/r when w lt1 and u-u0 lt T
• Implicit boundary (u,w, t)
• Or blend by subdivision surface

w
17
Figural Coordinates for Multiple Objects

• Inside objects or on boundary
• Per object
• In neighbor objects coordinates
• Interobject space
• In neighbor objects coordinates
• Far outside boundary (u,v,t, t) via distance
  (scale) related figural convexification ??
• ??

18
Heuristic Medial Correspondence
Original (Spline Parameter)
Arclength
1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
1
Radius
Coordinate Mapping
19
Continuous Analytical Features

• Can be sampled arbitrarily.
• Allow functional shape analysis
• Possible at many scales medial, bdry, other
  object

Boundary texture scale
Medial Curvature
20
Feature-Based Correspondence on Medial Locus by
Statistical Registration of Features
curvature
dr/ds
dr/ds
Also works in 3D
21
What is Statistical Geometric Characterization

• Given a population of instances of an object
  class
• e.g., subcortical regions of normal males of age
  30
• Given a geometric representation z of a given
  instance
• e.g., a set of positions on the boundary of the
  object
• and thus the description zi of the ith instance
• A statistical characterization of the class is
  the probability density p(z)
• which is estimated from the instances zi

22
Benefits of Probabilistically Describing Anatomic
Region Geometry

• Discrimination among geometric classes, Ck
• Compare probabilities p(z Ck)
• Comprehension of asymmetries or distinctions of
  classes
• Differences between means
• Difference between variabilities
• Segmentation by deformable models
• Probability of geometry p(z) provides prior
• Provides object-intrinsic coords in which
  multiscale image probabilities p(Iz) can be
  described
• Educational atlas with variability
• Monte Carlo generation of shapes, of diffeo-
  morphisms, to produce pseudo-patient test images

23
Necessary Analysis Provisions To Achieve Locality
Training Feasibility

• Multiple scales
• Allows few random variables per scale
• At each scale, a level of locality (spatial
  extent) associated with its random variable
• Positional correspondence
• Across instances
• Between scales

Large scale Smaller scale
24
Discussion of Scale

• Spatial aspects of a geometric feature
• Its position
• Its spatial extent
• Region summarized
• Level of detail captured
• Residues from larger scales
• Distances to neighbors with which it has a
  statistical relationship
• Markov random field
• Cf PDM, spherical harmonics, dense Euclidean
  positions, landmarks, m-reps

Large scale Smaller scale
25
Scale Situations in Various Statistical Geometric
Analysis Approaches
Level of Detail
Global coef for Multidetail feature, Detail
residues each level of detail, E.g., spher.
harm. E.g., boundary pt. E.g., object hierarchy
Fine
Coarse
Location
Location
Location
26
Principles of Object-Intrinsic Coordinates at a
Scale Level

• Coordinates at one scale must relate to parent
  coordinates at next larger scale
• Coordinates at one scale must be writable in
  neighbors coordinate system
• Statistically stable features at all scales must
  be relatable at various scale levels

27
Figurally Relevant Spatial Scale Levels
Primitives and Neighbors

• Multi-object complex
• Individual object
• multiple figures
• in geom. reln to neighbors
• in relation to complex
• Individual figure
• mesh of medial atoms
• subfigs in relation to neighbors
• in relation to object
• Figural section
• multiple figural sections
• each centered at medial atom
• medial atoms in relation to neigbhors
• in relation to figure
• Figural section residue, more finely spaced, ..
• gt multiple boundary sections (vertices)
• Boundary section
• vertices in relation to vertex neighbors
• in relation to figural section

28
Multiscale Probability Leads to Trainable
Probabilities

• If the total geometric representation z is at
  all scales or smallest scale, it is not stably
  trainable with attainable numbers of training
  cases, so multiscale
• Let zk be the geometric representation at scale
  level k
• Let zki be the ith geometric primitive at scale
  level k
• Let N(zki) be the neighbors of zki (at level k)
• Let P(zki) be the parent of zki (at level k-1)
• Probability via Markov random fields
• p(zki P(zki), N(zki) )
• Many trainable probabilities
• If p(zki rel. to P(zki), zki rel. to N(zki) )
• Requires parametrized probabilities

29
Multi-Scale-Level Image AnalysisGeometry
Probability

• Multiscale critical for effectiveness with
  efficiency
• O(number of smallest scale primitives)
• Markov random field probabilistic basis
• Vs. methods working at small scale only or at
  global scale small scale only

30
Multi-Scale-Level Image Analysisvia M-reps

• Thesis multi-scale-level image analysis is
  particularly well supported by representation
  built around m-reps
• Intuitive, medically relevant scale levels
• Object-based positional and orientational
  correspondence
• Geometrically well suited to deformation

31
Geometric Typicality MetricsStatistical Metrics
32
Statistics/Probability Aspects Principal
component analysis

• Any shape, x, can be written as
• x xmean Pb r
• log p(x) f(b1, bt,r2)

b1
33
Visualizing Measuring Global Deformation

• Shape Measurement
• Modes of shape variation across patients
• Measurement z amount of each mode

c cmean z1s1p1
c cmean z2s2p2
34
Statistics/Probability Aspects Markov random
fields

• Suppose zT (z1 zn)
• p(zi zj, j?i)
• p(zi zk k a neighbour of i)
• (i. e., assume sparse covariance matrix)
• Need only evaluate O(n) terms to optimize p(z) or
  p(z image)
• Can only evaluate p(zi), i.e., locally
• Interscale within scale by locality

35
Multiscale Geometry and Probability

• If z is at all scales or smallest scale, it is
  not stably trainable, so multiscale
• Let zk be the geometric repn at scale k
• Let zki be the ith geometric primitive at scale k
• Let N(zki) be the neighbors of zki
• Let P(zki) be the parent of zki
• Let C(zki) be the children of zki
• Probability via Markov random fields
• p(zki P(zki), N(zki), C(zki) )
• Many trainable probabilities
• Requires parametrized probabilities for training

36
Examples with m-reps components p(zki P(zki),
N(zki), C(zki) )

• z1 (necessarily global) similarity transform for
  body section
• z2i similarity transform for the ith object
• Neighbors are adjacent (perhaps abutting) objects
• z3i similarity transform for the ith figure of
  its object in its parents figural coordinates
• Neighbors are adjacent (perhaps abutting) figures
• z4i medial atom transform for the ith medial
  atom
• Neighbors are adjacent medial atoms
• z5i medial atom transform for the ith medial
  atom residue at finer scale (see next slide)
• z6i boundary offset along medially implied
  normal for the ith boundary vertex
• Neighbors are adjacent vertices
• Probability via Markov random fields
• p(zki P(zki), N(zki), C(zki) )
• Many trainable probabilities
• Requires parametrized probabilities for training

37
Multiscale Geometry and Probability for a Figure
coarse, global

• Geometrically ? smaller scale
• Interpolate (1st order) finer spacing of atoms
• Residual atom change, i.e., local
• Probability
• At any scale, relates figurally homologous points
• Markov random field relating medial atom with
• its immediate neighbors at that scale
• its parent atom at the next larger scale and the
  corresponding position
• its children atoms

coarse resampled
fine, local
38
Published Methods of Global Statistical Geometric
Characterization in Medicine

• Global variability
• via principal component analysis on
• features globally, e.g., boundary points or
  landmarks, or
• global features, e.g., spherical harmonic
  coefficients for boundary

• Global difference
• via linear (or other) discriminant on features
• globally or on global features
• Globally based diagnosis
• via linear (or other) discriminant on features
• globally or on global features
• Example authors BooksteinGolland Gerig
  Joshi Thompson TogaTaylor

39
Published Methods of Local Statistical Geometric
Characterization
A

• Local variability
• via principal component analysis on features
  globally or on global features, plus display of
  local properties of principal component
• Local difference
• via linear (or other) discriminant on global
  geometric primitives, plus display of local
  properties of discriminant direction
• On displacement vectors
  signed, unsigned re inside/outside
• Example authors Gerig Golland Joshi
  Taylor Thompson Toga

Displacement significance Schizophrenic vs.
control hippocampus
40
Shortcomings of Published Methods of Statistical
Geometric Characterization

• Unintuitive
• Would like terms like bent, twisted, pimpled,
  constricted, elongated, extra figure
• Frequently nonlocal or local wrt global template
• Depends on getting correspondence to template
  correct
• Need where the differences are in object
  coordinates
• Which object, which figure, where in figure,
  where on boundary surface
• Requires infeasible number of training cases
• Due to too many random variables (features)

41
Overcoming Shortcomings of Methods of Statistical
Geometric Characterization

• Intuitive
• Figural (medial) representation provides terms
  like bent, twisted, pimpled, constricted,
  elongated, extra figure
• Local
• Hierarchy by scale level provides appropriate
  level of locality
• Object figure based hierarchy yields intuitive
  locality and good positional correspondences
• Which object, which figure, where in figure,
  where on boundary surface
• Positional correspondences across training cases
  scale levels
• Trainable by feasible number of cases
• Few features in residue between scale levels
• Relative to description at next larger scale
  level
• Relative to neighbors at same scale level

42
Conclusions re Object Based Image Analysis

• Work at multiple levels of scale
• At each scale use representation appropriate for
  that scale
• At intermediate scales
• Represent medially
• Sense at (implied) boundary
• Papers at midag.cs.unc.edu/pubs/papers

43
Extensions

• Variable topology
• jump diffusion (local shape)
• level set?
• Active Appearance Models
• shape and intensity
• explaining the image
• iterative matching algorithm

44
Recommended Readings

• For deformable sampled boundary models T Cootes,
  A Hill, CJ Taylor (1994). Use of active shape
  models for locating structures in medical images.
  Image Vision Computing 12 355-366.
• For deformable parametrized boundary models
  Kelemen, Gerig, et al
• For m-rep based shape Pizer, Fritsch, et al,
  IEEE TMI, Oct. 1999
• For 3D deformable m-reps Joshi, Pizer, et al,
  IPMI 2001 (Springer LNCS 2082) Pizer, Joshi, et
  al, MICCAI 2001 (Springer LNCS 2208)

45
Recommended Readings

• For Procrustes, landmark based deformation
  (Bookstein), shape space (Kendall) especially
  understandable in Dryden Mardia, Statistical
  Shape Analysis
• For iterative conditional posterior, pixel
  primitive based shape Grenander Miller Blake
  Christensen et al