Technology Project: Shape-Based Retrieval of 3D Craniofacial Data

1
Technology Project Shape-Based Retrieval of 3D
Craniofacial Data

• PI Linda Shapiro, Ph.D.
• Key Personnel James Brinkley, M.D., Ph.D.
• Michael Cunningham, M.D., Ph.D.
• Collaborators Carrie Heike, M.D. and Tim Cox,
  Ph.D.
• Postdoc Katarzyna Wilamowska, Ph.D.
• Postdoc Indriyati Atmosukarto, Ph.D.
• RA Shulin Yang, MS
• RA Jia Wu, MS
• RA Sara Rolfe, MS
• Undergrad RA Michael Lam

2
Progress on Specific Aims

• Aim 1 Software Tools for Quantification of
  Craniofacial Anatomy
• New method for learning to compute the plane of
  symmetry for human faces (paper accepted for the
  ACM Conference on Bioinformatics, Biology, and
  Biomedicine)
• Landmark-free framework for the detection and
  description of shape differences in chicken
  embryos (paper submitted to the IEEE Conference
  on Engineering in Medicine and Biology)
• Aim 2 Similarity Measures
• Classification and interest-region localization
  on craniosynostosis skulls (paper accepted for
  the ACM Conference on Bioinformatics, Biology,
  and Biomedicine)

3

• Aim 3 Organization and Retrieval
• Subject database being set up at Seattle
  Childrens Hospital
• De-identified subject database being set up at
  University of Washington including useful
  attributes for retrieval (age, gender, race,
  reason for scan, diagnosis) and pointers to image
  data files
• Aim 4 Retrieval System
• Modules for 2D Azimuth-Elevation Histogram, Local
  Features, 2D Longitude-Latitude Signature Map,
  Pose Normalization, and Automatic Cranial Image
  Generation delivered to the HUB.
• Reference manual has been delivered to the HUB.
• Graphical user interface that can use these
  modules is in progress.
• Retrieval system will be built (probably in year
  4) to use both the database from Aim 3 and the
  completed feature extraction and similarity
  modules.

4
Learning to Compute the Plane of Symmetry for
Human Faces

• We have started to work with 3D mesh data from
  subjects
• who have clefts.
• Faces are no longer expected to be nearly
  symmetric.
• Standard pose normalization is not guaranteed to
  work.
• Instead, we have developed a method for
  computing the
• plane of symmetry using regions about landmarks
  that are
• learned from training data.

5
Methodology

• 1. Use training data head meshes on which
  experts have marked landmarks
• Train component detection classifiers to
  recognize regions (components)
• surrounding these landmarks using the
  curvature of the mesh points
• Using the known plane of symmetry, train
  component goodness classifiers
• to determine which detected components are
  good for computing the plane
• of symmetry
• components that lie on the plane of symmetry
• component pairs that lie an equal distance from
  the plane of symmetry

6

• On independent test data
• Apply component detection classifiers to find
  components
• Apply component goodness classifiers to select
  those to be
• used for determining the plane of symmetry
• Apply the RANSAC algorithm to fit the plane of
  symmetry to the
• center points of single components and
  points halfway between
• centers of pairs of components, while
  throwing out outliers.

Single Components Pairs
Good Computed

Components Symmetry

Plane
7
Landmark-Free Framework for the Detection and
Description of Shape Differences in Embryos
GOALS

• Identify surface in problematic optical
  projection tomography (OPT) images.
• Describe changes in shape during embryo
  development without the use of landmarks.
• Differentiate normal shape changes from those due
  to cleft lip/palate defect.

Problematic Tomographic Data
Reconstructed 3D contour
8
Overview of Methodology

• Flow vectors represent change in shape.
• The features extracted from the flow vectors are
• vector magnitude
• difference from surface normal
• difference from reference vector
• local similarity measure
• local entropy

9
Feature Cluster Examples
Brain
Brain
Eye
Eye
Midface
Midface
Embryo1 Embryo 2
Flow Vectors
flow vector flow
vector distance flow vector distance
magnitude clusters from reference
clusters from normal clusters
(red high) (groups of similar
angles) (red similar to normal)
10
Classification and Interest Region Localization
for Craniosynostosis Skulls

• In prior work, we developed the Cranial Image
  (CI)
• representation of skull shape, a matrix of
  point distances.
• Under FaceBase support, we developed an
  automatic
• procedure for computing the CI, providing a
  general tool
• for analysis of craniofacial shape.
• Our tool allows users to select
• how many planes on which to detect points
• where these planes should be located
• how many points per plane
• With 10 planes and 100 points per plane, the CI
  is
• too big for many classification/description
  tasks.

11
Coronal Metopic
Sagittal

• We used several forms of machine learning to
  both
• classify and quantify head shape and to
• reduce the number of point pairs required.
• logistic regression
• L1-regularized logistic regression
• fused lasso
• clustering lasso
• These machine-learning methods
• identify the most useful point pairs for
  classification
• provide a probability value for each
  classification
• that can be used for quantification.

12

• Misclassification rates are shown for each
  method.
• Our new clustering lasso method is best overall.
• The method also allows us to determine the most
  useful
• point pairs for classification of each class
  vs. the other
• two.

13
Pairs of Points Useful for Classification
14
The Graphical User Interface in Progress