Overview of Medical

1
Overview of Medical
Image Registration
J. Michael Fitzpatrick, Department of Electrical
Engineering and Computer ScienceVanderbilt
University, Nashville, TN CS/EECE 359, Fall,
2007
2
Acknowledgements

• Benoit M. Dawant, PhD, EECS
• Robert L. Galloway, PhD, BME
• William C. Chapman, MD, Surgery
• Jeannette L. Herring, PhD, EECS
• Jim Stefansic, PhD, Psychology
• Diane M. Muratore, MS, BME
• David M. Cash, MS, BME
• Steve Hartman, MS, BME
• W. Andrew Bass, BME

NSF NIH
Matthew Wang, PhD, IBM Jay B. West, PhD, Accuray,
Inc. Derek L. G. Hill, PhD Kings CollegeCalvin
R. Maurer, Jr., PhD, Stanford U.
3
Computed Tomography (1972)
Siemens CT Scanner (Somatom AR)
4
3D Cross-sectional Image
voxels (volume elements)
5
Magnetic Resonance Imaging
GE MR Scanner (Signa 1.5T)
6
Positron Emission Tomography
GE PET Scanner
7
Physician has 3 or more views.
MR (wet tissue)
PET (biologicalactivity)
CT (bone)
8
Combining multiple images requires image
registration
9
Image Registration Definition
Determination of corresponding points in two
different views of the same object
10
Motion relative to the scanners can be
three-dimensional.
11
Slice orientations vary widely.
transverse
sagittal
coronal
12
Views may be very different.
13
But all orientations and all views can be
combined if we have the 3D point mapping.
14
Combining Registered Images Image Fusion
15
Rigid Registration Definition

• Rigid Registration Registration using a rigid
  transformation

16
Rigid Transformation
Distances between all points remain constant.
Rigid
Non-rigid
17
Nonrigid Transformationscan be very complex!
Thompson, 1996
18
Registration Methods

• Retrospective methods
• Match anatomical features e.g., surfaces
• Maximize similarity of intensity patterns
• Prospective methods
• Non-invasive Match skin markers
• Invasive Match bone-implanted markers

19
Mutual Information An Example of
Intensity-Based Registration
Also known as Voxel-Based Registration
20
2D Intensity Histogram (Hill94)
21
Misregistration Blurs It
5 cm
0 cm
2 cm
MR
CT
MR
PET
Hill, 1994
22
Mutual Information(Viola, Collignon, 1996)

• A measure of histogram sharpness
• Most popular intensity method
• Assumes a search method is available
• Stochastic, multiresolution search common
• Requires a good starting pose
• May not find global optimum

23
Example Mutual Information
Studholme, Hill, Hawkes, 1996, Automated 3D
registration of MR and CT images of the head,
MIA, 1996
24
Goal for 1st Day
25
The Iterative Closest-Point AlgorithmAn
Example of Surface-Based Registration
26
Iterative Closest-Point Method(Besl and McKay,
1992)

• Minimizes a positive distance function
• Most popular surface method
• Assumes surfaces have been delineated
• Guaranteed to converge
• Requires a good starting pose
• May not find global optimum

27
Start with two surfaces
28
Reorient one (somehow)
29
Reorient one (somehow)
30
Reorient one (somehow)
31
Pick points on moving surface
32
Pick points on moving surface
33
Remove moving surface
34
Points become proxy for surface
35
Find closest points on stationary surface
36
Got here 10/4/2005
37
Measure the total distance
38
Remove stationary surface
39
Points become proxy for surface
40
Register point sets (rigid)
41
Register point sets (rigid)
42
Restore stationary surface
43
Find (new) closest points
44
Find (new) closest points
45
Remove stationary surface
46
Remove stationary surface
47
Register Points
48
Register Points, and so on
49
Iterative Closest-Point Algorithm

• Find closest points
• Measure total distance
• Register points

50
Example ICP for Head
Dawant et al.
51
Example ICP for Vertebra
Muratore, Herring, Dawant, Galloway
52
Got to here in CS 395, 8/28/03
53
ICP requires surface delineation, which is a
problem in Image Segmentation
Example Level Set Segmen-tation (Dawant et al.)
http//www.vuse.vanderbilt.edu/dawant/levelset_ex
amples/
54
(1) Point-Based Registration (2) Prospective
Methods(3) Image-guided Therapy
The Fiducial Marker An Example of
55
Image-Guided Surgery
Just another image registration problem.

• ...and the other is the patient.

56
AcustarAllen, Maciunas, Fitzpatrick, and
Galloway1986-1996 (JJ ? Z-Kat)
Posts
are implanted into the skull.
Maurer, et al., TMI, 1997
57
AcustarAllen, Maciunas, Fitzpatrick, and
Galloway1986-1996 (JJ ? Z-Kat)
Liquid in marker shows up in image
Maurer, et al., TMI, 1997
58
AcustarAllen, Maciunas, Fitzpatrick, and
Galloway1986-1996 (JJ ? Z-Kat)
Marker center and cap center occupy the same
position relative to the post
Maurer, et al., TMI, 1997
59
AcustarAllen, Maciunas, Fitzpatrick, and
Galloway1986-1996 (JJ ? Z-Kat)
Marker center and cap center occupy the same
position relative to the post
Maurer, et al., TMI, 1997
60
Point-based, Rigid Registration
61
What to Optimize?

• Mean-square Fiducial Registration Error (FRE2)
• Known as the Orthogonal Procrustes Problem in
  statistics since 1950s.
• Robust estimators (median, M-estimators)
• Less sensitive to outliers

Color key Major problems solved, Negligible
work
62
Minimization of FRE2(Shönemann, Farrell, 1966)

• Minimizes a positive distance function
• Most popular point method
• Assumes points have been localized
• Guaranteed to converge
• Does not require a good starting pose
• Always finds global optimum

63
Sum of Squares Step 1
Center the points
64
Step 2 (Shönemann, Farrell, 1966)
Centered
Determine the Rotation
Centered and Rotated
65
Finding Points Localization
66
Competition Acustar v. Leibinger
67
Competition Acustar v. Leibinger
68
Accuracy
69
Measures of Registration Error
View 2
View 1
Registered Views
70
Acustar, 3mm Slice CT-physical TRE 0.3 to
0.7 mm
71
Registration of Head Images The State of the Art
Retrospective Median Maximum
(Acustar) Best CT-MR 0.6 mm 3.0 mm
(0.5 mm) Poor CT-MR 5.4 mm 61 mm
(0.5 mm) Best PET-MR 2.5 mm 6.0
mm (1.7 mm) Poor PET-MR 5.3 mm
15 mm (1.7 mm)
And how do we know?
72
Retrospective Image Regstration Evaluation
1995-2007
External site
Access 150 participants in 20
countries Evaluation 57 participants in 17
countries
Vanderbilt
73
Goal for 2nd Day
74
Error Theory for Minimization ofMean-square FRE
75
Start with Assumptions about FLE
Independent, normal, isotropic, zero mean
76
Effective FLE
Space 2
Space 1
77
FRE Statistics Sibson 79
Approximate Solution
78
FRE Statistics Sibson 1979
Approximate Solution
79
TRE statistics, 1998
Approximate Solution
Fitzpatrick, West, Maurer, TMI, 98
Configuration doesmatter.
Principal axes
80
TRE forFLEof1mm
Marker Placement
West et al., Neurosurgery, April, 2001
FRE 1mm
81
A distribution would be better
ltTRE2gt
Probability density
TRE2
82
And what about direction?
83
TRE statistics, 2001
Approximate Solution
TRE1
TRE3
TRE2
West and Fitzpatrick., TMI, Sep 2001
84
Some Remaining Problems
85
Isotropic Scaling
Actually now solved Batchelor, West,
Fitzpatrick, Proc. of Med. Im. Undstnd. Anal.
,Jul 2002
86
Anisotropic Scaling
(Iterative Solution Only)
87
Register M points sets simultaneously
View 1
View 2

The Generalized Procrustes Problem
View 3
View M
(Iterative Solution Only)
88
Spatial Weighting
(Iterative Solution Only)
89
Other Unsolved Problems

• What is the statistical effect on TRE of dropping
  or adding a fiducial?
• Does anisotropy in FLE always, sometimes, or
  never makes TRE worse?
• How do we configure markers on a given surface so
  as to minimize TRE over a given region?
• Is there a correlation between FRE and TRE?

90
Other Unsolved Problems (cont.)

• Extenion to perspective transformations.
• Extension to surface matching.

91
End of TalkAdditional slides follow
92
Categories within error prediction

• Number of point sets Two or more
• Scaling Isotropic or anisotropic
• Point-wise weighting equal or unequal
• Anisotropic weighting
• Cost function squared error or other
• Point-wise FLE equal or unequal
• Spatial FLE isotropic or anisotropic...

Key Approximate, Negligible progress
93
Anisotropic Scaling
Iterative Algorithm
Problem Statement
94
Scaling Anisotropic II
Iterative Algorithm
Problem Statement
R, t rotation, translationwi2 point
weighting S diag( sx , sy , sz )
Given xi yi wi find R, t, S to minimize
mean FRE2
95
Spatial Weighting
Iterative Algorithm
Problem Statement
Partial Solution
96
Generalized Procrustes Problem
Cost function
Iterative method(only)
97
Add Isotropic Scaling
Approximate Solution
FRE2 sum of squared fiducial registration errors
98
FRE Generalized Scaling
Approximate Solution
FRE2 sum of squared fiducial registration errors
99
TRE statistics with scaling
Approximate Solution
TRE2 target registration error
100
Applications of TREStatistics
101
Error Bounds
102
Probe Design
103
Fiducial-Specific FRE
Poor fiducial alignment tends to occur where
target registration is good!!
104
Four Solution Methods
( All work equally well Eggert91! )
105
Generalized Procrustes Problem
(Weve already done it for M2.)
Problem Statement
Illustration
106
Generalized Procrustes Problem
Iterative Algorithm
Illustration
Subject to S(m) normalization
107
Approximation Method
(due to Sibson, 1979)
108
Approximation Method (cont.)
109
FRE Statistics
Approximate Solution
Problem Statement
110
TRE statistics with scaling
Approximate Solution
Problem Statement
111
What do solved and unsolved mean?

• Solved, working definition
• Reduced to solving algebraic equations
• Iterative algorithm that converges to solution
• Approximate solution accurate to
• Unsolved
• Not solved

112
Point-wise weighting Equal or Unequal
(Weve just looked at this one.)
Problem Statement
Solution
See previousslides again!
113
1. Performing a Registration
a.k.a. The Orthogonal Procrustes Problem
Problem Statement
114
2. Predicting Registration Error

• Input---
• fiducial positions
• target position, r
• FLE distribution

View 2
View 1
Registered Views
115
Isotropic Scaling
Problem Statement
Solution