Landmark and Intensitybased, Consistent Thinplate Spline Image Registration

1
Landmark and Intensity-based, Consistent
Thin-plate Spline Image Registration

• Hans J. Johnson
• Gary E. Christensen
• Electrical and Computer Engineering,
• The University of Iowa

2
Hierarchy of Registration Methods

• Landmark
• Manual identification, low-dimensional
• Contour
• Manual/semi-automatic, correspondence ambiguity
• Surface
• Semi-automatic/automatic, correspondence
  ambiguity
• Volumetric intensity based
• Automatic, correspondence ambiguity

? Higher Dimensionality
3
Introduction

• Combine landmark and intensity-based methods
• Best of both methods
• Define correspondence at identifiable landmarks
• Define correspondence away from landmarks based
  on intensity

4
Limitations of Existing Methods

• Landmark-based (Thin-plate spline TPS)
• Correspondence only at the landmarks

T(y)
S(x)
T(h(x))
?
h(x)
5
Introduction

• Consistent image registration
• The transformation from image A?B has the same
  correspondence relationship as the transformation
  from image B?A
• Constrain the forward and reverse transformations
  to be inverses of one another

6
Algorithm
Analysis
7
Identify Landmarks
A Image
B Image
8
Algorithm
Landmark Identification (periodic extension)
Forward Reverse Thin-plate Spline Estimation
Consistent Intensity Landmark Estimation
Analysis
9
Thin-plate Spline Algorithm

• Align landmarks
• Minimize the bending energy
• Linear system of equations solved with singular
  value decomposition

10
Simple Example
11
(No Transcript)
12
Replicate Landmarks

• Problem
• Traditional thin-plate spline registration
  assumes infinite boundary conditions
• Periodic boundaries are needed for the consistent
  intensity landmark registration

• Solution
• A thin-plate spline interpolant with periodic
  boundaries is approximated by replicating the
  landmarks

13
Landmark Replication

• Each landmark is replicated 8 times to
  approximate periodic borders for the center image
• This defines new interactions between landmarks

14
Replicated Landmark Interactions

• Distance between landmarks define a set of
  parameters used to solve the thin-plate spline
  interpolant

15
Replicated Landmark Interactions
16
Algorithm
Landmark Identification (periodic extension)
Forward Reverse Thin-plate Spline Estimation
Consistent Intensity Landmark Estimation
Analysis
17
Consistent Image Registration

• Jointly estimate the transformations h and g such
  that h maps T to S and g maps S to T subject to
  the constraint that h g-1

18
Consistent Intensity Landmarks

• Cost minimization problem

Similarity Cost
Regularization Cost
Inverse Consistency Cost
Landmark Cost
Consistent landmarks ?0 Consistent
Intensity ?i 0
19
Consistent Intensity Landmarks

• Cost minimization problem

20
Consistent Intensity Landmarks

• Cost minimization problem

21
Consistent Intensity Landmarks

• Cost minimization problem

22
Consistent Intensity Landmarks

• Cost minimization problem

23
Consistent Intensity Landmarks

• Cost minimization problem

24
Transformation Parameterization

• u(x) and w(x) parameterized by 3D Fourier series
• Periodic boundary conditions
• Gradient descent used to estimate parameters

25
Displacement Field Initialization

• u(x) and w(x) independently estimated using
  periodic thin-plate spline algorithm
• Displacement fields initialized by

26
Results
27
2D Landmark Experiment
Forward
Reverse

• Compare thin-plate spline algorithms
• Infinite vs. periodic boundary conditions
• Unidirectional vs. consistent registration

Consistent Registration 2000 iterations, X
harmonics 50, Y harmonics 50
28
2D Landmark Experiment(Deformed Grids)
Infinite Boundary
Point Displacement
AB
BA
29
2D Landmark Experiment (Jacobian Values)
Infinite Boundary
Point Displacement
Periodic Boundary
3.1
AB
0.31
3.1
BA
0.31
30
Inverse Consistency Error
yg(x)
y
x
x
x h(y)
Inverse Consistency Error x-x where
xh(g(x))
31
Inverse Consistency Error(Cyclic Boundary
Conditions)
ABA
BAB
ABA
5.0
TPS
0.00
32
Inverse Consistency Error(Cyclic Boundary
Conditions)
ABA
BAB
ABA
5.0
TPS
0.00
33
Inverse Consistency Error(Cyclic Boundary
Conditions)
ABA
BAB
5.0
TPS
0.00
0.01
Consistent TPS
0.00
34
Global Error Measures(Measured in Pixels)
35
2D MRI Experiment

• Compare landmark and intensity-based algorithms
• Landmark only
• Consistent landmark
• Consistent intensity
• Consistent landmark and intensity

36
2D MRI Experiment(64x80 images)

• Consistent landmark
• X harmonics 32, Y harmonics 40
• Consistent intensity
• X harmonics 1..32, Y harmonics 1..40
• Harmonics increased every 200 iterations
• Consistent landmark and intensity
• X harmonics 32, Y harmonics 40

37
2D MRI Experiment Run Times(667MHz 264DP alpha
processor)

• Landmarks only 10 seconds
• Consistent landmarks 12 minutes
• 2100 iterations
• Consistent intensity 35 minutes
• 10400 iterations
• Consistent landmark and intensity 25 minutes
• 2100 iterations

38
2D MRI Experiment Errors
Consistent Landmarks and Intensity
Consistent Landmark TPS
Consistent Intensity
Landmark TPS
Intensity Difference Error
39
2D MRI Experiment Errors
Consistent Landmarks and Intensity
Consistent Landmark TPS
Consistent Intensity
Landmark TPS
Intensity Difference Error
40
Global Error Measures(Measured in Pixels)
41
Conclusions

• Unidirectional thin-plate spline can have a lot
  of inverse consistency error.
• inner to outer dots 5 pixel max. error
• Inverse consistency error is reduced by enforcing
  that the forward and reverse transformations are
  inverses of one another.

42
Conclusions

• Enforcing inverse consistency does not
  significantly affect landmark matching.
• Intensity information provides more local
  deformation than landmarks alone.
• The consistent landmark and intensity-based
  registration performed better than the
  unidirectional TPS, consistent TPS, and
  consistent intensity based methods.

43
Acknowledgements

• This work was supported by NIH grant NS35368 and
  a grant from the Whitaker Foundation.
• We would also like to thank Richard Robb of the
  Mayo Clinic for his support in providing
  AnalyzeTM.

44
(No Transcript)