Medical Image Registration

1
Medical Image Registration

• Yujun Guo
• Dept.of CS
• Kent State University

2
Outline

• Why registration
• Registration basics
• Rigid registration
• Non-rigid registration
• Applications

3
Modalities in Medical Image

• Computed Tomography (CT), Magnetic Resonance
  (MR) imaging, Ultrasound, and X-ray give anatomic
  information.
• Positron Emission Tomography (PET) and Single
  Photon Emission CT (SPECT) give functional
  information.

4
Registration

• Monomodality
• A series of same modality images (CT/CT, MR/MR,
  Mammogram pairs,).
• Images may be acquired weeks or months apart
  taken from different viewpoints.
• Aligning images in order to detect subtle changes
  in intensity or shape
• Multimodality
• Complementary anatomic and functional information
  from multiple modalities can be obtained for the
  precise diagnosis and treatment.
• ExamplesPET and SPECT (low resolution,
  functional information) need MR or CT (high
  resolution, anatomical information) to get
  structure information.

5
Registration Problem Definition
6
Example Mapping Function
q (912,632)
p (825,856)
Pixel scaling and translation
7
Image Registration

• Define a transform T that will map one image onto
  another image of the same object such that some
  image quality criterion is maximized.
• A mapping between two images both spatially and
  with respect to intensity
• I2 g (T(I1))

8
Registration Scheme
9
Components

• Feature Space
• Search Space or transformation
• Similarity Metric
• Search Strategy

10
Feature Space

• Geometric landmarks
• Points
• Edges
• Contours
• Surfaces, etc.
• Intensities
• Raw pixel values

1. 35
2. 56

Feature-based Intensity-based
11
Image transformations
Rigid Non-rigid
12
Similarity Metric

• Absolute difference
• SSD (Sum of Squared Difference)
• Correlation Coefficient
• Mutual Information / Normalized Mutual Information

13
Search Strategy

• Powells direction set method
• Downhill simplex method
• Dynamic programming
• Relaxation matching
• Hierarchical techniques

14
Multi-modality Brain image registration

• Intensity-based
• 3D/3D Rigid transformation, DOF6 (3
  translations, 3 rotations)
• Maximization of Normalized Mutual Information
• Simplex Downhill
• Multi-resolution
• Dataset Vanderbilt University
• http//www.vuse.vanderbilt.edu/image/registration
  /results.html

15
Mutual Information as Similarity Measure

• Mutual information is applied to measure the
  statistic dependence between the image
  intensities of corresponding voxels in both
  images, which is assumed to be maximal if the
  images are geometrically aligned.

16
Normalized Mutual Information

• Extension of Mutual Information
• Maes et. al.
• Studholme et. Al.
• Compensate for the sensitivity of MI to changes
  in image overlap

17
Geometry Transformation

• Image Coordinate transform
• The features (dimension, voxel size, slice
  spacing, gantry tilt, orientation) of images,
  which are acquired from different modalities, are
  not the same.
• From voxel units (column, row, slice spacing) to
  millimeter units with its origin in the center of
  the image volume.

18
Target Image Template Image
19
Images from the same patient
Target Image ? Template Image ?
Images provided as part of the project
Retrospective Image Registration Evaluation,
NIH, Project No. 8R01EB002124-03, Principal
Investigator, J. Michael Fitzpatrick, Vanderbilt
University, Nashville, TN.
20
Interpolation

• Nearest Neighbor
• Tri-linear Interpolation
• Partial-Volume Interpolation
• Higher order partial-volume interpolation

21
Evaluating similarity measure for each
transformation
y
y
Transform
x
x
Template Image
Target Image
22
Optimization

• Powells Direction Set method
• Downhill Simplex method

23
Multi-resolution

• Why Multi-resolution
• Methods for detecting optimality can not
  guarantee that a global optimal value will be
  found.
• Time to evaluate the registration criterion is
  proportional to the number of voxels.
• The result at coarser level is used as the
  starting point for the finer level.
• Currently multi-resolution approaches
• Sub-sampling
• Averaging
• Wavelet

24
Registration Result (I)
A typical superposition of CT-MR images. Left
before registration Right after registration.
25
Rigid transformation (II)
A typical superposition of MR-PET images. Left
before registration Right after registration.
26
Mammography

• Breast cancer is the second leading cause of
  death among women in USA.
• Detected in its early stage, breast cancer is
  most treatable.
• Mammography is the main tool for detection and
  diagnosis of breast malignances.
• It reduces breast cancer mortality by 25 to 30
  for women in the 50 to 70 age group

27
Mammogram Registration

• Temporal/bilateral mammograms vary
• Breast compression
• Breast position
• Imaging Technique
• Change in Breast

28
Mammogram registration techniques

• Whole breast area vs. regional
• Nipple location
• Control-point location
• Rigid non-rigid registration

29
Non-rigid Mammogram Registration

• Intensity-based
• Elastic transformation
• Multi-resolution
• Demons algorithm (Thirion, 1996)

30
Demons
Transform
Scene (Target)
Model (Template)
31
Demons (Cont.)
Transform
Scene
Forces
Model
32
Demons (Cont.)
Current Estimation
Intensity
Space
Gradient
Desired Displacement
Scene
33
Demons

• From Optical Flow
• Scene f, Model g
• Assumption The intensity of a moving object is
  constant with time

(1)
(2)
34
Description of the Approach

1. Select demon points.
2. Compute the force u on the model at each of the
   selected demons
3. Determine a global transformation based on the
   computed u and apply it to the model
4. If the model images is now registered to the
   scene image, stop. Else, go to Step 2.

35
Registration Components

• Image Intensities
• Non-rigid transformation, one displacement vector
  for each pixel
• Bilinear interpolation
• Absolute difference as similarity metric
• Multi-resolution
• Dataset MIAS,DDSM

36
Demons Results (I) Synthetic Images
Level2
Level3
Level5
Level4
37
Demons Result (II) MIAS

Original images
Before registration
After rigid registration
After non-rigid registration
38
Ongoing registration topics

• Trade-off of computation and accuracy
• Evaluation of registration results
• Visualization of registration

39
Applications Change Detection

• Images taken at different times
• Following registration, the differences between
  the images may be indicative of change
• Deciding if the change is really there may be
  quite difficult

40
Other Applications

• Multi-subject registration to develop organ
  variation atlases.
• Used as the basis for detecting abnormal
  variations
• Object recognition - alignment of object model
  instance and image of unknown object
  (segmentation)

41
References

• Maes F,Collignon A, et al. Multimodality image
  registration by maximization of mutual
  information. IEEE Trans. Med. Imaging. 1997,
  V16,pp187-198
• L.G.Brown, A survey of image registration
  techniques, ACM Computing Surveys, vol. 24, no.
  4, pp. 325376, 1992.
• Jean-Philippe Thirion, Non-Rigid Matching Using
  Demons, IEEE Conference on Computer Vision and
  Pattern Recognition,1996