A Survey of State-of-Arts Retina Image Registration Methods

1
A Survey of State-of-Arts Retina Image
Registration Methods

• Presented By Jian Shi
• University of South Carolina

2
Overview

• Motivation
• Methods
• Conclusion
• Future work

3
Motivation

• Motivation
• Retina image processing is greatly required in
  diagnosing and treatment of many diseases
  affecting retina.
• The registration of retina images is very useful
  in helping physicians to do a reliable diagnosis
  as composing a complete retina map
• Many research paper talk about various retina
  registration schemes using different techniques
  and algorithms. It is still under rapid
  development, new requirements and new methods
  continue
• My intention is to do a survey research of these
  methods, trying to categorize them, making
  comparison between them, then see whatd be
  useful and valuable for future study and research

4
Motivation

• The basic several steps of image registration
• Feature detection
• Feature matching
• Transform modal estimation
• Image resampling and transformation
• Feature detection and matching
• The key components of retina image registration
• Detecting landmarks based on blood vessels and
  cross points consists the majority group of
  methods
• New schemes that are going beyond this boundary
• I am trying to introduce them in a developing
  point of view

5
Segmentation before Registration

• Popular methods
• Detection of blood Vessel boundaries by the
  difference operators
• Sobel operators, smoothing effect. See Rafael C.
  Gonzalez, Richard E. Woods,Digital image
  processing, second edition,Prentice Hall, pp
  567-585, 2002
• Caney edge detector, good at detecting blood
  vessels boundaries. See Bill Green, Canny Edge
  Detection Tutorial, http//www.pages.drexel.edu/
  weg22/can_tut.html, 2002.
• Boundary detection using image statistics
• See E. Aniram, H. Aydinoglu, and I. C. Goknar,
  Decision Based Directional Edge Detector,
  Signal Proc., vol. 35, pp. 149-156, 1993.

6
Segmentation before Registration

• More
• Extraction of blood vessel boundaries using
  deformable models (snake model)
• See T. McInerney and D. Terzopoulos, Deformable
  Models in Medical Image Analysis a survey, Med
  Image Analysis, vo. 1, no. 2, pp. 91-108, 1996.
• Boundary Detection Using Morphological Gradient
• Extraction of the Core Area of the Blood Vessel
  Tree by Matching the Image with Gaussian Filter
• Segmentation Using Watersheds
• Extraction of Blood Vessel Tree Using the
  Morphological Reconstruction
• Etc

7
Vessels are reliable landmarks in retinal images
because they are almost rigid structures and they
are depicted in all modalities. In the following
part I will introduce 2 representative
registration method using vessel structures as
their features. I dont talk about the detail of
feature extraction because I assume it is well
resolved
8
A Multimodal Registration

• Main idea
• Assumptions
• input retina images are restricted to central
  images of the retina containing the macula, the
  papilla, and temporal vessels in order to limit
  deformations.
• Detecting vessel structures and extracting
  bifurcation points as features
• Vessels has to be connected
• Have a fixed Width less than a threshold and are
  locally linear
• A global affine transform then performed
• Given M is a bifurcation point that to be
  transformed into M using

9
A Multimodal Registration

• Invariant properties
• Angles between edges of bifurcation points are
  preserved
• The point M can be defined as M(m, a1,a2,a3,a4),
  a14 are the possible 4 directions of a
  bifurcation point
• A matching example a counter
  example

10
A Multimodal Registration

• Matching algorithm
• A Bayesian Hough transform is used
• That is to decompose affine transformation into a
  translation S, a rotation R, a homothetie T
• Each pair of feature points could generate a set
  of transformation matrix, calculate among 20 sets
  of matrix to satisfy a min-square estimation
• Drawbacks
• Too many constraints and assumptions on inputs
• Heuristic threshold settings
• Cant deal with many types of retina images
• Advantages
• It well defined the invariant features using
  bifurcation points
• It has refinement process

11
Using Creases as Landmark

• Main idea
• Treat vessels as creases (ridges or valleys) when
  images are seen as landscapes.
• Extract the invariant property we define creases
  using level set extrinsic curvature (LSEC, see A.
  Lopez, D. Lloret, J. Serrat, and J. Villanueva.
  Multilocal creasness based on the level set
  extrinsic curvature. Computer Vision and Image
  Understanding, (77), 2000.). The level set of a
  constant level curvature L consists of a set of
  points X that L(X) L
• Matching scheme
• Start from an initial guess, transform creaseness
  image g to f until its properly aligned, use the
  correlation function to check the quality of
  alignment

12
Using Creases as Landmark

• When Ct reach a maximum value, it is the best
  alignment
• Here f and g are not the whole image, theyre
  creaseness image that contains pixels with
  creaseness values higher than a threshold
• Advantages
• Fast processing speed, since the pixels involved
  in computation are only a part of the whole image
• High accuracy, since the quality function is
  iteratively checked and could reach a global
  maximum
• Drawbacks
• Need a good initial guess
• Cant deal with images without clear vessel
  structure

Retinal image registration using creases as
anatomical landmarks David Lloret, Joan Serrat,
Antonio M. Lopez, Andres Soler, Juan J.
Villanueva
13
The previous two methods are representative
because they can be categorized into two
different groups The former one uses geometric
transformation to do the registration. And the
latter one uses similarity detection by
correlation to register retina images. However,
their performance are based on the quality of
input retina images, yet in reality, there exist
many unclear or ill-formed retina images. In
those cases, the pervious two methods are not
enough. Next Ill introduce a novel scheme called
dual-bootstrap published In 2002, for better
understanding, we talk about ICP first on which
dual-bootstrap is based
14
Point-based Registration

• ICP (Iterative Closest Point)
• Explanation
• Point here means raw measurements that locally
  summarize the geometric structure of the data.
  Such as (x, y, z) values from range images,
  intensity points in three-dimensional medical
  images, and edge elements, corners and interest
  points.
• Main steps
• Given two dataset i, j and T (the parameter
  vector of the transformation mapping the
  coordinate system of i onto the coordinate system
  of j). Repeat
• using a fixed estimate, T, the transformation is
  applied to each point from image dataset i and
  the closest point in image dataset j is found as
  a temporary match
• using constraints formed from these matches, a
  new best T is computed
• Until T stabilizes, that is, algorithm converges

15
Point-based Registration

• ICPs Problems

• converge to an incorrect final registration
  starting from an initial estimate that locally
  appears correct.
• Same image viewed from different perspective
• (a), (b) are used in initializing ICP (c), (d)
  are used in constraining ICP in iterations.

16
Point-based Registration

• An example of ICPs failure

• (a) shows the initial alignment based on the
  single correct correspondence
• (b) shows the final result after convergence

17
Dual Bootstrap ICP

• Dual bootstrap ICP
• Three enhancement to original ICP
• The bootstrap region, its a small region at the
  beginning and gradually increase to the entire
  image
• Robust ICP, with a carefully estimated error
  scale
• Bootstrapping the model, different models for
  different regions
• Main steps
• Start with a initial bootstrap region
• Repeat applying robust ICP only in the bootstrap
  region, when the robust ICP converges, increase
  the bootstrap region size and do it again
• Until the bootstrap region covers the whole image
• The transformation model is automatically
  selected during the size of bootstrap region
  increasing

18
Dual Bootstrap ICP

• Robust ICP
• Objective function to be minimized
• ? is the transformation parameter vector M is
  the mapping from p1 in I1 into I2, q is the
  corresponding point in I2, D is the distance
  metric between p and q, ?is a loss function,sis
  the error scale factor, they both are for help
  rejecting mismatches

Mathematical details about the equation above
could be found in the original paper The
Dual-Bootstrap Iterative Closest Point Algorithm
with Application to Retinal Image Registration
Charles V. Stewart Chia-Ling Tsai Badrinath
Roysam
19
Dual Bootstrap ICP

• Automatic model selection

20
Dual Bootstrap ICP

• Bootstrap region increment
• A configurable parameter b
• Move out the perpendicular of each side by b, the
  default value is sqrt(2) 1. to make sure in
  each iteration, the region doesnt expand to its
  double size
• Different value of b will affect the performance
  of the whole algorithm
• Invariant properties
• Angles between edges of bifurcation points, very
  similar to the first introduced method

21
Dual Bootstrap ICP
22
Dual Bootstrap ICP

• Advantages
• Very high accuracy, 97 success rate both for
  healthy and pathologic retina. Data gathered from
  over 6,000 images
• Robust to retina image noise and pathologic
  retina images, which might not have very clear
  vessel structures could be processed well
• Average 5 second per image pair
• Drawbacks
• Initial regions still need human interaction
• Processing speed various largely by the area of
  initial regions size
• Possible failures still exist

23
Dual-bootstrap is a proven very accurate and
sophisticated scheme, but it still limited by
image quality, although greatly improved
comparing to previous schemes. SURF is a
feature detection and description method that
proposed in 2006, it totally puts aside the
vessel structures, crossing points. So the
quality of input image is not going be affect the
registration results
24
SURF

• Surf detectors
• Based on a Hessian matrix
• sis a value that changes by different filter
  size, 1.2 when in a 9x9 filter, 3.6 when in a
  27x27 filter. The determinant of Hessian matrix
  of the interesting point would remain same while
  filter size increasing.
• Scale invariance use a 3x3 neighbor space
  interpolation, and then the maxima of the Hessian
  matrix are interpolated in scale using the method
  in Brown, M., Lowe, D. Invariant features from
  interest point groups. In BMVC.(2002)

25
SURF

• Rotation invariance
• firstly, calculate the Haar wavelet response in
  x, y direction around a circular neighbor area
  around the interesting point.
• The dominant orientation is estimated by
  calculating the sum of all responses within a
  sliding orientation window covering an angle of
  p/3 .
• The horizontal and vertical responses within the
  window are summed. The two summed responses then
  yield a new vector. The longest such vector lends
  its orientation to the interest point. The
  windows size is a experimental parameter

26
SURF

• Advantages
• Surfs blob-like invariant features can totally
  overcome the mismatch caused by non-clear vessel
  structures, it totally doesnt care about
  vessels, bifurcation points, etc.
• Surf uses integral image techniques and achieves
  very good performance
• It needs no human interaction, so it can be
  applied in automatic retina image registration
• Drawbacks
• It still fails if the image pairs have poor
  overlap
• in experiment, it fails for image pairs that have
  lower than 14 overlapping part

27
For better efficiency and higher accuracy,
automatic retina image registration is in the
need. Schemes using SURF method can achieve this
goal. Before it, therere other automatic retina
image registration schemes
28
Auto by Global Optimization Method

• Preprocessing
• Vessel structures are extracted
• Input image type can only be Red-Free (RF),
  Fluoroscein Angiography (FA) and Indocyanine
  Green Chorioangiography (ICG). That is, they all
  are easy to be transferred into binary format
• Choosing the best transformation
• T are 3 kinds of transformations affine,
  bi-linear and projective transformation

29
Auto by Global Optimization Method

• Determining the transformation parameters
• Use three different optimization methods to
  calculate different weight values for different
  methods, respectively

30
Auto by Global Optimization Method

• Drawbacks
• Need preprocessing
• Low effect using global optimization search
• Need input to be well formed
• Advantages
• It is in 1999
• The result is more accurate than manual
  registration

31
Auto by Spatial Referencing

• Pre-conditions
• An referencing retina image spatial map is
  pre-calculated
• Transformation model is fixed similarity
  transformation
• Constellation landmark pairs (constraints pairs
  used as constellation cant have a spatial
  distance gt 20 or image size )

32
Auto by Spatial Referencing

• Matching
• Invariant vectors are computed from the
  orientation and position of constellations,
  results are stored in database
• A candidate image firstly calculate a vector,
  then try to match the corresponding part in the
  spatial map stored

33
Auto by Spatial Referencing

• Drawbacks
• Still need preprocessing
• Still need input to be well formed
• Rejection rate might be very high (no
  experimental result yet)
• Advantages
• Very high processing speed, ideally 30 images per
  second.
• Increased accuracy

34
Conclusion

• An survey research of seven representative retina
  image registration technologies is presented
• Brief introduction and cons and pros for these
  methods are provided
• Vessel structure based methods have a unavoidable
  limitation that the quality of input retina
  images will affect the registration results.
• New methods that can go beyond this, like SURF,
  would be more promising

35
Future work

• More representative schemes will be added
• Registration using Gabor filter
• Each image has its own orientation, for example,
  an image with a lot of building has much more
  horizontal frequency than vertical ones. Garbor
  filter could be used to help estimating
  directional blob or direction edge components
• This is not used in retina image registration yet
  (or I havent read about it). It is also not
  depend vessel structures, like SURF.
• If possible, implement some of those schemes

36
(No Transcript)