Antonio Plaza

1
Automated Image Registration Using Morphological
Region of Interest Feature Extraction

• Antonio Plaza
• University of Extremadura. Caceres, Spain
• Jacqueline Le Moigne
• NASA Goddard Space Flight Center, USA
• Nathan Netanyahu
• Bar-Ilan University, Israel University of
  Maryland, USA

2
Earth Science Data Integration
3
What is Image Registration ?

• Navigation or Model-Based Systematic Correction
• Orbital, Attitude, Platform/Sensor Geometric
  Relationship, Sensor Characteristics, Earth
  Model, ...
• Image Registration or Feature-Based Precision
  Correction
• Navigation within a Few Pixels Accuracy
• Image Registration Using Selected Features (or
  Control Points) to Refine Geo-Location Accuracy
• 2 Approaches
• (1) Image Registration as a Post-Processing
  (Taken here)
• (2) Navigation and Image Registration in a Closed
  Loop

4
Image Registration Challenges

• Multi-Resolution / Mono- or Multi-Instrument
• Multi-temporal data
• Various spatial resolutions
• Various spectral resolutions
• Sub-Pixel Accuracy
• 1 pixel misregistrationgt 50 error in NDVI
  computation
• Accuracy Assessment
• Synthetic data
• "Ground Truth" (manual registration?)
• Use down-sampled high-resolution data
• Consistency ("circular" registrations) studies

5
Image to Image Registration
Multi-Temporal Image Correlation  Landmarkin
g  Coregistration
Image Characteristics (Features) Extraction
Feature Matching
Incoming Data
Compute Transform
6
Image to Map Registration
Input Data
Masking and Feature Extraction
Feature Matching
Map
7
Multi-Sensor Image Registration
ETM/IKONOS Mosaic of Coastal VA Data
ETM
IKONOS
8
Image Registration Components

• Pre-Processing
• Cloud Detection, Region of Interest Masking, ...
• Feature Extraction (Control Points)
• Edges, Regions, Contours, Wavelet Coefficients,
  ...
• Feature Matching
• Spatial Transformation (a-priori knowledge)
• Search Strategy (Global vs Local,
  Multi-Resolution, ...)
• Choice of Similarity Metrics (Correlation,
  Optimization Method, Hausdorff Distance, ...)
• Resampling, Indexing or Fusion

9
Image Registration Subsystem Based on a Chip
Database
UTM of 4 Scene Corners Known from Systematic
Correction
Landmark Chip Database
(1) Find Chips that Correspond to the
Incoming Scene (2) For Each Chip, Extract
Window from Scene, Using UTM of - 4
Approx Scene Corners - 4 Correct Chip
Corners (3) Register Each (Chip,Window)
Pair and Record Pairs of Registered Chip
Corners (4) Compute Global Registration from
Multiple Local Ones (5) Compute Correct UTM
of 4 Scene Corners of Input Scene
Correct UTM of 4 Chip Corners
10
Image Registration Subsystem Based on Automatic
Chip Extraction
UTM of 4 Scene Corners Known from Systematic
Correction
Reference Scene

• (1) Extract Reference Chips
• and Corresponding Input
• Windows Using Mathematical
• Morphology
• (2) Register Each (Chip,Window)
• Pair and Record Pairs of
• Registered Chip Corners
• (refinement step)
• (3) Compute Global Registration
• from Multiple Local Ones
• (4) Compute Correct UTM
• of 4 Scene Corners of
• Input Scene

• Advantages
• Eliminates Need for Chip Database
• Cloud Detection Can Easily be Included in
  Process
• Process Any Size Images
• Initial Registration Closer to Final
  Registration gt
• Reduces Computation Time and Increases Accuracy.

11
Step 1 Chip-Window Extraction UsingMathematical
Morphology

• Mathematical Morphology (MM) Concept
• Nonlinear spatial-based technique that provides
  a framework.
• Relies on a partial ordering relation between
  image pixels.
• In greyscale imagery, such relation is given by
  the digital value of image pixels

Original image
Greyscale MM Basic Operations
K
K
Structuring element
(4-pixel radius Disk SE)
Dilation
Erosion
12
Step 1 (Cont.)
Binary Erosion
Structuring element
Structuring element
Structuring element
13
Step 1 (Cont.)
Binary Dilation
Structuring element
Structuring element
Structuring element
14
Step 1 (Cont.)
Greyscale Morphology Combined Operations e.g.,
Erosion Dilation Opening
K
15
Step 1 Chip-Window Extraction UsingMathematical
Morphology

• Scale-Orientation Morphological Profiles (SOMP)
  From Openings and Closings with SEsLine Segments
  of Different Orientations
• SOMP Feature Vector D(x,y) at each Pixel
  (various scales orientations)
• Entropy of D(x,y) H(D(x,y))
• Algorithm
• a. Compute D(x,y) for each (x,y) in reference
  scene
• b. Extract reference chip centered around
  (x,y) with MaxH(D(x,y)), e.g. 256x256
• c. Compute D(X,Y) for each (X,Y) in search area
  input scene centered (e.g., 1000x1000) around
  location (x,y)
• Compute RMSE(D(X,Y),D(x,x)) for all (X,Y) in
  search area
• Extract input window centered around (X,Y) with
  Min(RMSE)
• Return to step 2. until predefined number of
  chips is extracted

16
Step 1 Chip-Window Extraction UsingMathematical
MorphologyResults(Landsat-7/ETM Data - Central
VA)
10 Chips Extracted from Reference Scene (Oct. 7,
1999)
10 Windows Extracted from Input Scene (Nov. 8,
1999)
17
Step 2 Chip-Window RefinedRegistration Using
Robust Feature Matching

• Overcomplete Wavelet-type Decomposition
  Simoncelli Steerable Pyramid
• Maxima Extraction Top 5 of Histogram

18
Step 2 Robust Feature Matching Using Hausdorff
Distance

• Search Transformation Space through Hierarchical
  Spatial Subdivisions
• Perform Monte Carlo Sampling of Control Points
• Compute Robust Similarity Measure
• k-th smallest squared distance to nearest
  neighbors, i.e., partial Hausdorff
  DistancePartial Hausdorff Distance
• Hk(A, B) Kth a in A minb in B dist (a,b)
• (1 k A Kth is the kth smallest element
  of set dist(a,b) Euclidean distance)
• Prune Search Space by "Range" Similarity
  Estimates
• Iterate and Refine on each Level of Wavelet
  Decomposition

19
Step 3 Compute Global Registrationfrom All
Local Registrations

• From each Local Registration, Window-Chip
• Corrected Locations of Four corners of Each
  Window
• i.e. for each chip-window i, pair
  correspondences
• (UL_i_X1,UL_i_Y1) to (UL_i_X2,UL_i_Y2)
• (UR_i_X1,UR_i_Y1) to (UR_i_X2,UR_i_Y2)
• (LL_i_X1,LL_i_Y1) to (LL_i_X2,LL_i_Y2)
• (LR_i_X1,LR_i_Y1) to (LR_i_X2,LR_i_Y2)
• Use of a Least Mean Square (LMS) Procedure to
  Compute Global Image Transformation (in pixels)
• If n chips, 4n points used for the LMS
• gt Step 4 Use Global Transformation to Compute
  new UTM Coordinates for each of the 4 Corners of
  the Incoming Scene

20
Results of Global RegistrationOn Landsat-7 VA
Test Data
21
Conclusions

• Fully Automated System for Registration of
  Multi-Temporal Landsat Scenes of Any Size, Using
  Mathematical Morphology and Robust Feature
  Matching Techniques
• MM Chip-Window Extractor Can be Used with Any
  Other Registration Method
• Eliminates Need of Database
• Provides Close Initial Match gt Follow-up
  Computations Faster and More Accurate
• Further Experimentation On-Going