Adaptive Image Registration

1
Adaptive Image Registration
Line Eikvil 1) Per Ove Husøy 1) Alessandro Ciarlo
2) 1) Norsk Regnesentral 2) ESA-ESRIN
ESA-EUSC IIM
ESRIN, October 5-7, 2005
2
Outline of presentation

• Problem and background
• Description of approach
• System overview
• Examples and results

3
Problem

• Co-registration important in many remote sensing
  applications.
• Automatic techniques exist, but there is no one
  registration technique that works equally well
  for all image types.
• More than 90 of studies in remote sensing that
  could have used automated approaches for
  registration of images do not use them.
• The lack of a more general tool for helping in
  this process may be one of the reasons for this.
• Useful to have a more general tool for image
  registration that could be used for several
  applications.

4
Objective

• Develop a co-registration tool
• for homogeneous time series of images
• which is general and can handle time series
• From different sensors
• With different contents
• Acquired under different circumstances
• By using and adaptive approach providing
• a selection of different methods
• and intelligence enabling selection of the most
  appropriate method for each problem.

5
Main idea

• Integrate existing methods and tools for
  registration.
• Develop methods and functionality for
  automatically choosing the most appropriate
  registration approach based on image
  characteristics.

6
Overview of approach

• Feature extraction
• Images are divided into regions.
• Features are extracted from each region
• Selection of regions and methods
• The expected performance of each method is
  predicted.
• Regions and methods are selected based on the
  predictions.
• Transform estimation
• Local co-registration is performed with the
  selected method.
• A global transform is estimated from the set of
  local transforms.

7
Feature extraction

• The images are subdivided into rectangular
  regions.
• Regions can be discarded.
• Different methods can be used for different
  regions.
• Features are extracted from a pair of regions.
• The features from the two regions are merged into
  a joint feature vector

8
Features

• Features are selected to say something about
• The information content in the region
• Difference between fixed and moving
• Texture features
• GLCM and gradient features
• from moving image
• difference between fixed and moving image
• Statistical features
• Region and zone means, variance, entropy
• difference between fixed and moving image
• A total of 26 features are extracted from each
  region

9
Predicting performance

• From the extracted features a neural net is used
  to predict the performance of each method for
  each region.
• The net is defined as follows
• N input nodes (N nof features)
• One layer of hidden nodes
• M output nodes (M nof methods)

10
Training of the neural net
Features

• Regions and features are extracted from a set of
  image pairs with known geometric displacement.
• All different matching methods are applied to
  each region.
• The distance between the estimated transform and
  the true transform is computed for each region
  and method.
• Features and truncated distances are used to
  train the net.

Distance from known distortion using each method.
11
Region and method selection
Scores

• Regions with low scores are discarded.
• For the remaining regions, selection is performed
  to retain a good distribution over the image.
• For each of these regions the method with the
  best score is selected.
• Local region matching can then be performed with
  the selected method.

S s(m1), .., s(mm)
12
Methods for region matching
Methods selected from ITK

• Metric
• Normalized cross-correlation
• Mean squares
• Mutual information (different varieties)
• Optimizer
• Gradient Descent
• Regular step gradient descent
• Genetic algorithm
• Transform
• Interpolator

13
Choice of matching methods

• We define a matching method as
• a combination of a metric and an optimizer.
• A selection of 10 different methods is used.

14
Transform estimation

• The selected matching method is used to estimate
  the transform for each of the selected regions.
• The set of estimated transforms is analysed to
  remove obvious outliers.
• Control points are computed for each region based
  on the estimated transforms.
• A global transform is computed from the set of
  control points.

15
Overview of process
Fixed
Moving
Selected regions and methods
Set of region transforms
Reduced set of region transforms
Set of control points
X x1, , xn
16
System overview
17
Test examples

• NOAA-AVHRR
• Challenges Clouds, varying snow cover
• Landsat
• Challenges Clouds, phenology
• ERS1
• Challenges Varying soil moisture, crop maturity

18
NOAA-AVHRR
May 31, 2003
July 7, 2003
19
NOAA-AVHRR
20
NOAA-AVHRR Regions
21
NOAA-AVHRR Region selection
22
NOAA-AVHRR Region selection
23
NOAA-AVHRR Method selection
M1
M4
M5
M6
M7
M9
24
Landsat
Aug 19, 1995
July 31, 1994
25
Landsat
26
Landsat Region/method selection
27
ERS1
April 30, 1993
July 9, 1993
28
ERS1
April 30, 1993
July 9, 1993
29
ERS1
April 30, 1993
July 9, 1993
30
Summary of results

• The test examples have shown that the adaptive
  approach has ability to
• discard areas not suited for registration
• Clouds, Ocean, etc.
• select appropriate methods for the remaining
  areas.
• perform registration under varying conditions
• Varying snow cover, differences in phenology,
  crop maturity, soil moisture etc.

31
Summary

• A general tool for registration has been
  developed.
• The tool uses and adaptive approach and provides
  
• a selection of registration techniques
• intelligence for automatic selection of
• the regions to be used
• the registration techniques to be used
• It works on different images without specific
  tuning.
• It has been tested on time-series of optical and
  radar images and results are promising.

32
End