Automatic Image Registration and Image Time Series Mining

1
Automatic Image RegistrationandImage Time
Series Mining

• Alain Giros - CNES

2
Automatic Image Registration
3
Objectives

• Given 2 images satisfying some conditions
• Sufficient overlapping
• Same observed objects
• One image being considered as the reference
• Find the geometric deformation of the second
  image so that the deformed image is
  superimposable to the reference one

4
Rationale

• Image registration is needed for
• Consistent browsing of multisensor catalogs
• Building mosaics made of several images
• Computing geometrically consitent image features
• Image registration should be
• Automatic
• Fast (less than some minutes for an image pair)
• Accurate (some hundredths of a pixel)

5
General Schema
Geometric deformation
Deformation model estimation
Reference Image
Step 1  Deformation Model Estimation
Step 2  Geometric Deformation by Interpolation
Input Image
Registered image
Deformation Model
6
General Schema

• Geometric deformation of an image is well known.
• use of ORION

Geometric deformation
Step 2  Geometric Deformation by Interpolation
Registered image
Deformation Model
Input Image
7
General Schema
Deformation model estimation
Reference Image
Step 1  Deformation Model Estimation
Step 2  Geometric Deformation by Interpolation
Input Image
Registered image
Deformation Model
8
Deformation model estimation

• Conditions on the two images
• Must overlap enough
• Comparable resolution
• 110 è 101
• Acquired signals shall come from the same objects
  in the scene
• Cloud free optical and SAR è OK
• Clouded optical and SAR è NOK

Reference Image
Step 1  Deformation Model Estimation
Input Image
Deformation Model
9
Deformation model estimation

• The estimation is made in 2 steps
• Global transform estimation
• Local disparity map

Reference Image
Step 1  Deformation Model Estimation
Input Image
10
Global transform estimation

• The global geometric transform model is linear
• Or, expressed as similitudes transforms

11
Global transform estimation

• We have 4 unknowns k, q, Tx, Ty
• Possible solutions
• Automatic estimation using image content only
• Automatic computation using the scene
  geographical coordinates
• Manual pointing of homologous points (GCPs)

12
Global transform estimation

• GCPs or scenes coordinates
• A pair of homologous points gives 2 equations.
  Thus we need 2 pairs of corresponding points to
  solve the system

13
Global transform estimation

• GCPs or scenes coordinates
• Then the solution is given by

14
Deformation model estimation

• Local disparity map estimation
• Feature extraction (simplestpixels)
• Similarity measures between features
• Optimization strategy

Reference Image
Step 1  Deformation Model Estimation
Input Image
15
What is the disparity map ?

• Given a pair of images, the vector field which
  allows to map each pixel of one image into one
  pixel of the other is called the disparity map.
• When the disparity map is known, one can
• Register one image onto the other by
  interpolation
• Use the disparity map as a transformation to
  track information from one image to other images.
• Interpret the values of the disparities to devise
  pertinent information (DTM, DEM, perturbations, )

16
Building the disparity map

• Starting with a pair of images
• One considered as the reference
• Compute the disparity at selected nodes

17
Building the disparity map

• For a given node
• Extract a chip surrounding the node

18
Building the disparity map

• Select a corresponding area in the other image,
  depending on the desired searching amplitude

19
Building the disparity map

• For each possible position of the chip into the
  searching image, compute a similarity measure

Local similarity function
20
Building the disparity map

• Take the location of the maximum of the local
  similarity function
• The Vector joining the center of the local
  similarity function to the location of the
  maximum is the desired disparity
• Possibly get subpixel accuracy by interpolating
  the searching image

Local similarity function
21
Deformation model estimation

• The fine estimation is done with MEDICIS
• It produces a deformation model

Reference Image
Step 1  Deformation Model Estimation
Input Image
Deformation Model
22
The deformation model
Registration Model of image I onto object O
Deformation Model of I
Registration quality of image I onto object O
23
The deformation model

• Used to describe the geometric transformation to
  be applied on an image

Deformation model
Geometric models
Validity domains
24
The deformation model

• Geometric model
• Two types
• Coordinate transform (acts on the point
  coordinates)
• Sampled geometric model (acts on the point
  coordinates and on its values)
• Both rely on analytical models
• One or several equations
• Parameters

25
The deformation model

• Coordinate transform
• Some examples

Affine X ax by c Y dx ey f
Polynomial X a bx cx² zxn Y a
bx cx² zxn
Projective X (ax by c)/(dx ey l) Y
(fx gy h)/(dx ey l)
26
The deformation model
- Sampled Geometric Model - Samples
Interpolation
Samples types
Interpolation types
Bilinear
Bicubic
Nearest Neighbour
Regular Grid
Sparse Samples
Least Squares
B-Splines
Apodized sinc
27
The deformation model
Geometric Deformation Model
Is a
Sampled Geometric Model
Is a
Samples model
Coordinate Transform
uses
Interpolation Model
Is an
Is an
Analytical Model
28
The deformation model

• Validity domains
• Used to represent
• Different deformations on different image areas
• Hierarchical decomposition of the image space

29
The deformation model

• Validity domains
• Different deformations on different image areas

Deformation model D1
Deformation model D3
Deformation model D2
30
The deformation model

• Validity domains
• Different deformations on different image areas

31
The deformation model

• Validity domains
• Hierarchical decomposition of the image space

Quadtree decomposition
Irregular decomposition
Domain D1
Domain D1
D1.1
D1.3
D1.2
D1.1
D1.2
D1.4
D1.3
32
The deformation model

• Validity domains model

Validity Domain
Links
Priority
Inheritance
Connexity
Contour
Complex
Simple
Portions
Circle
Rectangle
Triangle
Line
Circle arc
33
The deformation model

• Implementation

34
The deformation model

• Implementation

An UML Diagram
An XML Format
  lt?xml version"1.0" encoding"UTF-8"
standalone"no" ?gt - ltXML_MODELE_TESTgt-
- ltModele_de_deformationgt -
ltDeformation Nom"TRANSFOCOORD_AFFINE"gt
ltListe_Paramètresgt1.0 0.0 0.0 1.0 0.0
0.0lt/Liste_Paramètresgt
ltPrecision_modelegt0.005lt/Precision_modelegt
lt/Deformationgt lt/Modele_de_deformati
ongt lt/XML_MODELE_TESTgt
35
Registration vs Georeferencing

• Image registration links every point in image A
  to a point in image B (in the overlapping area)
• (xA, yA) çè (xB, yB)
• Georeferencing links every coordinate in image A
  to a point in a geographical reference system
• (xA, yA) çè (lG, fG,h)

36
Registration vs Georeferencing

• If image A is already georeferenced
• (xA, yA) çè (lG, fG,h)
• And we register image B onto image A
• (xB, yB) çè (xA, yA)
• Then image B inherits the georeferencing of image
  A
• (xB, yB) çè (xA, yA) çè (lG, fG,h)
• But this inherintance cumulates the errors done
• During the registration process
• During the georeferencing of image A

37

• Image registration test case
• Ikonos image over Landsat image

38
Original Images
Ikonos Pan
Landsat B4-B5-B7
39
Registration Problems

• Choice of the Landsat band closest to the Ikonos
  Pan
• Estimation of a rough global geometric transform
  between the 2 images
• Fine registration between the 2 images

40
Global transform estimation
Landsat and Ikonos images at full resolution, 40
dpi
Obviously, the image content is not
compatible (texture on landsat, geometry and
texture on Ikonos)
41
Global transform estimation
Landsat image at full resolution, Ikonos image
subsampled 125, 40 dpi
42
Global transform estimation

• Manual pointing of homologous points (GCPs)

43
Global transform estimation

• Manual pointing of homologous points (GCPs)
• With these 4 GCPs we have 6 different solutions
• Accuracy is consistent with the pointing
  precision, which is some tenths of a Landsat
  pixel.

44
Results
Original Landsat - Excerpt
45
Results
Original Ikonos subsampled 130, rotated 12,88
46
Results
Ikonos co-registered subsampled 130, rotated
12,88, locally distorded
47
Results
Disparity map in lines and columns
48
Actions

• Test registration with Ikonos as reference
• Test with SPOT and ERS pair
• Assess the acurracy of the registration
  (checkerboard )

49
Image Time Series Mining
50
Time Series Data
51
Time Series Data

• Low Resolution ITS
• Some global phenomena
• Available physical models
• Regularly sampled
• High Resolution ITS
• Natural Anthropic phenomena
• Numerous
• Complex
• Different Time-scale
• No physical models available
• Irregularly sampled

52
Information in HR time Series
53
ITS Representation mining
54
(No Transcript)
55
(No Transcript)
56
Implementation
57
Semantics attached to sub-graphs Spatio-temporal
patterns
58
(No Transcript)
59
Implementation
60
1
2
3-
4-