An image registration technique for recovering rotation, scale and translation parameters

1
An image registration technique for recovering
rotation, scale and translation parameters

• March 25, 1998
• Morgan McGuire

2
Acknowledgements

• Dr. Harold Stone, NEC Research Institute
• Bo Tao, Princeton University
• NEC Research Institute

3
Problem Domain
Satellite, Aerial, and Medical sensors produce
series images which need to be aligned for
analysis. These images may differ by any
transformation (possible noninvertible).
Images courtesy of Positive Systems
4
New Technique

• Solves subproblem (practical case)
• O(ns(NlogN)/4kNk) compared to O(NlogN), O(N3)
• Correlations typically gt .75 compared to .03

5
Structure of the Talk

• Differences Between Images
• Fourier RST Theorem
• Degradation in the Finite Case
• New Registration Algorithm
• Edge Blurring Filter
• Rotation Scale Signatures
• Experimental Results
• Conclusions

6
Differences Between Images

• Alignment
• Occlusion
• Noise
• Change

7
Sub-problem Domain

• Alignment RSTL
• Occlusion lt 50
• Noise Change Small
• Square, finite, discrete images
• Image cropped from arbitrary infinite texture

8
RST Transformation
9
Fourier Rotation, Scale, and Translation Theorem
Pixel Domain Fourier Domain p rotate(r,
f) P rotate(R, f) p dilate(r, s) Fp s2
. dilate(Fr, 1/s) p translate(r, Dx, Dy) ÐFp
translate(ÐFr, Dx, Dy)
10
For Infinite Images
11
In practice, we use the DFT
Let X0 DFT(x0) X0 and x0 are discrete, with N
non-zero coefficients. Let X DTFT(x)
X0 and x0 are sub-sampled tiles (one period
spans) of X and x. The Fourier RST theorem holds
for X and x... does it also hold for X0 and x0?
12
Fourier Transform and Rotations
13
Theorem
Infinite case Fourier transform commutes
with rotation
Folklore It is true for the finite case
14
Using Fourier-Mellin Theory

• Magnitude of Fourier Transform exhibits rotation,
  but not translation
• Registration algorithm
• Correlate Fourier Transform magnitudes for
  rotation
• Remove rotation, find translation
• Generalizes to find scale factors, rotations, and
  translation as distinct operations

15
Folklore is wrong
Image
Tile
Rotate
Tile
Image
Rotate
16
The Mathematical Proof
The Finite Fourier transform
continuous
Windowing, sampling, infinite tiling
Transform, then rotate
17
The Mathematical Proof
Rotate, then transform
18
Finite-Transform Pairs
19
The Artifacts
20
Fourier Transforms
Oppenheim Willsky Signals Systems Oppenheim
and Schafer, Discrete-Time Signal Processing
21
Tiling does not Commute with Rotation
Tiled Image
Rotated Tiled Image
Tiled Rotated Image
so the Fourier RST Theorem does not hold for DFT
transforms.
22
Correlation Computation
23
Prior Art

• Alliney Morandi (1986)
• use projections to register translation-only in
  O(n), show aliasing in Fourier T theorem
• Reddy Chatterji (1996)
• use Fourier RST theorem to register in O(NlogN)
• Stone, Tao McGuire (1997)
• show aliasing in Fourier RST theorem

24
An Empirical Observation
Even though the Fourier RST Theorem does not hold
for finite images, we observe the DFT does have a
signature that transforms in a method predicted
by the Theorem.
Image
DFT Magnitude
25
Sources of Degradation

• Frequency
• Aliasing (from Tiling)
• Artifact
• Sampling Error
• Pixel
• Image Window Occlusion
• Image Noise

26
Algorithm Overview
1. Pre-Process
5. Recover Translation Parameters
2. FMLP Transform
4. Recover Rotation Parameter
3. Recover Scale Parameter
Norm. Circ. Corr.
27
Problem Artifact
28
Solution Edge-Blurring Filter, G
29
ProblemNeed Orthogonal Invariants
Fourier-Mellin transform
In the log-polar (logr,q) domain
30
Mapping (wx,wy) to (logr,q)
wx4 wy4
31
Sample Image Pair
f 17.0o s 0.80 Dx 10.0 Dy -15.0
N 65536 k 2
G(r)
G(p)
32
Nonzero Fourier Coefficients
P
R
33
Solution I Rotation Signature
1. Selectively weight edge coefficients (J
filter) 2. Integrate along r axis F is Scale and
Translation Invariant. Pixel rotation appears as
a cyclic shift gt use simple 1d O(nlogn)
correlation to recover rotation parameter.
34
F Signatures of r and p
35
F Correlations
36
Solution II Scale Signature
1. Integrate along q axis (rings) 2. Normalize by
r (area) 3. Enhance S/N ratio (H filter) S is
Rotation and Translation Invariant. Pixel
dilation appears as a translation gt use simple
1d O(nlogn) correlation to recover scale
parameter.
37
Raw S Signature
38
Filtered S Signature
39
S Correlation
40
New Registration Algorithm
Norm. Circ. Corr.
Compute full-resolution Correlation for small
neighborhood of Coarse (Dx, Dy) to refine.
41
Recovered Parameters
42
Disparity Map
43
Multiresolution for Speed

• Algorithm is O(NlogN) because of FFTs
• With kth order wavelet, O((NlogN)/4k)
• To refine, search 22k 4k positions
• Using binary search, k extra trials _at_ O(N) each
• Total algorithm is O((NlogN)/4k Nk)

44
Results Confidence
45
Analysis of Results
46
Future Directions

• Better scale signature
• Use occlusion masks for FM techniques?
• Combining FM technique with feature based
  techniques