Image registration of satellite images

1
IEEE conf. on Computer Vision and Pattern
RecognitionAnchorage, Alaska, June 24-26, 2008
Overview of image fusion techniques
Jan Flusser and B. Zitova
flusser, zitova_at_utia.cas.cz
Department of Image ProcessingInstitute of
Information Theory and AutomationPod vodarenskou
vezi 4, Prague 8, Czech Republic
2
Motivation
many image analysis tasks are hard to solve from
a single image
3
Motivation
traffic surveillance - can we read the license
plates?
4
Motivation
combining different modalities
Courtesy of Z. Ambler et al.
5
Motivation
combining different modalities
6
Motivation
low-resolution video
7
Empirical observation

• one image is not enough

8
Image Fusion
solution to previous
input several images of the same scene
output one image of higher quality

• the quality - depends on the application area

9
Basic fusion strategy
- acquisition of different images
- image-to-image registration
- image fusion
10
Fusion categories
- multiview fusion
- multitemporal fusion
- multimodal fusion
- multisetting fusion
- multichannel deconvolution
- superresolution
11
Multiview fusion

• images of the same modality, taken at the same
  time but from different places
• Goal to supply complementary information
• from different views

12
Multiview fusion
Metrodome
B. Zitová and J. Flusser, Image registration
methods A survey, Image and Vision Computing,
21(11) 977-1000, 2003
13
Multiview fusion
Courtesy of CMP, CVUT, Prague
S. Seitz et al. A Comparison and Evaluation of
Multi-View Stereo Reconstruction Algorithms, CVPR
2006, vol. 1, pages 519-526
14
Multitemporal fusion

• images of the same scene, taken at different
  times (usually of the same modality)
• Goal detection of changes
• Method subtraction, false color synthesis

1000
1100
1200
1300
15
digital subtraction angiography
Multitemporal fusion
Courtesy of Y. Bentoutou et al.
16
Multitemporal fusion
The Last Judgement Mosaic
2000
1879
17
Multitemporal fusion
The Last Judgement Mosaic change detection
R. Radke et al.Image change detection
algorithms a systematic survey, IEEE
Transactions on Image Processing, Vol.14, March
2005 pp. 294 - 307
18
Multitemporal fusion
image synthesis images of a dynamic scene taken
at certain time

• Goal synthesis of intermediate images
• Method warping blending

19
Multitemporal fusion
20
Multimodal fusion

• images of the different modalities
• (PET, CT, visible, IR, UV, etc.)
• Goal to emhasize band-specific information

UV
visible
IR
SEM
Methods - pixel averaging - fusion in
transform domains - object-level fusion
21
Multimodal fusion
medical imaging pixel averaging
22
Multimodal fusion
security application

• weighted average

Courtesy of R.Blum et al.
Multi-Sensor Image Fusion and Its Applications,
Eds Blum R., Liu Z., CRC Press, (2005)
23
Multimodal fusion
fusion of images with different resolution
high spatial low spectral
low spatial high spectral

• Goal high spatial and spectral resolution
• Method replacing intensity in IHS
• replacing high frequencies
• replacing bands in WT

24
Multimodal fusion
WT based
IWT -gt fused image
25
Multimodal fusion
Courtesy of A.Garzelli
26
Multisetting fusion
- high dynamic range images
- noisy images
- multifocus fusion
- different exposure
27
Multisetting fusion
high dynamic range images
Courtesy of Image Fusion Systems Research
Yuan, L., Sun, J., Quan, L., and Shum, H.-Y.
2007. Image Deblurring with Blurred/Noisy Image
Pairs. In SIGGRAPH 2007
28
Multisetting fusion
denoising via time averaging
before registration
after registration
Courtesy of J. Jan et al.
29
Multifocus fusion

• the acquired image - divided into regions
• - every region is in focus in
• at least 1 channel
• Goal image everywhere in focus
• Method - identify the regions in focus
• - maximizing proper focus measure
• - combine them together

30
Focus measures
- gray-level variance
- energy of gradient
- energy of Laplacian
- moments
- energy of high-pass bands of WT
31
Multifocus fusion in wavelet domain
H. Li, B. S. K. Manjunath and S. Mitra,
"Multisensor Image Fusion Using the Wavelet
Transform," Proc. ICIP 94, Austin, Texas,  Vol.
I, pp. 51-55, Nov 1994.
32
Multifocus fusion in wavelet domain
images with different areas in focus
33
Multifocus fusion in wavelet domain
34
Multifocus fusion in wavelet domain
regularized decision map
max rule
35
Multifocus fusion - microscopic data
36
Multifocus fusion surface reconstruction
37
Multifocus fusion surface reconstruction
38
Multichannel deconvolution
Idea - input images are blurred by
convolution with different convolution
kernels - by fusion via deconvolution the
original scene can be estimated
39
Acquisition model with blur
40
MC Blind Deconvolution

• Energy minimization problem (well-posed)

41
Regularization terms

• Q(u)

42
PSF Regularization
u
43
Incorporating a between-image shift
44
Alternating minimization (AM) of E

• - AM of E(u,gi) over u and gi
• Input - blurred images
• - estimation of the PSF size

Output - reconstructed image - the
PSFs
45
Simulated blurring
46
Multiple blurred images
Blind Image Deconvolution Theory and
Application, Eds Campisi P., Egiazarian K., CRC
Press (2007)
47
Multiple blurred images
The Poor Fisherman, Paul Gauguin, 1896
48
Vibrating scene
49
Out-of-focus camera
50
Astronomical imaging
degraded images
51
Conclusions

• Image fusion is a very powerful tool for
• - improving image quality
• - recognizing objects
• - scene understanding
• whenever more images are available
• Wide variety of fusion methods

52
Thank you!