Diapositiva 1

1
Information-Theoretic Image Fusion Assessment
Without Reference
Luciano Alparone1, Bruno Aiazzi2, Stefano
Baronti2, Andrea Garzelli3, Filippo Nencini3
1 Department of Electronics Telecommunications,
University of Florence, 3 via Santa Marta, 50139
Florence, Italy 2 Institute of Applied Physics
Nello Carrara IFAC-CNR, 10 via Madonna del
Piano, 50039 Sesto F.no (FI), Italy 3 Department
of Information Engineering, University of Siena,
56 via Roma, 53100 Siena, Italy
2
Outline

• Quality assessment of multispectral (MS) images
  spatially enhanced by means of a panchromatic
  (Pan) image
• Overcoming the need of a reference for measuring
  a quality index blind assessment
• Definition of a blind spectral distortion index
• Definition of a blind spatial distortion index
• A cumulative blind index measuring the quality
  of pan-sharpened MS images
• Comparisons on spatially degraded Ikonos data
  between the proposed index and other quality
  indices requiring a reference
• Inter-scale comparisons of fusion quality of
  Ikonos data
• Concluding remarks and future developments

3
Recent Pan-Sharpening Algorithms

• Fusion based on component substitution
• Generalized IHS (Tu et al., Information Fusion,
  2001)
• Fusion with multiresolution analysis
• Additive Wavelet (AWL) (Nunez et al., IEEE TGARS,
  1999)
• ARSIS, (Ranchin and Wald., PERS, 2000)
• GLP with Context-Based Decision, (CBD), (Aiazzi
  et al., IEEE TGARS, 2002)
• WiSpeR Method (Otazu et al., IEEE TGARS, 2005)
• Genetic Algorithm Fusion, (GA), (Garzelli et al.,
  IJRS, 2006)
• Commercial Software solutions
• Labens Gram-Schmidt (GS) Spectral Sharpening,
  (ENVI Software)
• Zhangs UNB-Pansharp, (PCI Geomatica Software)
• Lius Smoothing Filter-based Intensity Modulation
  (SFIM) (ER Mapper Software)

4
Fusion Assessment Walds Protocol

• To assess the quality of the fused images, three
  criteria should be verified as expressed by
• Wald et al., PERS, 1997, Best Paper Award
  97
• 1. Any fused image, once degraded to its
  original resolution, should be equal to the
  original.
• 2. Any fused MS image should be as identical as
  possible to the MS image that the corresponding
  sensor would observe with the high spatial
  resolution of the Pan sensor.
• 3. The MS set of fused images should be as
  identical as possible to the set of MS images
  that the corresponding sensor would observe with
  the high spatial resolution of Pan.
• Since the reference MS images are not available
  to verify the second and the third property, Wald
  suggests applying the following protocol
• - Degrade spatially the Pan and MS images by the
  same factor,
• - Fuse the MS images at the degraded scale,
• - Compare the fused MS images with the
  original-reference MS images.
• Several score indices are used to evaluate the
  performances of the fused images
• - Intra-band indices to measure spatial
  distortions (radiometric and geometric
  distortions),
• - Inter-band indices to measure spectral
  distortions (colour distortions).
• Performances of fusion methods are supposed to be
  invariant when fusion algorithms are applied to
  the full spatial resolution.

5
Quality Indices (1/2)
Spectral Distortion

• Spectral Angle Mapper (SAM)

Radiometric Distortion

• Root Mean Square Error (RMSE)
• Mean Bias

Geometric Distortion

• Correlation Coefficient (CC)

6
Quality Indices (2/2)
Radiometric and Spectral Distortions

• Erreur Relative Globale Adimensionelle de
  Synthese, (ERGAS), (Ranchin Wald, PERS 2000)

Radiometric, Geometric and Spectral Distortions

• Q4 Index, (Alparone et al., IEEE GRSL, 2004,
  Best Paper Award 04)

7
Fusion Assessment Zhous Protocol

• As an alternative to Wald's protocol, the problem
  of measuring the fusion quality may be approached
  at the full spatial scale without any degradation
  by applying Zhous Protocol (Zhou et al., IJRS,
  1998).
• The spectral and spatial qualities are separately
  evaluated from the available data from the low
  resolution MS bands and high resolution Pan
  image.
• The spectral quality is calculated for each band
  as absolute cumulative difference between the
  fused and the input MS images.
• The spatial quality is derived as the correlation
  coefficient (CC) between the spatial details,
  extracted by means of a Laplacian filter, of the
  Pan image and of each of the fused MS bands.
• Problems
• - The two quality measures are not combined
  together,
• - At degraded scale, results are not in
  agreement with objective quality indices.

8
Blind Quality Index

• We present a blind index capable of jointly
  measuring the spectral and spatial quality at the
  full scale, same as in Zhou's protocol, and of
  matching the outcome of the main objective
  indices, when working at degraded scale.
• The spatial and spectral distortion are
  calculated from the mutual information between
  either original MS image and spatially degraded
  Pan image, or fused MS image and original Pan
  image.
• The rationale is that the mutual information
  relationships between any two spectral bands and
  between each band and the Pan image should be
  unchanged after fusion.
• The spectral and spatial distortion indices are
  complemented and combined together to obtain a
  single index that measures the global quality of
  the fused image.

9
Mutual Information

• Mutual Information (MI)
• By modelling the two information sources as a
  locally stationary and ergodic bivariate Gaussian
  random process, the MI may be calculated as

10
Spectral Distortion Index

• The spectral distortion is given by the
  difference of MI values calculated from the fused
  MS bands and from the input
  MS bands , resampled to the
  spatial resolution of Pan
• The MI is calculated for each couple of bands of
  the fused and resampled MS data to form two
  matrices with main diagonal equal to 1
• The spectral distortion is measured by a value
  proportional to the p-norm of the difference of
  the two matrices

11
Spatial Distortion Index

• The MI relationships between each of the MS bands
  and Pan should not change with scale
• The MI is calculated between
• - each fused MS band and the Pan image,
• - each input MS band and the spatially degraded
  Pan image,
• The spatial distortions are calculated by a value
  proportional to the q-norm of the differences

12
Jointly Spectral and Spatial Quality Index

• The separate use of the two indices does not
  allow a performance ranking among fusion methods
  to be established
• A unique index, namely QNR (Quality with No
  Reference) is introduced as
• The and exponents separately attribute
  the relevance of spectral and spatial distortions
  to the overall quality and jointly determine the
  non-linearity of response in the interval 0,1,
  same as a gamma exponent.
• The highest values of QNR are obtained when
  spectral and spatial distortions are
  simultaneously close to 0 (
  )

13
Fusion Results (4m)
(REF)
(CBD)
(GIHS-GA)
(GS)
(Tu-IHS)
(EXP)
14
Separate Evaluation of the Distortion Indices

• 2048 ? 2048 fragment from urban area of an Ikonos
  image, degraded by 4 to compare the proposed
  index with other quality indices
• Spectral and spatial distortions
• are measured for some
• fusion methods

Spectral Distortions
Spatial Distortions
15
Evaluation of the Cumulative Quality Index
QNR
16
Results at Full Scale (1m)

• The discrimination capability is preserved at the
  full scale
• The best fusion results are given by the CBD
  algorithm followed by the GIHS-GA method

17
Conclusions and Developments

• It has been demonstrated that the MI between
  fused MS bands and either the resampled originals
  or the Pan image can be profitably used to
  measure either spectral and spatial qualities,
  without resorting to spatial degradation of the
  dataset to a coarse scale.
• Experimental results, carried out on Ikonos data
  by means of several fusion methods, demonstrate
  that the results provided by the proposed
  information-theoretic method substantially agree
  with evaluations performed on spatially degraded
  data by using reference originals.
• The performance ranking of fusion methods may
  depend on the spatial scale at which fusion is
  accomplished.
• Current investigations are on tools for measuring
  inter-band and band-to-Pan relationships better
  than approximations of mutual information do.