MULTISCALE FILTERBASED TEXTURE ANALYSIS FOR CONTENTBASED IMAGE DATABASE RETRIEVAL

1
MULTI-SCALE FILTER-BASED TEXTURE ANALYSIS FOR
CONTENT-BASED IMAGE DATABASE RETRIEVAL

• New developments in filter-based texture feature
  extraction, feature representation and distance
  measure selection
• A. Baraldi and P. Blonda
• ISSIA-CNR, Bari, Italy

2
Introduction

• GOAL. Starting from existing single- and
  multi-scale filter-based CBIR systems,
• reduce the similarity inaccuracy introduced by
• the feature extraction stage, intrinsically
  non-injective.
• The feature representation stage, intrinsically
  non-injective.
• The similarity measure, whose selection depends
  on the set of features.
• Constrained by
• linear increase of computation complexity.
• Relationships to
• chromatic and achromatic visual information
  processing in the human visual system.
• Image texture analysis/synthesis.
• Statistical pattern recognition.
• Digital signal processing design of
  discrete-time band-pass filters.

3
Topics of discussion

• Survey of existing single- and multi-scale
  filter-based CBIR systems.
• Architectural properties.
• Feature extraction
• Feature representation.
• Distance metric.
• Implementation details.
• Filter types, scales and orientations (if any).
• Relationships to other CBIR systems.
• Advantages and limitations.
• Possible improvements subjected to a linear
  increase of computational complexity.
• Experimental results.

4
Survey of existing filter-based CBIR systems

• Boujemaa et al. (INRIA, Fr, 2000-2002).
• De Bonet (Rosetta system, MIT,1997)
• Manjunath et al.(UC at Santa Barbara, 1996-1997).
• Mojsilovic et al (IBM and Bell Labs, 2000-2001).

5
Survey of existing filter-based CBIR systems 1.
Boujemaas approach.

• Architectural properties.
• Feature extraction. Single-scale edge strength,
  L2 aggregation (rectification) of the squared
  Laplacian ?2(i,j)w(i,j) ?0, i??0,M-1?,
  j??0,N-1?.
• Feature representation. Weighted histogram,
  combining an image-wide first order color
  distribution (color histogram)with spatial
  properties of the color distribution.
• Distance metric. L1 metric, ?2 test.
• Implementation details.
• Filter type. Single-scale (e.g. 7?7) isotropic
  Laplacian filter.

6
Survey of existing filter-based CBIR systems - 1.
Boujemaas approach.

• Advantages. A weighted histogram combines spatial
  with color information.
• Limitations.
• Application-dependent single-scale image
  analysis.
• Non-injective nature of the edge strength
  computation (different edges values may have the
  same strength).
• Non-injective nature of the weighted histogram
  computation (2 pixels with edge strength 5 ltgt
  1 pixels with edge strength 10).
• Proposed improvements.
• Multi-scale image analysis at least
• 4 spatial scales are necessary to mimic
• the human visual system
• ltgt application independence.
• Separate from edge values in
• sample statistic estimation.
• 2-D (color histogram, weighted
• histogram) feature representation.

7
Survey of existing filter-based CBIR systems 2.
De Bonets approach.

• Architectural properties.
• Feature extraction. Rectified (L2-aggregated)
  multi-scale oriented filter transformation.
• Feature representation. Mean of each filtered
  image as a leaf of the decomposition tree.
• Distance metric. L1 metric.
• Implementation details.
• Filter types. Scales S4, 25 oriented filters for
  edge and bar detection.
• Advantages. Multi-scale image analysis.
• Limitations.
• Non-injective nature of the filtered image
  rectification.
• Non-injective nature of the mean sample
  computation.
• ltgt Poor color and structural element (texture)
  representation (query deer retrieved image
  car).

8
Survey of existing filter-based CBIR systems 3.
Manjunaths approach.

• Architectural properties.
• Feature extraction. Rectified (L2-aggregated)
  multi-scale oriented filter transformation.
• Feature representation. Mean and StDev,
  standardized over the training data set.
• Distance metric.
• Implementation details.
• Filter types. Non-orthogonal Gabor wavelets
  (redundant information decomposition). Scales
  S4, orientations O6.

9
Survey of existing filter-based CBIR systems - 3.
Manjunaths approach.

• Advantages. Multi-scale image analysis.
• Limitations.
• Non-injective nature of the filtered image
  rectification.
• Non-injective nature of the mean and stdev sample
  computation.
• ltgt 74 of the correct patterns are in the top 15
  retrieved images, 92 in the top 100 retrieved
  images.
• Proposed improvements.
• Complete (near-orthogonal) basis for the Gabor
  wavelet transform of the input image.
• Separate from convolution values in sample
  statistic estimates.

10
Survey of existing filter-based CBIR systems 3.
Manjunaths approach.

• Limitation Information loss in Gabor filter
  response rectification.

11
Survey of existing filter-based CBIR systems 4.
Mojsilovics approach.

• Architectural properties.
• (Color and achromatic texture) Feature
  extraction.
• Single-scale dominant Lab color extraction at
  full resolution.
• Lab color space transformation (L luminance,
  (a, b) chrominance).
• Uniform discretization of linear L at a given L
  value, (a, b) discretization by Fibonacci
  lattices.
• Consider dominant those discrete colors with
  occurrence gt 3 image area.
• Achromatic texture map extraction from dominant
  colors.
• Level 0 is assigned to the dominant color
  featuring the largest percentage of pixels, the
  next level is assigned to the second dominant
  color, etc., until level 255 is assigned to the
  dominant color featuring the lowest area
  percentage.
• Rectified (L2-aggregated) multi-scale achromatic
  texture map wavelet transform.

12
Survey of existing filter-based CBIR systems 4.
Mojsilovics approach.

• Architectural properties.
• (Color and achromatic texture) Feature
  extraction.
• 2. Achromatic texture map extraction from
  dominant colors ltgt Two images featuring the same
  spatial distribution of colors (i.e. which do not
  differ in terms of form, shape and orientation)
  but employ different color combinations do
  generate the same texture map, see Fig. 1.

13
Survey of existing filter-based CBIR systems 4.
Mojsilovics approach.

• Architectural properties.
• Feature representation. Mean and StDev,
  standardized over the training data set (like in
  Manjunaths approach).
• Color distance metric. Given the n-th dominant
  color element, (nA, pn, A) in image A, where nA ?
  ?1, DA ?, pn, A ? 0, 1, the distance between
  color element (nA, pn,A) and the set of color
  elements ?(iB, pi,B), iB1,,DB? belonging to
  image B is

14
Survey of existing filter-based CBIR systems 4.
Mojsilovics approach.

• Architectural properties.
• Achromatic texture distance metric. Let us
  consider two images, A and B, texturally
  parameterized as
• fT(A)? ? and
• fT(B)? ?,
  respectively.
• If then
• Weighting factors ?M and ?? are designed
  such that when the difference in standard
  deviation is small, the first term is more
  dominant as it increases, the second term
  becomes dominant, in line with human perception

Logistic funtion
1
0.5
(0,0)
Distance
15
Survey of existing filter-based CBIR systems 4.
Mojsilovics approach.

• Implementation details.
• Filter types. Multi-scale oriented Derivative of
  Gaussian (DOG) filters. Scales S4, orientations
  O6 (like in Manjunaths approach).
• Advantages.
• Separate (but interactive) processing of
  luminance, chromaticity and achromatic texture
  information to mimic the human visual system.
  Like in the in the human visual system, color
  information is processed first, and then
  achromatic texture. This approach allows queries
  such as
• find all patterns of a given color, but a bit
  lighter,Find all patterns of similar
  colors,Find more saturated patterns,Find the
  same pattern (irrespective of color),Find
  similar patterns (irrespective of color),Find
  similar patterns of similar colors.
• Biologically plausible compact (dominant) color
  representation.
• Biologically plausible multi-scale achromatic
  texture analysis, unrelated to chrominance and
  luminance. Rather, patterns are perceived through
  the interaction of image edges at different
  orientations and at different scales.
• New single-scale color and multi-scale texture
  distance functions that correlate with human
  performance.

16
Survey of existing filter-based CBIR systems 4.
Mojsilovics approach.

• Limitations.
• Single-scale color analysis.
• Rectification and thresholding of filter
  convolution values.
• Proposed improvements.
• Multi-scale color analysis.
• Complete (near-orthogonal) basis for the Gabor
  wavelet transform of the input image.
• Separate from convolution values in sample
  statistic estimates.

17
Experimental results

• Data sets.
• Synthetic (intensity, hue, saturation,
  orientation, regularity, structural elements).
• Wave, 100 instances, 4 classes.
• Wood, 100 instances, 5 classes.
• Natural. Polishing glazed ceramic tiles, 100
  instances, 3 classes.

18
Experimental results

• Implementations.
• Adapted Boujemaas approach.
• 2-D combined (color histogram, edge
  strength-weighted histogram) feature space.
• 1- to 5-nearest neighbor classifier
  (non-parametric benchmark classifier in pattern
  recognition applications).
• Adapted Manjunaths approach.
• Complete (near-orthogonal) even-symmetric Gabor
  filter basis for the multi-scale wavelet
  transform of the input image.
• Separate from convolution values in sample
  statistic estimates.
• 1- to 5-nearest neighbor classifier
  (non-parametric benchmark classifier in pattern
  recognition applications).

19
Experimental results

• Results. Non-parametric 1-nearest neighbor
  classifier.

20
Conclusions

• Result assessment.
• Feature extraction.
• Near-orthogonal even-symmetric multi-scale Gabor
  transform.
• Feature representation.
• In wavelet transform sample statistic estimates,
  separate from values to avoid rectification.
• Best feature representation and distance measures
  are application-dependent.
• Significant performance enhancement with simple
  adaptations requiring a linear increase of
  computational complexity.
• Future developments.
• Enhanced Mojsilovics approach.

21
References

• J. S. De Bonet, Novel Statistical Multiresolution
  Techniques for Image Synthesis, Discrimination,
  and Recognition, Pd.D. Thesis, MIT, 1997.B. S.
  Manjunath and W. Ma, "Texture features for
  browsing and retrieval of image data," IEEE
  Trans. Pattern Anal. Machine Intel.,vol. 18, no.
  8, pp. 837-842, 1996.A. Mojsilovic, J.
  Kovacevic, Jianying Hu, R. Safranek, and S. Kicha
  Ganapathy, "Matching and retrieval based on the
  vocabulary and grammar of color patterns," IEEE
  Trans. Image Processing, vol. 9, no. 1, pp.
  38-54, 2000.C. Vertan and N. Boujemaa,
  "Upgrading color distributions for image
  retrieval can we do better?", Int. Conf. Visual
  Information Systems, VISUAL 2000,
  http//www-rocq.inria.fr/boujemaa/Publications.h
  tml.H. R. Wilson and J. R. Bergen, A four
  mechanism model for threshold spatial vision'',
  Vision Res., vol. 19, pp. 19-32, 1979.A. Jain,
  R. Duin, and J. Mao, Statistical pattern
  recognition A review,' IEEE Trans. Pattern
  Anal. Machine Intell., vol. 22, no. 1, pp. 4-37,
  2000.A. Baraldi and F. Parmiggiani, Combined
  detection of intensity and chromatic contours in
  color images,' Optical Engineering, vol. 35, no.
  5, pp. 1413-1439, May 1996.