TEXTURE FEATURE EXTRACTION: improving image contentbased retrieval in multimedia databases by Fiona

1
TEXTURE FEATURE EXTRACTION improving image
content-based retrieval in multimedia
databasesbyFiona Holder Arun George

• PREPARED FOR
• CS848 Prof. Gisli Hjaltason
• University of Waterloo
• Winter, 2003

2
OUTLINE

• Project Motivation
• Background feature extraction
• Texture Extraction
• Review Gabor Wavelets
• Other Wavelet Transforms
• Expanding Usefulness of Gabor Filters
• Co-occurrence Matrix Representation
• Visual Appearance Representation
• Fractal Representation
• Other Methods
• Objectives/Improvements

3
MOTIVATION

• Previous work on Gabor Filters on Textures -
• TEXTURE FEATURES FOR BROWSING RETRIEVAL OF
  DATA by B.S Manjunauth W.Y Ma 1996
• Interest in other methods of feature extraction
  in CBIR literature

4
background

• Feature Extraction - method of capturing visual
  content of images for indexing retrieval.

Image
Feature Extraction
Dimensional Reduction
Embedding Method
Indexing Method
5
background

• Colour Extraction

Example Query Input
Feature Extraction Process
Color Palette Input
Retrieval Results
6
background

• Shape Extraction
• Challenge 3D-shape similarity input

• Methods
• Global Features
• (Moment Invariant,
• Aspect Ratio
• Circularity)
• 2) Local Features
• Boundary segments

Example Query Input
Feature Extraction Process
Color Palette Input
Retrieval Results
7
TEXTURE EXTRACTION

• TEXTURE ANALYSIS
• Wave Transforms
• Co-occurrence Matrix
• Fractal Representation
• Visual Properties
• 5) Random Field Models
• 6) Other Representation

Example Query Input
Feature Extractor
Original Image
FEATURE VECTOR
Distance Calculation
PRECISION Retrieved Images Relevant images
Retrieval Results
Performance Measure
8
TEXTURE EXTRACTIONgabor filters

• High Redudancy
• High Redudancy

High Redudancy
Spatially homogenous
Original Sub-Image
Filter Design Strategy
Tuned Gabor Wavelets/Signals
Low Redundancy High Dimensionality
Dimensionality Reduction
Adaptive Filter Selection
Texture Feature Vector
9
TEXTURE EXTRACTIONwavelet tranforms

• Biorthogonal, Orthogonal Unorthogonal
• Gabor, Multiresolution Simultaneous
  Autoregressive model(MR-SAR) and Tree Structured
  Pyramid Wavelet Transform(PWT)
• Tree-structured Wavelet Transform(TWT)

PWT v/s TWT
Gaussian Filter Bank
10
TEXTURE EXTRACTION wavelet tranforms CONTD
N.B Decomposition of PWT is based on LL while TWT
is based on energy calculations at LL, LH HL
Bands
11
TEXTURE EXTRACTIONwavelet tranform coMPARISONS
12
TEXTURE EXTRACTION EXPANDING Usefulness OF GABOR
FILTERS

• Rotational Invariant Classification
• Analytical Gabor Wavelets based on amplitude,
  frequency(2 invariants) orientation/directional
  ity
• Sample Microfeatures are statistically
  transformed to Macrofeature Rotational
  invariants
• Learning Similarity by Neural Network
• Texture Thesaurus Image Indexing

Sample of a Texture Feature Class
13
TEXTURE EXTRACTION EXPANDING Usefulness OF GABOR
FILTERS
Learning by Kohonen Mapping
Learning by Vector Quantization
Neural Netowork
Texture Feature Space
Clustering by Texture Class
a) Learning Similarity on 2 Stage Neural Network
b) Indexing Strategy
using Dictionary
14
TEXTURE EXTRACTION co-occurrence matrix
representation

• Method of extracting properties of an image by
  comparing gray-tone spatial dependencies between
  pixels
• Matrices of the frequencies(probabilities) of
  going from one gray level to another at a
  predefined distance and different orientations is
  derived
• 14 Statistical measures of texture can be
  extracted from the matrix into a feature vector,
  E.g. Inverse Difference Moment, Energy(Angular
  Second Moment), Contrast, Correlation, Entropy

15
TEXTURE EXTRACTION visual properties
representation

• Tamura compared the Psychological results with
  the Computational results for a few Brodatzs
  images.
• Images where graded according to 6 basic textural
  features namely Coarseness, Contrast,
  Directionality, Line Likeness, Regularity and
  Roughness.
• The results obtained by comparing the Coarseness,
  Directionality and contrast from the above is
  highly correlated but the same cant be said
  about the other 3.
• Another experiment conducted by selecting similar
  type images by humans and Computer was found to
  be accurate by just 41.
• N.B. Lui Picard1995 used a similar approach
  to measure Periodicity, Directionaloty
  Randomness.

16
TEXTURE EXTRACTION fractals representation

• This is a new method developed by Soundararajan
  Ezekiel and John A Cross. They tried to set a
  relationship between Fractal dimension
  coarseness.
• Hurst Exponent is defined by H E1 - FD Where E
  Euclidean Dimension (E0 for point 1 for line 2
  for a surface)
• H is calculated using the following formula
• It was found that small values of FD result in a
  large values of H, which represents texture that
  is fine and if FD is large and H is small then
  the texture is considered coarse.

17
TEXTURE EXTRACTION circular mellin FEATURES

• Disadvantage of Gabor filter is, would be
  challenging to identify identical Texture with
  less or more resolution than the original image
  presented to the filter as the same texture
  scale variant.
• A new method for Texture segmentation is to use
  Circular Mellin features.
• It uses Fourier features with polar log
  coordinates, which help in both rotation as well
  as scaling(invariants).
• where q order of the circular harmonic
  function and p order of the mellin harmonic
  function

18
TEXTURE EXTRACTION black box approach

• VisualHarness is a system developed to retrieve
  data that are homogenous. Reference images in the
  traditional black box system are complete black
  or white images

• Images are usually retrieved with the help of
  the black and white reference images. The result
  obtained during the test is not good when
  retrieving images with the White and Black
  reference images.
• Improvement- Finding a better reference image.
  Correlate the answers by context and content
  based. Make the n features of the objects into
  a n-dimensional and them cluster them If the VIR
  supports 3 types of properties say P1 P2 P3 then
  the user can assign different weights to the
  properties given by w1 w2 w3 then the overall
  weight of the retrieval will be equal to w where
  w w1P1w2P2w3P3 where all w lies between the
  range of 0 and 1.

19
objectives / improvements

• Method Comparison - Critique
• Arbitrary Texture Inputs by edges, Pattern
  sketches or templates
• Performance Similarity measure relevance
  feedback, Neural n/w
• Scale Invariant Classification
• Hybrid Solutions
• Intra-methods invariants, Gabor expansion,
  structural methods
• Inter-methods waveform co-occurrence or
  fractal, psychology
• Inter-feature texture plus shape, colour
  texture
• Address efficient effectiveness issues

20
conclusion

• ANY
• SUGGESTIONS?
• THANKS!