Multiresolution Histograms and their Use for Texture Classification

1
Multiresolution Histograms and their Use for
Texture Classification

• Stathis Hadjidemetriou, Michael Grossberg and
  Shree Nayar
• CAVE Lab, Columbia University
• Partially funded by NSF ITR Award, DARPA/ONR MURI

2
Fast and Simple Feature
3
Histograms of Filtered Images
Histograms
Histograms
Bin Count
Bin Count
Graylevel
Graylevel
Bin Count
Bin Count
Resolution
s
Graylevel
Graylevel
Bin Count
Bin Count
Graylevel
Graylevel
Bin Count
Bin Count
Graylevel
Graylevel
4
Analysis of Multiresolution Histograms
Bin Count
Graylevel
?
Bin Count Change
Bin Count
Graylevel
Graylevel
Bin Count
Bin Count Change
Graylevel
Graylevel
5
Tools for Analysizing the Histogram

• Shanon Entropy
• Change in Shanon Entropy Fisher Information
• Generalization
• Tsallis Entropy/Generalized Fisher Information

Multiresolution Histogram
Resolution
Bin
Filter Dependent Constant
6
Relating Histogram Change to Image

• Fisher Information
• Measure of image sharpness Stam, 59, Plastino
  et al, 97

Image Gradient
Image
Image Domain
Edge filter never computed Implicit
7
Analysis of Multiresolution Histograms
Bin Count
Graylevel

• Shape Elongation
• Shape Boundary
• Texel Repetition
• Texel Placement

Bin Count Change
Bin Count
Graylevel
Fisher Information
Graylevel
Resolution s
Bin Count
Bin Count Change
Graylevel
Graylevel
8
Shape Elongation and Fisher Information

• Gaussian

• Pyramid

St. dev. along axes sx, sy.
Sides of base rx, ry.
Elongation
Elongation
6
5
(analytically)
4
J
3
2
1
2
3
4
5
r
9
Shape Boundary and Fisher Information
Superquadrics
6
5
J
4
3
2
2
4
6
0
h
10
Texel Repetition and Fisher Information
Tileing
8
6
4
J
2
0
1
4
2
3
5
6
Tileing p
x 103
8
6
4
J
2
0
1
4
2
3
5
6
Tileing p
(analytically).
11
Texel Placement and Fisher Information
Stand. dev. of perturbation
x 103
6.6
6.4
6.2
J
6
5.8
0
15
5
10
20
St. Dev ( of Texel Width)
2.9
2.8
2.7
J
2.6
2.5
0
15
5
10
20
Average of 20 trials
St. Dev ( of Texel Width)
12
Matching Algorithm
Multiresolution histogram with Burt-Adelson
Pyramid
Cumulative histograms
Compute Feature
Difference histograms between consecutive
resolutions
Concatenate to form feature vector
L1 norm
13
Histograms Bin Width

• Histogram bin width

• Subsampling factor in pyramid

14
Parameters of Multiresolution Histogram

• Histogram smoothing to avoid aliasing
• Database images
• Test images

• Histogram normalization
• Image size
• Histogram size

15
Databases for Matching

• Database of Brodatz textures Brodatz, 66
• 91 images 7 images
• Histogram equalized

• Database of CUReT textures Dana et al, 99
• 8,046 images 61 materials
• Histogram equalized

16
Database of Brodatz Textures
Samples of equalized images
17
Match Results for Brodatz Textures
Match under Gaussian noise of st.dev. 15
graylevels
18
Class Matching Sensitivity Brodatz Textures
100
80
60
Class matched
40
20
0
0
10
20
30
40
50
60
St dev. of noise sn
19
Class Matching Sensitivity Brodatz Textures
100
95
90
85
80
75
70
65
60
0
10
20
30
40
50
60
St dev. of noise sn
256 Constant 256, Higher Subsampling 22/3 256,
Lower Subsampling 21/2
smoothing adaptive bin size
20
Database of Curet Textures
Samples of equalized images
21
Match Results for Curet Textures

• Match under Gaussian noise of st.dev. 15
  graylevels.

22
Class Matching Sensitivity CUReT Textures
100
90
80
Class matched
70
60
50
0
10
20
30
40
50
St dev. of noise sn
256 Constant 256, Higher Subsampling 22/3 256,
Lower Subsampling 21/2
Difference norm Smoothing

• Match 100 randomly selected images per noise
  level

23
Comparison with Low-level Features

• Fourier power spectrum annuli
• Gabor features
• Daubechies wavelet features
• Auto-cooccurrence matrix
• Markov random field parameters

24
Comparison with Low-Level Features

• Fourier power spectrum annuli

h
z
r1
r2

• Gabor features

• Auto-cooccurrence matrix

25
Comparison with Low-Level Features

• Wavelet coefficient energies

Wavelet packets decomposition
Wavelets decomposition

• Markov random field parameters

26
Comparison of Computation Costs
1 Markov random field parameters O(n(l2-1)2-(l2-1)3/3)
2 Gabor features O( (logn1)nlogn1/2)
3 Fourier power spectrum features O(n3/2)
4 Auto-cooccurrence matrix O(nl2)
5 Wavelet coefficient energies O(nll)
6 Multiresolution histograms O(nl)
decreasing cost
n- number of pixels l- window width l-
resolution levels
27
Sensitivity Comparison to Transformations
Feature Translation Rotation Uniform Scaling
1 Fourier power spectrum annuli invariant robust equivariant
2 Gabor features invariant variant equivariant
3 Daubechies wavelet energies variant variant variant
4 Multiresolution histograms invariant invariant equivariant
5 Auto-cooccurrence matrix invariant robust equivariant
6 Markov random field parameters invariant variant variant
28
Matching Comparison of Features Brodatz

• Brodatz textures database

100
80
Multiresolution Diff. Histograms Fourier Power
Spectrum Gabor Features Wavelet
Packets Cooccurence Matrix Markov Random Fields
60
Class matched
40
20
0
0
10
20
30
40
50
60
St dev. of noise sn
29
Matching Comparison of Features CUReT

• Curet textures database

100
80
Multiresolution Diff. Histograms Fourier Power
Spectrum Gabor Features Wavelet
Packets Cooccurence Matrix Markov Random Fields
60
Class matched
40
r1
20
0
0
10
20
30
40
50
St dev. of noise sn

• Match 100 randomly selected images per noise
  level

30
Sensitivity of Features to Recognition
Feature Gaussian Noise Database size,classes Illumination Parameter selection
Fourier power spectrum annuli sensitive sensitive robust very sensitive
Gabor features robust robust robust sensitive
Daubechies wavelet energies sensitive robust robust robust
Multiresolution histogram robust robust robust robust
Auto-cooccurrence matrix very sensitive very sensitive very sensitive very sensitive
Markov random field parameters very sensitive very sensitive sensitive N/A
31
Colors