What%20is%20Texture?

1
What is Texture?

• Texture depicts spatially repeating patterns
• Many natural phenomena are textures

radishes
rocks
yogurt
2
Texton Discrimination (Julesz)
Human vision is sensitive to the difference of
some types of elements and appears to be numb
on other types of differences.
3
Search Experiment I
The subject is told to detect a target element in
a number of background elements. In this example,
the detection time is independent of the number
of background elements.
4
Search Experiment II
In this example, the detection time is
proportional to the number of background
elements, And thus suggests that the subject is
doing element-by-element scrutiny.
5
Heuristic (Axiom) I
Julesz then conjectured the following axiom
Human vision operates in two distinct modes
1. Preattentive vision parallel,
instantaneous (100--200ms), without scrutiny,
independent of the number of patterns,
covering a large visual field. 2.
Attentive vision serial search by focal
attention in 50ms steps limited to small aperture.
Then what are the basic elements?
6
Heuristic (Axiom) II
Juleszs second heuristic answers this
question Textons are the fundamental
elements in preattentive vision, including
1. Elongated blobs
rectangles, ellipses, line segments with
attributes color, orientation,
width, length, flicker rate. 2.
Terminators ends of line segments.
3. Crossings of line segments.
But it is worth noting that Juleszs conclusions
are largely based by ensemble of artificial
texture patterns. It was infeasible to
synthesize natural textures for controlled
experiments at that time.
7
Examples
Pre-attentive vision is sensitive to size/width,
orientation changes
8
Examples
Sensitive to number of terminators Left
fore-back Right back-fore See previous
examples For cross and terminators
9
Julesz Conjecture

• Textures cannot be spontaneously discriminated if
  they have the same first-order and second-order
  statistics and differ only in their third-order
  or higher-order statistics.
• (later proved wrong)

10
1st Order Statistics
11
2nd Order Statistics
12
Capturing the essence of texture

• for real images
• We dont want an actual texture realization, we
  want a texture invariant
• What are the tools for capturing statistical
  properties of some signal?

13
Multi-scale filter decomposition
Filter bank
Input image
14
Filter response histograms
15
Heeger Bergen 95

• Start with a noise image as output
• Main loop
• Match pixel histogram of output image to input
• Decompose input and output images using
  multi-scale filter bank (Steerable Pyramid)
• Match subband histograms of input and output
  pyramids
• Reconstruct input and output images (collapse the
  pyramids)

16
Image Histograms
Cumulative Histograms
s T(r)
17
Histogram Equalization
18
Histogram Matching
19
Match-histogram code
20
Image Pyramids

• Known as a Gaussian Pyramid Burt and Adelson,
  1983
• In computer graphics, a mip map Williams, 1983
• A precursor to wavelet transform

21
Band-pass filtering
Gaussian Pyramid (low-pass images)

• Laplacian Pyramid (subband images)
• Created from Gaussian pyramid by subtraction

22
Laplacian Pyramid
Need this!
Original image

• How can we reconstruct (collapse) this pyramid
  into the original image?

23
Steerable Pyramid
Input image
7 filters used
24
Heeger Bergen 95

• Start with a noise image as output
• Main loop
• Match pixel histogram of output image to input
• Decompose input and output images using
  multi-scale filter bank (Steerable Pyramid)
• Match subband histograms of input and output
  pyramids
• Reconstruct input and output images (collapse the
  pyramids)

25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Simoncelli Portilla 98

• Marginal statistics are not enough
• Neighboring filter responses are highly
  correlated
• an edge at low-res will cause an edge at high-res
• Lets match 2nd order statistics too!

29
Simoncelli Portilla 98

• Match joint histograms of pairs of filter
  responses at adjacent spatial locations,
  orientations, and scales.
• Optimize using repeated projections onto
  statistical constraint sufraces

30
(No Transcript)
31
Texture for object recognition
A jet
32
(No Transcript)
33
Analogy to documents
Of all the sensory impressions proceeding to the
brain, the visual experiences are the dominant
ones. Our perception of the world around us is
based essentially on the messages that reach the
brain from our eyes. For a long time it was
thought that the retinal image was transmitted
point by point to visual centers in the brain
the cerebral cortex was a movie screen, so to
speak, upon which the image in the eye was
projected. Through the discoveries of Hubel and
Wiesel we now know that behind the origin of the
visual perception in the brain there is a
considerably more complicated course of events.
By following the visual impulses along their path
to the various cell layers of the optical cortex,
Hubel and Wiesel have been able to demonstrate
that the message about the image falling on the
retina undergoes a step-wise analysis in a system
of nerve cells stored in columns. In this system
each cell has its specific function and is
responsible for a specific detail in the pattern
of the retinal image.
34
(No Transcript)
35
(No Transcript)
36
1.Feature detection and representation
37
Feature detection

• Sliding Window
• Leung et al, 1999
• Viola et al, 1999
• Renninger et al 2002

38
Feature detection

• Sliding Window
• Leung et al, 1999
• Viola et al, 1999
• Renninger et al 2002
• Regular grid
• Vogel et al. 2003
• Fei-Fei et al. 2005

39
Feature detection

• Sliding Window
• Leung et al, 1999
• Viola et al, 1999
• Renninger et al 2002
• Regular grid
• Vogel et al. 2003
• Fei-Fei et al. 2005
• Interest point detector
• Csurka et al. 2004
• Fei-Fei et al. 2005
• Sivic et al. 2005

40
Feature detection

• Sliding Window
• Leung et al, 1999
• Viola et al, 1999
• Renninger et al 2002
• Regular grid
• Vogel et al. 2003
• Fei-Fei et al. 2005
• Interest point detector
• Csurka et al. 2004
• Fei-Fei et al. 2005
• Sivic et al. 2005
• Other methods
• Random sampling (Ullman et al. 2002)
• Segmentation based patches
• Barnard et al. 2003, Russell et al 2006, etc.)

41
Feature Representation

• Visual words, aka textons, aka keypoints
• K-means clustered pieces of the image
• Various Representations
• Filter bank responses
• Image Patches
• SIFT descriptors
• All encode more-or-less the same thing

42
Interest Point Features
Compute SIFT descriptor Lowe99
Normalize patch
Detect patches Mikojaczyk and Schmid 02 Matas
et al. 02 Sivic et al. 03
Slide credit Josef Sivic
43
Interest Point Features
44
Patch Features
45
dictionary formation
46
Clustering (usually k-means)
Vector quantization
Slide credit Josef Sivic
47
Clustered Image Patches
Fei-Fei et al. 2005
48
Filterbank
49
Textons (Malik et al, IJCV 2001)

• K-means on vectors of filter responses

50
Textons (cont.)
51
Image patch examples of codewords
Sivic et al. 2005
52
Visual synonyms and polysemy
Visual Polysemy. Single visual word occurring on
different (but locally similar) parts on
different object categories.
Visual Synonyms. Two different visual words
representing a similar part of an object (wheel
of a motorbike).
53
Image representation
frequency
codewords
54
Scene Classification (Renninger Malik)
55
kNN Texton Matching
56
Discrimination of Basic Categories
57
Discrimination of Basic Categories
chance
58
Discrimination of Basic Categories
chance
59
Discrimination of Basic Categories
chance
60
Discrimination of Basic Categories
chance
61
Discrimination of Basic Categories
chance
62
Object Recognition using texture
63
Learn texture model

• representation
• Textons (rotation-variant)
• Clustering
• K2000
• Then clever merging
• Then fitting histogram with Gaussian
• Training
• Labeled class data