Wavelets (Chapter 7)

1
Wavelets (Chapter 7)

• CS474/674 Prof. Bebis

2
STFT (revisited)

• Time/Frequency localization depends on window
  size.
• Once you choose a particular window size, it will
  be the same for all frequencies.
• Many signals require a more flexible approach -
  vary the window size to determine more accurately
  either time or frequency.

3
The Wavelet Transform

• Overcomes the preset resolution problem of the
  STFT by using a variable length window
• Use narrower windows at high frequencies for
  better time resolution.
• Use wider windows at low frequencies for better
  frequency resolution.

4
The Wavelet Transform (contd)
Wide windows do not provide good localization at
high frequencies.
5
The Wavelet Transform (contd)
Use narrower windows at high frequencies.
6
The Wavelet Transform (contd)
Narrow windows do not provide good localization
at low frequencies.
7
The Wavelet Transform (contd)
Use wider windows at low frequencies.
8
What are Wavelets?

• Wavelets are functions that wave above and
  below the x-axis, have (1) varying frequency, (2)
  limited duration, and (3) an average value of
  zero.
• This is in contrast to sinusoids, used by FT,
  which have infinite energy.

Sinusoid
Wavelet
9
What are Wavelets? (contd)

• Like sines and cosines in FT, wavelets are used
  as basis functions ?k(t) in representing other
  functions f(t)
• Span of ?k(t) vector space S containing all
  functions f(t) that can be represented by ?k(t).

10
What are Wavelets? (contd)

• There are many different wavelets

Morlet
Haar
Daubechies
11
What are Wavelets? (contd)
(dyadic/octave grid)
12
What are Wavelets? (contd)
j
scale/frequency localization
time localization
13
Continuous Wavelet Transform (CWT)
Scale parameter (measure of frequency)
Translation parameter, measure of time
Normalization constant
Forward CWT
Continuous wavelet transform of the signal f(t)
Mother wavelet (window)
Scale 1/j 1/Frequency
14
CWT Main Steps

• Take a wavelet and compare it to a section at the
  start of the original signal.
• Calculate a number, C, that represents how
  closely correlated the wavelet is with this
  section of the signal. The higher C is, the more
  the similarity.

15
CWT Main Steps (contd)

• 3. Shift the wavelet to the right and repeat
  steps 1 and 2 until you've covered the whole
  signal.

16
CWT Main Steps (contd)

• 4. Scale the wavelet and repeat steps 1 through
  3.
• 5. Repeat steps 1 through 4 for all scales.

17
Coefficients of CTW Transform

• Wavelet analysis produces a time-scale view of
  the input signal or image.

18
Continuous Wavelet Transform (contd)

• Inverse CWT

double integral!
19
FT vs WT
weighted by F(u)
weighted by C(t,s)
20
Properties of Wavelets

• Simultaneous localization in time and scale
• - The location of the wavelet allows to
  explicitly represent the location of events in
  time.
• - The shape of the wavelet allows to represent
  different detail or resolution.

21
Properties of Wavelets (contd)

• Sparsity for functions typically found in
  practice, many of the coefficients in a wavelet
  representation are either zero or very small.
• Linear-time complexity many wavelet
  transformations can be accomplished in O(N) time.

22
Properties of Wavelets (contd)

• Adaptability wavelets can be adapted to
  represent a wide variety of functions (e.g.,
  functions with discontinuities, functions defined
  on bounded domains etc.).
• Well suited to problems involving images, open
  or closed curves, and surfaces of just about any
  variety.
• Can represent functions with discontinuities or
  corners more efficiently (i.e., some have sharp
  corners themselves).

23
Discrete Wavelet Transform (DWT)
(forward DWT)
(inverse DWT)
where
24
DFT vs DWT

• FT expansion
• WT expansion

one parameter basis
or
two parameter basis
25
Multiresolution Representation using
fine details
narrower, small translations
j
coarse details
26
Multiresolution Representation using
fine details
j
coarse details
27
Multiresolution Representation using
fine details
wider, large translations
j
coarse details
28
Multiresolution Representation using
high resolution (more details)
j

low resolution (less details)
29
Approximation Pyramid (revisited)
scale1/j
30
Prediction Residual Pyramid (revisited)

• Prediction residual pyramid can be represented
  more
• efficiently.
• In the absence of quantization errors, the
  approximation
• pyramid can be reconstructed from the
  prediction residual
• pyramid.

31
Efficient Representation Using Details
details D3
details D2
details D1
L0
32
Efficient Representation Using Details (contd)
representation L0 D1 D2 D3
in general L0 D1 D2 D3DJ
(analysis)

• A wavelet representation of a function consists
  of
• a coarse overall approximation
• detail coefficients that influence the function
  at various scales.

33
Reconstruction (synthesis)
H3L2D3
H2L1D2
details D3
details D2
H1L0D1
details D1
L0
34
Example - Haar Wavelets

• Suppose we are given a 1D "image" with a
  resolution of 4 pixels
• 9 7 3 5
• The Haar wavelet transform is the following

L0 D1 D2 D3
35
Example - Haar Wavelets (contd)

• Start by averaging the pixels together (pairwise)
  to get a new lower resolution image
• To recover the original four pixels from the two
  averaged pixels, store some detail coefficients.

36
Example - Haar Wavelets (contd)

• Repeating this process on the averages gives the
  full decomposition

37
Example - Haar Wavelets (contd)

• The Harr decomposition of the original four-pixel
  image is
• We can reconstruct the original image to a
  resolution by adding or subtracting the detail
  coefficients from the lower-resolution versions.

1 -1
2
38
Example - Haar Wavelets (contd)
Note small magnitude detail coefficients!
Dj
Dj-1
How to compute Di ?
D1
L0
39
Multiresolution Conditions

• If a set of functions can be represented by a
  weighted sum of ?(2jt - k), then a larger set,
  including the original, can be represented by a
  weighted sum of ?(2j1t - k)

high resolution
j
scale/frequency localization
low resolution
time localization
40
Multiresolution Conditions (contd)

• If a set of functions can be represented by a
  weighted sum of ?(2jt - k), then a larger set,
  including the original, can be represented by a
  weighted sum of ?(2j1t - k)

Vj span of ?(2jt - k)
Vj1 span of ?(2j1t - k)
41
Nested Spaces Vj
Vj space spanned by ?(2jt - k)
Basis functions
?(t - k)
V0
f(t) ? Vj
?(2t - k)
V1

Vj
?(2jt - k)
Multiresolution conditions ? nested spanned
spaces
i.e., if f(t) ? V j then f(t) ? V j1
42
How to compute Di ?
f(t) ? Vj
IDEA define a set of basis Functions that span
the differences between Vj
43
Orthogonal Complement Wj

• Let Wj be the orthogonal complement of Vj in
  Vj1
• - i.e., all functions in Vj that are
  orthogonal to Wj

Vj1 Vj Wj
44
How to compute Di ? (contd)

• If f(t) ? Vj1, then f(t) can be represented
  using basis functions f(t) fromVj1

Vj1
Alternatively, f(t) can be represented using two
basis functions, f(t) from Vj and ?(t) from Wj
Vj1 Vj Wj
45
How to compute Di ? (contd)
Think of Wj as a means to represent the parts of
a function in Vj1 that cannot be represented in
Vj
differences between Vj and Vj1
Vj, Wj
46
How to compute Di ? (contd)

• ? using recursion
  on Vj

Vj1 Vj Wj
Vj1 Vj-1Wj-1Wj V0 W0 W1 W2
Wj
if f(t) ? Vj1 , then
V0 W0, W1, W2, basis
functions basis functions
47
Wavelet expansion (Section 7.2)

• Efficient wavelet decompositions involves a pair
  of waveforms (mother wavelets)
• The two shapes are translated and scaled to
  produce wavelets (wavelet basis) at different
  locations and on different scales.

encode low resolution info
encode details or high resolution info
f(t) ?(t)
f(t-k) ?(2jt-k)
48
Wavelet expansion (contd)

• f(t) is written as a linear combination of
  f(t-k) and ?(2jt-k)

scaling function wavelet function
Note in Fourier analysis, there are only two
possible values of k ( i.e., 0 and p/2) the
values j correspond to different scales (i.e.,
frequencies).
49
1D Haar Wavelets

• Haar scaling and wavelet functions

f(t) ?(t)
computes details
computes average
50
1D Haar Wavelets (contd)

• Think of a one-pixel image as a function that is
  constant over 0,1)
• We will denote by V0 the space of all such
  functions.

Example
0 1
51
1D Haar Wavelets

• Think of a two-pixel image as a function having
  two constant pieces over the intervals 0, 1/2)
  and 1/2,1)
• We will denote by V1 the space of all such
  functions.
• Note that

Examples
0 ½ 1


52
1D Haar Wavelets (contd)

• V j represents all the 2j-pixel images
• Functions having constant pieces over 2j
  equal-sized intervals on 0,1).
• Note that

width 1/2j
Examples
? Vj
? Vj
53
1D Haar Wavelets (contd)
V0, V1, ..., V j are nested
i.e.,
VJ V2 V1
fine details
coarse details
54
1D Haar Wavelets (contd)

• Mother scaling function
• Lets define a basis for V j

1
0 1
note alternative notation
55
1D Haar Wavelets (contd)
56
1D Haar Wavelets (contd)

• Suppose Wj is the orthogonal complement of Vj in
  Vj1
• i.e., all functions in Vj which are orthogonal to
  Wj

57
1D Haar Wavelets (contd)

• Mother wavelet function
• Note that f(x) . ?(x) 0 (i.e., orthogonal)

1
-1
0 1/2
1
1
1
0
.
0 1
-1
0 1/2
1
58
1D Haar Wavelets (contd)

• Mother wavelet function
• Lets define a basis ? ji for Wj

1
-1
0
1
note alternative notation
59
1D Haar Wavelets (contd)
basis for V 1
Note that inner product is zero!
basis W 1
j1
60
1D Haar Wavelets (contd)

• Basis functions ? ji of W j
• Basis functions f ji of V j

form a basis in V j1
61
1D Haar Wavelets (contd)
V3 V2 W2
62
1D Haar Wavelets (contd)
V2 V1 W1
63
1D Haar Wavelets (contd)
V1 V0 W0
64
1D Haar Wavelets (contd)
f(t)
?(t)
65
Example - Haar basis (revisited)
66
Decomposition of f(x)
f(x)
V2
67
Decomposition of f(x) (contd)
V1and W1
V2V1W1
68
Example - Haar basis (revisited)
69
Decomposition of f(x) (contd)
V0 ,W0 and W1
V2V1W1V0W0W1
70
Example - Haar basis (revisited)
71
Example
72
Example (contd)
73
Filter banks (analysis)

• The lower resolution coefficients can be
  calculated from the higher resolution
  coefficients by a tree-structured algorithm
  (filter bank).

Example
?(2t - k)
a1k (j1)
?(t - k)
a0k (j0)
74
Filter banks (analysis) (contd)

• The lower resolution coefficients can be
  calculated from the higher resolution
  coefficients by a tree-structured algorithm
  (filter bank).

Subband encoding!
h0(-n) is a lowpass filter and h1(-n) is a
highpass filter
75
Example - Haar basis (revisited)
9 7 3 5
low-pass, down-sampling
high-pass, down-sampling
(97)/2 (35)/2 (9-7)/2 (3-5)/2
V1 basis functions
76
Filter banks (analysis) (contd)
77
Example - Haar basis (revisited)
9 7 3 5
high-pass, down-sampling
low-pass, down-sampling
V1 basis functions
(84)/2 (8-4)/2
78
Convention for illustrating 1D Haar wavelet
decomposition
x x x x x x x
x
average
detail

re-arrange
V1 basis functions
re-arrange
79
Convention for illustrating1D Haar wavelet
decomposition (contd)
average
x x x x x x x
x
detail

80
Orthogonality and normalization

• The Haar basis forms an orthogonal basis
• It can become orthonormal through the following
  normalization

since
81
Examples of lowpass/highpass analysis filters
h0
Haar
h1
h0
Daubechies
h1
82
Filter banks (synthesis)

• The higher resolution coefficients can be
  calculated from
• the lower resolution coefficients using a
  similar structure.

83
Filter banks (synthesis) (contd)
84
Examples of lowpass/highpass synthesis filters
g0
Haar (same as for analysis)
g1
g0
Daubechies
g1

85
2D Haar Wavelet Transform

• The 2D Haar wavelet decomposition can be computed
  using 1D Haar wavelet decompositions (i.e., 2D
  Haar wavelet basis is separable).
• Two decompositions
• Standard decomposition
• Non-standard decomposition
• Each decomposition corresponds to a different set
  of 2D basis functions.

86
Standard Haar wavelet decomposition

• Steps
• (1) Compute 1D Haar wavelet decomposition of
  each row of the original pixel values.
• (2) Compute 1D Haar wavelet decomposition of
  each column of the row-transformed pixels.

87
Standard Haar wavelet decomposition (contd)
average
(1) row-wise Haar decomposition
detail
re-arrange terms


x x x x x x x
x . x x x ...
x

.
.

88
Standard Haar wavelet decomposition (contd)
average
(1) row-wise Haar decomposition
detail
row-transformed result
from previous slide




.
.

89
Standard Haar wavelet decomposition (contd)
average
detail
(2) column-wise Haar decomposition
row-transformed result
column-transformed result





.
.


90
Example
91
Example (contd)
column-transformed result


.

92
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis,
consider all possible outer products of 1D basis
functions.
Example
V2V0W0W1
93
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis,
consider all possible outer products of 1D basis
functions.
f00(x), f00(x)
?00(x), f00(x)
?01(x), f00(x)
94
2D Haar basis of standard decomposition
V2
95
Non-standard Haar wavelet decomposition

• Alternates between operations on rows and
  columns.
• (1) Perform one level decomposition in each row
  (i.e., one step of horizontal pairwise averaging
  and differencing).
• (2) Perform one level decomposition in each
  column from step 1 (i.e., one step of vertical
  pairwise averaging and differencing).
• (3) Repeat the process on the quadrant
  containing averages only (i.e., in both
  directions).

96
Non-standard Haar wavelet decomposition (contd)
one level, horizontal Haar decomposition
one level, vertical Haar decomposition


x x x x x x x
x . x x x . . .
x


.
.


Note averaging/differencing of detail
coefficients shown
97
Non-standard Haar wavelet decomposition (contd)
re-arrange terms

one level, vertical Haar decomposition on
green quadrant

one level, horizontal Haar decomposition on
green quadrant
.





.

98
Example
re-arrange terms


.


99
Example (contd)


.

100
2D Haar basis for non-standard decomposition

• Defined through 2D scaling and wavelet functions

101
2D Haar basis for non-standard decomposition
(contd)

• Three sets of detail coefficients (i.e., subband
  encoding)

LL
LL average
Detail coefficients
LH intensity variations along columns
(horizontal edges)
HL intensity variations along rows
(vertical edges)
HH intensity variations along
diagonals
102
2D Haar basis for non-standard decomposition
(contd)
V2
103
Forward/Inverse DWT(using textbooks notation)
iH,V,D H? LH V? HL D? HH
LL
104
2D DWT using filter banks (analysis)
LH
HH
H? LH V? HL D? HH
LL
HL
105
Illustrating 2D wavelet decomposition
HH
LH
LL
LH
HH
HL
LL
HL
The wavelet transform can be applied again on the
lowpass-lowpass version of the image, yielding
seven subimages.
106
2D IDWT using filter banks (synthesis)
H? LH V? HL D? HH
107
Applications

• Noise filtering
• Image compression
• Fingerprint compression
• Image fusion
• Recognition
• G. Bebis, A. Gyaourova, S. Singh, and I.
  Pavlidis, "Face Recognition by Fusing Thermal
  Infrared and Visible Imagery", Image and Vision
  Computing, vol. 24, no. 7, pp. 727-742, 2006.
• Image matching and retrieval
• Charles E. Jacobs Adam Finkelstein
  David H. Salesin, "Fast Multiresolution Image
  Querying", SIGRAPH, 1995.

108
Image Querying
query by content or query by
example Typically, the K best matches are
reported.
109
Fast Multiresolution Image Querying
painted
low resolution
target
queries