Extensions of wavelets

1
Extensions of wavelets

• ECE 802

2
M-Band Wavelet Systems

• Generalization of dyadic wavelets
• Scale factor of M
• More flexible tiling of the time-frequency plane

3
Properties

• Scaling Equation
• Existence and Orthogonality
• M-1 wavelets

4
MRA

• At each scale j, there are M-1 wavelet functions
  and one scaling function
• If the wavelets are orthogonal to the scaling
  function at the same scale

5
Analysis and Synthesis

• The expansion is
• The filter bank structure will now have M
  branches
• Gives a mixture of a logarithmic and linear
  frequency resolution.
• Easier to design for M2k

6
Wavelet Packets

• M2 results in a logarithmic frequency
  resolution. The low frequencies have narrow
  bandwidths and the high frequencies have wide
  bandwidths.
• Wavelet packet system proposed by Coifman
• Adjustable resolution of frequencies at high
  frequencies
• Computational complexity O(NlogN)

7
Wavelet Packet Decomposition

• In order to have higher resolution decomposition
  at high frequencies, iterate the highpass wavelet
  branch
• Split both the lowpass and highpass bands at all
  stages
• Evenly spaced frequency resolution
• In DWT we consider the outputs of each channel.
• In WPD, we have more outputs than inputs?
  redundant system
• Choose an independent set as basis (not one
  unique basis)

8
Optimization Criteria

• Search based on minimizing a cost function on the
  transform coefficients.
• Binary search algorithm for additive cost
  function
• How do we choose the best basis?
• Shannon entropy
• Thresholding the coefficients
• Log Energy
• Norm of the coefficients
• Two approaches
• Choose a particular decomposition based on the
  signal class
• Adapt the decomposition to each signal

9
Complexity

• P(J) The number of J-scale orthonormal wavelet
  packet transforms
• P(1)1
• P(J)P(J-1)21
• Application FBI standard for fingerprint image
  compression

10
Haar Wavelet Packets
11
Wavelet Packet Tree
12
Optimization Functions

• Shannon Entropy
• Norm
• Log-Energy
• Threshold Entropy Number of times the
  coefficient is larger than a threshold

13
Example Minimum Entropy Decomposition

• Start with a constant original signal. w00
  ones(1,16)0.25
• Compute entropy of original signal.
• e00 wentropy(w00,'shannon') e00 2.7726
• Then split w00 using the haar wavelet.
• w10,w11 dwt(w00,'db1')
• Compute entropy of approximation at level 1.
• e10 wentropy(w10,'shannon') e10 2.0794
• The detail of level 1, w11, is zero the entropy
  e11 is zero. Due to the additivity property the
  entropy of decomposition is given by
  e10e112.0794. This has to be compared to the
  initial entropy e002.7726. We have e10 e11 lt
  e00, so the splitting is interesting.
• Now split w10 (not w11 because the splitting of a
  null vector is without interest since the entropy
  is zero).
• w20,w21 dwt(w10,'db1')
• We have w200.5ones(1,4) and w21 is zero. The
  entropy of the approximation level 2 is
• e20 wentropy(w20,'shannon') e20 1.3863
• Again we have e20 0 lt e10, so splitting makes
  the entropy decrease.
• Then
• w30,w31 dwt(w20,'db1') e30
  wentropy(w30,'shannon')     e30 0.6931
  w40,w41 dwt(w30,'db1')     w40 1.0000
      w41 0 e40 wentropy(w40,'shannon')     e40
  0
• Perform wavelet packets decomposition of the
  signal s.
• t wpdec(s,4,'haar','shannon')

14
Best Tree
15
Overcomplete Representations, Frames, Redundant
Transforms

• There are many cases where a single basis is not
  effective for signal representation.
• Example Fourier basis is good for sinusoids, but
  bad for transients
• Efficiency of the transform can be improved by
  combining several basis systems.
• Combination of basis systems?Overcomplete
• Collection of basis systems is called a
  dictionary.

16
Desired Criteria

• Sparsity Efficient representation
• Separation Better ability to separate a mixture
  of signals
• Superresolution Higher resolution or detail
  compared to a single basis
• Stability Robust under noise, the selected atoms
  do not change
• Speed

17
Definitions

• Frame Generalization of a basis, a collection of
  functions that span the vector space, but are not
  linearly independent
• The frame condition
• 0ltAltBlt8
• If AB, tight frame
• If AB1, orthonormal basis

18
Frame Examples

• Tight Frame 4 basis functions in 3-dimensional
  space

19
Matching Pursuit

• Matching pursuit (MP) algorithm finds a
  sub-optimal solution to the problem of an
  adaptive approximation of a signal in a redundant
  set (dictionary) of functions.
• Look for a linear expansion of a signal in terms
  of elements (atoms) of a dictionary.

20
Algorithm Mallat, Zhang 1993

• At each step, try to find the element of the
  dictionary that best fits the signal.
• Energy Conservation
• For a complete dictionary as M?8, the residue
  should go to zero.
• Stopping Criteria Threshold the residue or
  pre-determine M

21
Dictionary

• Commonly used dictionary Gabor functions,
  dictionary of time-frequency atoms
• General and compact model for oscillations
• Compact time-frequency localization
• Restrict the search to a range of time,
  frequency, and scale values

22
Applications

• EEG Spike Parametrization

23
Extensions

• Multichannel MP Jointly represent a class of
  signals using the same elements of the dictionary
• Orthogonal Matching Pursuit (OMP)
• Efficient greedy algorithm
• Applies Gram-Schmidt orthogonalization to the
  selected atoms before computing the residue
• The selected atoms are orthogonalized with
  respect to the residue
• Faster convergence than MP

24
Basis PursuitChen, Donoho

• Convex optimization Find the representation that
  minimizes the l1 norm of the coefficients
• Solved using linear programming
• Nearly linear time
• l1 norm guarantees sparsity, l2 norm does not
  (Method of Frames)

25
Examples

• FM-Cosine signal

26
Comparisons

• MP and OMP are iterative algorithms.
• MP starts from an empty signal model and builds
  it up one atom at a time
• BP starts from a full model and iteratively
  improves the full model.
• Wavelet Packet Decomposition (Best Orthogonal
  Basis) focuses only on the orthogonal bases.
• MOF l2 solution, not sparse, can be noisy.