Discriminative Approach for Wavelet Denoising

1
Discriminative Approach forWavelet Denoising

• Yacov Hel-Or and Doron Shaked
• I.D.C.- Herzliya HPL-Israel

2
Motivation Image denoising
- Can we clean Lena?
3
Some reconstruction problems
Sapiro et. al.
Images of Venus taken by the Russian lander
Ventra-10 in 1975
- Can we see through the missing pixels?
4
Image Inpainting
Sapiro et.al.
5
Image De-mosaicing
- Can we reconstruct the color image?
6
Image De-blurring
Can we sharpen Barbara?
7

• Inpainting
• De-blurring
• De-noising
• De-mosaicing

• All the above deal with degraded images.
• Their reconstruction requires solving an
• inverse problem

8
Typical Degradation Sources
Low Illumination
Optical distortions (geometric, blurring)
Sensor distortion (quantization, sampling,
sensor noise, spectral sensitivity, de-mosaicing)
Atmospheric attenuation (haze, turbulence, )
9
Reconstruction as an Inverse Problem
noise
Original image
Distortion H
measurements
Years of extensive study Thousands of research
papers
10

• Typically
• The distortion H is singular or ill-posed.
• The noise n is unknown, only its statistical
  properties can be learnt.

11
Key point Stat. Prior of Natural Images
12
The Image Prior
Px(x)
1
Image space
0
13
Bayesian Reconstruction (MAP)

• From amongst all possible solutions, choose the
  one that maximizes the a-posteriori probability
  PX(xy)

PX(x)
measurements
P(xy)
Image space
14
So, are we set?

• Unfortunately not!
• The p.d.f. Px defines a prior dist. over natural
  images
• Defined over a huge dim. space (1E6 for 1Kx1K
  grayscale image)
• Sparsely sampled.
• Known to be non Gaussian.
• Complicated to model.

15
Example 3D prior of 2x2 image neighborhoods
form Mumford Huang, 2000
16
Marginalization of Image Prior

• Observation1 The Wavelet transform tends to
  de-correlate pixel dependencies of natural images.

W.T.
17
How Many Mapping Functions

• Observation2 The statistics of natural images
  are homogeneous.

Share the same statistics
18
Wavelet Shrinkage Denoising Donoho Johnston
94 (unitary case)

• Degradation Model
• The MAP estimator

19

• The MAP estimator gives

20

• The MAP estimator diagonalizes the system
• This leads to a very useful property
• Scalar mapping functions

21
Wavelet Shrinkage Pipe-line
Mapping functions Mi(yiw)
Transform W
Inverse Transform WT
xiw
yiw
Non linear operation
22
How Many Mapping Functions?

• Due to the fact that
• N mapping functions are needed for N sub-bands.

23
Subband Decomposition

• Wavelet transform
• Shrinkage

where
24
Wavelet Shrinkage Pipe-line
Shrinkage functions
Inverse transform
Wavelet transform
B1
B1
B1
B1

B1
B1
Bi
BTi
xiB
yiB
25
Designing The Mapping Function

• The shape of the mapping function Mj depends
  solely on Pj and the noise variance ?.

? (noise variance)

Modeling marginal p.d.f. of band j
MAP objective
yw
26

• Commonly Pj(yw) are approximated by GGD

for plt1
from Simoncelli 99
27
Hard Thresholding
Soft Thresholding
Linear Wiener Filtering
MAP estimators for GGD model with three different
exponents. The noise is additive Gaussian, with
variance one third that of the signal.
from Simoncelli 99
28

• Due to its simplicity Wavelet Shrinkage became
  extremely popular
• Thousands of applications.
• Hundreds of related papers (984 citations of DJ
  paper in Google Scholar).
• What about efficiency?
• Denoising performance of the original Wavelet
  Shrinkage technique is far from the
  state-of-the-art results.
• Why?
• Wavelet coefficients are not really independent.

29
Recent Developments

• Since the original approach suggested by DJ
  significant improvements were achieved

Original Shrinkage
Over-complete
Joint (Local) Coefficient Modeling

• Overcomplete transform
• Scalar MFs
• Simple
• Not considered state-of-the-art

• Multivariate MFs
• Complicated
• Superior results

30
Joint (Local) Coefficient Modeling
03
94
2000
06
97
HMM Crouse et. al.
Joint Bayesian Pizurika et. al
HMM Fan-Xia
Context Modeling Portilla et. al.
Sparsity Mallat, Zhang
Joint Bayesian Simoncelli
Context Modeling Chang, et. al.
Co-occurence Shan, Aviyente
Bivariate Sendur, Selesnick
Adaptive Thresh. Li, Orchad
31
Shrinkage in Over-complete Transforms
03
94
2000
06
97
Shrinkage D.J.
Ridgelets Candes
Ridgelets Carre, Helbert
Curvelets Starck et. al.
Ridgelets Nezamoddini et. al.
Undecimated wavelet Coifman, Donoho
Contourlets Matalon, et. al.
Steerable Simoncelli, Adelson
Contourlets Do, Vetterli
K-SVD Aharon, Elad
32
Over-Complete Shrinkage Denoising

• Over-complete transform
• Shrinkage
• Mapping Functions Naively borrowed from the
  Unitary case.

where
33
Whats wrong with existing MFs?

• Map criterion
• Solution is biased towards the most probable
  case.
• Independent assumption
• In the overcomplete case, the wavelet
  coefficients are inherently dependent.
• Minimization domain
• For the unitary case MFs optimality is expressed
  in the transform domain. This is incorrect in the
  overcomplete case.
• White noise assumption
• Image noise is not necessarily white i.i.d.

34
Why unitary based MFs are being used?

• Non-marginal statistics.
• Multivariate minimization.
• Multivariate MFs.
• Non-white noise.

Complicated
35
Suggested Approach

• Maintain simplicity
• Use scalar LUTs.
• Improve Efficiency
• Use Over-complete Transforms.
• Design optimal MFs with respect to a given set of
  images.
• Express optimality in the spatial domain.
• Attain optimality with respect to MSE.

36
Optimal Mapping Function

• Traditional approach Descriptive
• Suggested approach Discriminative

MAP objective
Modeling wavelet p.d.f.

Optimality criteria
37
The optimality Criteria

• Design the MFs with respect to a given set of
  examples xei and yei
• Critical problem How to optimize the non-linear
  MFs

38
The Spline Transform

• Let x?R be a real value in a bounded interval
  a,b).
• We divide a,b) into M segments qq0,q1,...,qM
• w.l.o.g. assume x?qj-1,qj)
• Define residue r(x)(x-qj-1)/(qj-qj-1)

a
b
x
xr(x) qj(1-r(x)) qj-1
x0,???,0,1-r(x),r(x),0,???q Sq(x)q
39
The Spline Transform-Cont.

• We define a vectorial extension
• We call this the
• Spline Transform (SLT) of x.

? ? ?
? ? ?
ith row
40
The SLT Properties

• Substitution property Substituting the boundary
  vector q with a different vector p forms a
  piecewise linear mapping.

x
x
p
x
Sq(x)
q
x
x
x
q2
q3
q4
q1
41
Back to the MFs Design

• We approximate the non-linear Mk with
• piece-wise linear functions
• Finding pk is a standard LS problem with a
  closed form solution!

42
Designing the MFs
B1
B1
B1
B1

B1
B1


Bk
BTk
Mk(y pk)
(BTB)-1
closed form solution
43
Results
44
Training Images



45
Tested Images


46
Simulation setup

• Transform used Undecimated DCT
• Noise Additive i.i.d. Gaussian
• Number of bins 15
• Number of bands 3x3 .. 10x10

47
Option 1 Transform domain independent bands
(BTB)-1
Mk(y pk)


(BTB)-1
48
Option 2 Spatial domain independent bands
(BTB)-1
Mk(y pk)


(BTB)-1
49
Option 3 Spatial domain joint bands
(BTB)-1
Mk(y pk)


(BTB)-1
50
Option 1
Option 2
Option 3
MFs for UDCT 8x8 (i,i) bands, i1..4, ?20
51
Comparing psnr results for 8x8 undecimated DCT,
sigma20.
52
8x8 UDCT ?10
53
8x8 UDCT ?20
54
8x8 UDCT ?10
55
The Role of Quantization Bins
8x8 UDCT ?10
56
The Role of Transform Used
?10
57
The Role of Training Image
58
The Role of noise variance
?5
?10
?15
?20
MFs for UDCT 8x8 (i,i) bands, i2..6.
59
The role of noise variance

• Observation The obtained MFs for different noise
  variances are similar up to scaling

60
Comparison between M20(v) and 0.5M10(2v) for
basis 24X24
61
Comparison with BLS-GSM
62
Comparison with BLS-GSM
63
Other Degradation Models
64
JPEG Artifact Removal
65
JPEG Artifact Removal
66
Image Sharpening
67
Image Sharpening
68
Conclusions

• New and simple scheme for over-complete transform
  based denoising.
• MFs are optimized in a discriminative manner.
• Linear formulation of non-linear minimization.
• Eliminating the need for modeling complex
  statistical prior in high-dim. space.
• Seamlessly applied to other degradation problems
  as long as scalar MFs are used for
  reconstruction.

69
Conclusions cont.

• Extensions
• Filter-cascade based denoising.
• Multivariate MFs (activity level).
• Non-homogeneous noise characteristics.
• Open problems
• What is the best transform for a given image?
• How to choose training images that form faithful
  representation?

70
Thank You
The End
71
MSE for MF scaling from ?10 to ?20
72
MSE for MF scaling from ?15 to ?20
73
MSE for MF scaling from ?25 to ?20
74
(No Transcript)
75
Image Sharpening
76
Wavelet Shrinkage Pipe-line
Shrinkage functions
Inverse transform
Wavelet transform
B1
B1
B1
B1
B1
B1

xiB
Bi
BTi
yiB
(BTB)-1
77
Option 1
MFs for UDCT 8x8 (i,i) bands, i1..4, ?20
78
Option 2
MFs for UDCT 8x8 (i,i) bands, i1..4, ?20
79
Option 3
MFs for UDCT 8x8 (i,i) bands, i1..4, ?20
80
Comparing psnr results for 8x8 undecimated DCT,
sigma20.
81
Comparing psnr results for 8x8 undecimated DCT,
sigma10.