Discriminative Approach for Transform Based Image Restoration

1
Discriminative Approach forTransform Based
Image Restoration
SIAM Imaging Science, July 2008

• Yacov Hel-Or Doron Shaked Gil
  Ben-Artzi

The Interdisciplinary Center Israel
Bar-Ilan Univ. Israel
HP Las
2
Motivation Image denoising
- Can we clean Lena?
3
Broader Scope

• Inpainting
• De-blurring
• De-noising
• De-mosaicing

• All the above deal with degraded images.
• Their reconstruction requires solving an
• inverse problem

4
Key point Stat. Prior of Natural Images
likelihood
prior
Bayesian estimation
5
Problem P(x) is complicated to model

• Defined over a huge dimensional space.
• Sparsely sampled.
• Known to be non Gaussian.

A prior p.d.f. of a 2x2 image patch
form Mumford Huang, 2000
6
The Wavelet Transform Marginalizes Image Prior

• Observation1 The Wavelet transform tends to
  de-correlate pixel dependencies of natural images.

W.T.
7

• Observation2 The statistics of natural images
  are homogeneous.

Share the same statistics
8
Donoho Johnston 94 Wavelet Shrinkage
Denoising Unitary Case

• Degradation Model
• MAP estimation in the transform domain

9

• The Wavelet domain diagonalizes the system.
• The estimation of a coefficient depends
  solely on its own measured value
• This leads to a very useful property
• Modify coefficients via scalar mapping
  functions

where Bk stands for the kth band
10
Shrinkage Pipe-line
BT1
BT1
B1
B3
y
B3
B1
BT2
BT2
B2
BT3
BT3
B2
y
?k(Bky)
BTk?k(Bky)
Bky
Image domain
Transform domain
Image domain
Result
11
Wavelet Shrinkage as aLocally Adaptive Patch
Based Method
KxK
12

• Can be viewed as shrinkage de-noising in a
  Unitary Transform (Windowed DCT).

KxK bands
WDCT
Unitary Transform
xiB
WDCT-1
yiB
13
Alternative Approach Sliding Window
KxK
14

• Can be viewed as shrinkage de-noising in a
  redundant transform (U.D. Windowed DCT).

UWDCT
Redundant Transform
xiB
UWDCT-1
yiB
15
How to Design the Mapping Functions?

• Descriptive approach The shape of the mapping
  function ?j depends solely on Pj and the noise
  variance ?.

noise variance (?)

Modeling marginal p.d.f. of band j
MAP objective
yw
16

• Commonly Pj(yB) are approximated by GGD

for plt1
from Simoncelli 99
17
Hard Thresholding
Soft Thresholding
Linear Wiener Filtering
MAP estimators for GGD model with three different
exponents. The noise is additive Gaussian, with
variance one third that of the signal.
from Simoncelli 99
18

• Due to its simplicity Wavelet Shrinkage became
  extremely popular
• Thousands of applications.
• Thousands of related papers
• What about efficiency?
• Denoising performance of the original Wavelet
  Shrinkage technique is far from the
  state-of-the-art results.
• Why?
• Wavelet coefficients are not really independent.

19
Recent Developments

• Since the original approach suggested by DJ
  significant improvements were achieved

Original Shrinkage
Redundant Representation
Joint (Local) Coefficient Modeling

• Overcomplete transform
• Scalar MFs
• Simple
• Not considered state-of-the-art

• Multivariate MFs
• Complicated
• Superior results

20
Whats wrong with existing redundant Shrinkage?

• Mapping functions
• Naively borrowed from the unitary case.
• Independence assumption
• In the overcomplete case, the wavelet
  coefficients are inherently dependent.
• Minimization domain
• For the unitary case MFs are optimized in the
  transform domain. This is incorrect in the
  overcomplete case (Parseval is not valid
  anymore).
• Unsubstantiated
• Improvements are shown empirically.

21
Questions we are going to address

• How to design optimal MFs for redundant bases.
• What is the role of redundancy.
• What is the role of the domain in which the MFs
  are optimized.
• We show that the shrinkage approach is comparable
  to state-of-the-art approaches where MFs are
  correctly designed.

22
Optimal Mapping Function

• Traditional approach Descriptive

Modeling marginal p.d.f. of band k
MAP objective
x
23
Optimal Mapping Function

• Suggested approach Discriminative
• Off line Design MFs with respect to a given set
  of examples xei and yei
• On line Apply the obtained MFs to new noisy
  signals.

Denoising Algorithm
24
Option 1 Transform domain independent bands




25
Option 2 Spatial domain independent bands




26
Option 3 Spatial domain joint bands




27
The Role of Optimization Domain

• Theorem 1 For unitary transforms and for any set
  of ?k
• Theorem 2 For over-complete (tight-frame) and
  for any set of ?k

28
Unitary v.s. OvercompleteSpatial v.s. Transform
Domain
Over-complete
Unitary
Spatial domain
gt

gt

Transform domain
29
Is it Justified to optimized in the transform
domain?

• In the transform domains we minimize an upper
  envelope.
• It is preferable to minimize in the spatial
  domain.

30
Optimal Design of Non-Linear MFs

• Problem How to optimize non-linear MFs ?
• Solution Span the non-linear ?k using a linear
  sum of basis functions.
• Finding ?k boils down to finding the span
  coefficients (closed form).

Mapping functions
?k(y)
y
For more details see Hel-Or Shaked IEEE-IP,
Feb 2008
31

• Let z?R be a real value in a bounded interval
  a,b).
• We divide a,b) into M segments qq0,q1,...,qM
• w.l.o.g. assume z?qj-1,qj)
• Define residue r(z)(z-qj-1)/(qj-qj-1)

a
b
z
z0,???,0,1-r(z),r(z),0,???q Sq(z)q
zr(z) qj(1-r(z)) qj-1
32
The Slice Transform

• We define a vectorial extension
• We call this the
• Slice Transform (SLT) of z.

? ? ?
? ? ?
ith row
33
The SLT Properties

• Substitution property Substituting the boundary
  vector q with a different vector p forms a
  piecewise linear mapping.

z
z
z
p
Sq(z)
q
z
z
z
q2
q3
q4
q1
34
Back to the MFs Design

• We approximate the non-linear ?k with
• piece-wise linear functions
• Finding pk is a standard LS problem with a
  closed form solution!

35
Results
36
Training Images



37
Tested Images


38
Simulation setup

• Transform used Undecimated DCT
• Noise Additive i.i.d. Gaussian
• Number of bins in SLT 15
• Number of bands 3x3 .. 10x10

39
MFs for UDCT 8x8 (i,i) bands, i1..4, ?20
Option 1
40
Why considering joint band dependencies produces
non-monotonic MFs ?
noisy image
Unitary MF
image space
Redundant MF
41
Comparing psnr results for 8x8 undecimated DCT,
sigma20.
42
8x8 UDCT ?10
43
8x8 UDCT ?20
44
8x8 UDCT ?10
45
Comparison with BLS-GSM
46
Comparison with BLS-GSM
47
Other Degradation Models
48
JPEG Artifact Removal
49
JPEG Artifact Removal
50
Image Sharpening
51
Image Sharpening
52
Conclusions

• New and simple scheme for over-complete transform
  based denoising.
• MFs are optimized in a discriminative manner.
• Linear formulation of non-linear minimization.
• Eliminating the need for modeling complex
  statistical prior in high-dim. space.
• Seamlessly applied to other degradation problems
  as long as scalar MFs are used for
  reconstruction.

53
Recent Results

• What is the best transform to be used (for a
  given image or for a given set)?

54
Thank You
The End