Image Enhancement

1
Image Enhancement

• Introduction
• Linear method
• Non-iterative techniques
• Inverse filtering and Wiener filtering
• Iterative techniques
• Landweber algorithm
• Nonlinear method
• Spatial domain techniques
• Point operations
• Histogram equalization
• Frequency domain techniques
• Unsharp masking
• Homomorphic filtering

2
Introduction

• What is image enhancement?
• A process of enhancing the visual quality of
  images due to nonideal image acquisition process
  (e.g., motion blurring, out-of-focus, poor
  illumination, coarse quantization etc.)
• Image visual quality assessment
• Objective quality metrics (e.g., MSE) might not
  always match subjective quality scores
• Human vision system (HVS) is the ultimate JUDGE.

3
A Plague in Image Processing Blur

• Where does blur come from?
• Optical blur camera is out-of-focus
• Motion blur camera or object is moving
• Why do we need deblurring?
• Visually annoying
• Wrong target for compression
• Bad for analysis
• Numerous applications in astronomical imaging,
  biomedical imaging, biometrics ...

4
Application (I) Astronomical Imaging

• The Story of Hubble Space Telescope (HST)
• HST Cost at Launch (1990) 1.5 billion
• Main mirror imperfections due to human errors
• Got repaired in 1993

5
Restoration of HST Images
6
Another Example
7
The Real (Optical) Solution
Before the repair
After the repair
8
Application (II) Medical Image Deblurring
(Deconvolution)
9
Application (III) Law Enforcement
Motion-blurred license plate image
10
Restoration Example
11
A Grand Challenge in Iris Recognition
out-of-focus iris image
12
Modeling Blurring Process
Linear degradation model
y(m,n)
x(m,n)
h(m,n)
blurring filter
additive white Gaussian noise
13
Blurring Filter Example
FT
Gaussian filter can be used to approximate
out-of-focus blur
14
Blurring Filter Example (Cont)
FT
MATLAB code hFSPECIAL('motion',9,30)
Motion blurring can be approximated by 1D
low-pass filter along the moving direction
15
The Curse of Noise
z(m,n)
y(m,n)
x(m,n)
h(m,n)
16
Image Example
BSNR10dB
x(m,n)
BSNR40dB
h(m,n) 1D horizontal motion blurring 1 1 1 1 1
1 1/7
17
Blind vs. Nonblind Deblurring

• Blind deblurring (deconvolution) blurring kernel
  h(m,n) is unknown
• Nonblind deconvolution
• blurring kernel h(m,n) is known
• In this course, we only cover the nonblind case
  (the easier case)

18
Non-iterative Solution (I) Inverse Filter
x(m,n)
h(m,n)
hI(m,n)
y(m,n)
blurring filter
inverse filter
hcombi (m,n)
To compensate the blurring, we require
19
Inverse Filtering (Cont)
x(m,n)
y(m,n)
h(m,n)

hI(m,n)
x(m,n)
inverse filter
Spatial
Frequency
amplified noise
20
Image Example
motion blurred image at BSNR of 40dB
deblurred image after inverse filtering
Q Why does the amplified noise look so bad? A
zeros in H(w1,w2) correspond to poles in HI
(w1,w2)
21
Pseudo-inverse Filter
Basic idea
To handle zeros in H(w1,w2), we treat them
separately when performing the inverse filtering
22
Image Example
motion blurred image at BSNR of 40dB
deblurred image after Pseudo-inverse
filtering (?0.1)
23
Non-iterative Solution (II) Wiener Filtering
Also called Minimum Mean Square Error (MMSE) or
Least-Square (LS) filtering
constant
noise energy
Example choice of K
signal energy
K0 ? inverse filtering
24
Image Example
motion blurred image at BSNR of 40dB
deblurred image after wiener filtering (K0.01)
25
Image Example (Cont)
K0.01
K0.001
K0.1
26
Constrained Least Square Filtering
Similar to Wiener but a different way of
balancing the tradeoff between
Example choice of C
Laplacian operator
?0 ? inverse filtering
27
Image Example
? 0.001
? 0.01
?0.1
28
Method of Successive Substitution

• A powerful technique for finding the roots of any
  function f(x)
• Basic idea
• Rewrite f(x)0 into an equivalent equation xg(x)
  (x is called fixed point of g(x))
• Successive substitution xi1g(xi)
• Under certain condition, the iteration will
  converge to the desired solution

29
Numerical Example
Two roots
successive substitution
30
Numerical Example (Cont)
Note that iteration quickly converges to x1
31
Landweber Iterative Deblurring
Linear blurring
We want to find the root of
relaxation parameter controls convergence
property
Successive substitution
32
Asymptotic Analysis
Assume convergence condition
we have
inverse filtering
33
Advantages of Landweber Iteration

• No inverse operation (e.g., division) is involved
• We can stop the iteration in the middle way to
  avoid noise amplification
• It facilitates the incorporation of a priori
  knowledge about the signal (X) into solution
  algorithm

More detailed analysis is included in EE565
Advanced Image Processing
34
Image Enhancement

• Introduction
• Linear method
• Non-iterative techniques
• Inverse filtering and Wiener filtering
• Iterative techniques
• Landweber algorithm
• Nonlinear method
• Spatial domain techniques
• Point operations
• Histogram equalization
• Frequency domain techniques
• Unsharp masking
• Homomorphic filtering

35
Why do We Need Nonlinear Method?

• Modeling image degradation process by a linear
  system is appealing mainly due to its
  mathematical tractability
• Numerous phenomenon in physical imaging and
  visualization can not be described by simple
  linear equations
• Examples relationship between illumination and
  luminance on a complex surface, quantization of
  intensity values, Gamma-correction in display
  devices

36
Point Operations Overview
Point operations are zero-memory operations
where a given gray level x?0,L is mapped to
another gray level y?0,L according to a
transformation
y
L
x
L
L255 for grayscale images
37
Lazy Man Operation
y
L
x
L
No influence on visual quality at all
38
Digital Negative
L
x
L
0
39
Contrast Stretching
yb
ya
x
a
b
L
0
40
Clipping
x
a
b
L
0
41
Range Compression
x
L
0
c100
42
Summary of Point Operation

• So far, we have discussed various forms of
  mapping function f(x) that leads to different
  enhancement results
• MATLAB function gtimadjust
• The natural question is How to select an
  appropriate f(x) for an arbitrary image?
• One systematic solution is based on the histogram
  information of an image
• Histogram equalization and specification

43
Histogram based Enhancement
Histogram of an image represents the relative
frequency of occurrence of various gray levels
in the image
MATLAB function gtimhist(x)
44
Why Histogram?
It is a baby in the cradle!
Histogram information reveals that image is
under-exposed
45
Another Example
Over-exposed image
46
How to Adjust the Image?

• Histogram equalization
• Basic idea find a map f(x) such that the
  histogram of the modified (equalized) image is
  flat (uniform).
• Key motivation cumulative probability function
  (cdf) of a random variable approximates a uniform
  distribution

Suppose h(t) is the histogram (pdf)
47
Histogram Equalization
Uniform Quantization
Note
y
cumulative probability function
1
L
x
L
0
48
MATLAB Implementation
function yhist_eq(x) M,Nsize(x) for i1256
h(i)sum(sum(x i-1)) End yxssum(h) for
i1256 Ifind(x i-1)
y(I)sum(h(1i))/s255 end
Calculate the histogram of the input image
Perform histogram equalization
49
Image Example
after
before
50
Histogram Comparison
after equalization
before equalization
51
Application (I) Digital Photography
52
Application (II) Iris Recognition
after
before
53
Application (III) Microarray Techniques
after
before
54
Frequency-Domain Techniques (I) Unsharp Masking
g(m,n) is a high-pass filtered version of x(m,n)
Example (Laplacian operator)
55
MATLAB Implementation
Implementation of Unsharp masking function
yunsharp_masking(x,lambda) Laplacian
operation h0 -1 0-1 4 -10 -1
0/4 dxfilter2(h,x) yxlambdadx
56
1D Example
xlp(n)
x(n)
g(n)x(n)-xlp(n)
57
2D Example
MATLAB command
gtroidemo
58
Frequency-Domain Techniques (II) Homomorphic
filtering
Basic idea
Illumination (low freq.)
reflectance (high freq.)
freq. domain enhancement
59
Image Example
after
before
60
Summary of Nonlinear Image Enhancement

• Understand how image degradation occurs first
• Play detective look at histogram distribution,
  noise statistics, frequency-domain coefficients
• Model image degradation mathematically and try
  inverse-engineering
• Visual quality is often the simplest way of
  evaluating the effectiveness, but it will be more
  desirable to measure the performance at a system
  level
• Iris recognition ROC curve of overall system
• Microarray ground-truth of microarray image
  segmentation result provided by biologists