Smoothing Techniques in Image Processing

1
Smoothing Techniques in Image Processing

• Prof. PhD. Vasile Gui
• Polytechnic University of Timisoara

2
Content

• Introduction
• Brief review of linear operators
• Linear image smoothing techniques
• Nonlinear image smoothing techniques

3
Introduction

• Why do we need image smoothing?
• What is image and what is noise?
• Frequency spectrum
• Statistical properties

4
Brief Review of Linear OperatorsPratt 1991

• Generalized 2D linear operator
• Separable linear operator
• Space invariant operator
• Convolution sum

5
Brief Review of Linear Operators

• Geometrical interpretation of 2D convolution

6
Brief Review of Linear Operators

• Matrix formulation
• Unitary transform
• Separable transform
• Transform is invertible

7
Brief Review of Linear Operators

• Basis vectors are orthogonal
• Inner product and energy are conserved
• Unitary transform a rotation in N-dimensional
  vector space

8
Brief Review of Linear Operators

• 2D vector space interpretation
• of a unitary transform

9
Brief Review of Linear Operators

• DFT unitary matrix

10
Brief Review of Linear Operators

• 1D DFT
• 2D DFT

11
Brief Review of Linear Operators

• Circular convolution theorem
• Periodicity of f,h and g with N are assumed

12
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator
13
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Separable
• Can be computed recursively, resulting in roughly
  4 operations per pixel

14
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Optimality properties
• Signal and additive white noise
• Noise variance is reduced N times

15
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Unknown constant signal plus noise
• Minimize MSE of the estimation g

16
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• i.i.d. Gaussian signal with unknown mean.
• Given the observed samples, maximize
• Optimal solution arithmetic mean

17
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Frequency response
• Non-monotonically decreasing with frequency

18
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• An example

19
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Image smoothed with 3?3, 5?5, 9?9
• and 11 ?11 box filters

20
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator
Lena image filtered with 5x5 box filter
Original Lena image
21
Linear Image smoothing techniques Binomial
filters Jahne 1995

• Computes a weighted average of pixels in the
  window
• Less blurring, less noise cleaning for the same
  size
• The family of binomial filters can be defined
  recursively
• The coefficients can be found from (1x)n

22
Linear Image smoothing techniques Binomial
filters. 1D versions
As size increases, the shape of the filter is
closer to a Gaussian one
23
Linear Image smoothing techniques Binomial
filters. 2D versions
24
Linear Image smoothing techniques Binomial
filters. Frequency response
Monotonically decreasing with frequency
25
Linear Image smoothing techniques Binomial
filters. Example
Original Lena image
Lena image filtered with binomial 5x5 kernel
Lena image filtered with box filter 5x5
26
Linear Image smoothing techniques Binomial and
box filters. Edge blurring comparison

• Linear filters have to compromise smoothing with
  edge blurring

27
Nonlinear image smoothingThe median filter
Pratt 1991

• Block diagram

N is odd
28
Nonlinear image smoothingThe median filter

• Numerical example

29
Nonlinear image smoothingThe median filter

• Nonlinearity

median f1 f2 ? median f1 median
f2. However median c f c median f
, median c f c median f .

• The filter selects a sample from the window,
  does not average
• Edges are better preserved than with liner
  filters
• Best suited for salt and pepper noise

30
Nonlinear image smoothingThe median filter
Noisy image
5x5 median filtered
5x5 box filter
31
Nonlinear image smoothingThe median filter

• Optimality
• Grey level plateau plus noise. Minimize sum of
  absolute differences
• Result
• If 51 of samples are correct and 49 outliers,
  the median still finds the right level!

32
Nonlinear image smoothingThe median filter

• Caution points, thin lines and corners are
  erased by the median filter

33
Nonlinear image smoothingThe median filter

• Cross shaped window can correct some of the
  problems

34
Nonlinear image smoothingThe median filter

• Implementing the median filter
• Sorting needs O(N2) comparisons
• Bubble sort
• Quick sort
• Huang algorithm (based on histogram)
• VLSI median

35
Nonlinear image smoothingThe median filter

• VLSI median block diagram

36
Nonlinear image smoothingThe median filter

• Color median filter
• There is no natural ordering in 3D (RGB) color
  space
• Separate filtering on R,G and B components does
  not guarantee that the median selects a true
  sample from the input window
• Vector median filter, defined as the sample
  minimizing the sum of absolute deviations from
  all the samples
• Computing the vector median is very time
  consuming, although several fast algorithms exist
  

37
Nonlinear image smoothingThe median filter

• Example of color median filtering
• 5x5 pixels window
• Up original image
• Down filtered image

38
Nonlinear image smoothingThe weighted median
filter

• The basic idea is to give higher weight to some
  samples, according to their position with respect
  to the center of the window
• Each sample is given a weight according to its
  spatial position in the window.
• Weights are defined by a weighting mask
• Weighted samples are ordered as usually
• The weighted median is the sample in the ordered
  array such that neither all smaller samples nor
  all higher samples can cumulate more than 50 of
  weights.
• If weights are integers, they specify how many
  times a sample is replicated in the ordered array

39
Nonlinear image smoothingThe weighted median
filter

• Numerical example for the weighted median filter

40
Nonlinear image smoothingThe multi-stage median
filter

• Better detail preservation

41
Nonlinear image smoothingRank-order filters (L
filters)

• Block diagram

42
Nonlinear image smoothingRank-order filters (L
filters)

• Some examples

Percentile filters (rank selection) 0, 50 100
Particular cases of grey level morphological
filters
43
Nonlinear image smoothingRank-order filters (L
filters)

• Optimality

Minkovski distance
Case r 1 best estimator is median. Case r 2,
best estimator is arithmetic mean. Case r ? ?,
best estimator is mid-range.
44
Nonlinear image smoothingRank-order filters (L
filters)

• Alpha-trimmed mean filter

• ? (2Q 1) / N, defines de degree of averaging
• 1 corresponds to arithmetic mean
• ? 1 / N corresponds to the median filter

Properties in between mean and median
45
Nonlinear image smoothingRank-order filters (L
filters)

• Median of absolute differences trimmed mean
• Better smoothing than the median filter and good
  edge preservation

M2 is a robust estimator of the variance
46
Nonlinear image smoothingRank-order filters (L
filters)

• k nearest neighbour (kNN) median filter
• Median of the k grey values nearest by rank to
  the central pixel
• Aim same as above

47
Nonlinear image smoothingSelected area filtering
Nagao 1980

• Kuwahara type filtering
• Form regions Ri
• Compute mean, mi and variance, Si for each
  region.
• Result mi mi lt mj for all j different from i,
  i,e. the mean of the most homogeneous region
• Matsujama Nagao

48
Nonlinear image smoothingConditional mean

• Pixels in a neighbourhood are averaged only if
  they differ from the central pixel by less than a
  given threshold

L is a space scale parameter and th is a range
scale parameter
49
Nonlinear image smoothingConditional mean

• Example with L3, th32

50
Nonlinear image smoothingBilateral filter
Tomasi 1998

• Space and range are treated in a similar way
• Space and range similarity is required for the
  averaged pixels
• Tomasi and Manduchi 1998 introduced soft
  weights to penalize the space and range
  dissimilarity.

s() and r() are space and range similarity
functions (Gaussian functions of the Euclidian
distance between their arguments).
51
Nonlinear image smoothingBilateral filter

• The filter can be seen as weighted averaging in
  the joint space-range space (3D for monochromatic
  images and 5D x,y,R,G,B - for colour images)
• The vector components are supposed to be properly
  normalized (divide by variance for example)
• The weights are given by

52
Nonlinear image smoothingBilateral filter

• Example of Bilateral filtering
• Low contrast texture has been removed
• Yet edges are well preserved

53
Nonlinear image smoothingMean shift filtering
Comaniciu 1999, 2002

• Mean shift filtering replaces each pixels value
  with the most probable local value, found by a
  nonparametric probability density estimation
  method.
• The multivariate kernel density estimate obtained
  in the point x with the kernel K(x) and window
  radius r is
• For the Epanechnikov kernel, the estimated
  normalized density gradient is proportional to
  the mean shift

S is a sphere of radius r, centered on x and nx
is the number of samples inside the sphere
54
Nonlinear image smoothingMean shift filtering

• The mean shift procedure is a gradient ascent
  method to find local modes (maxima) of the
  probability density and is guaranteed to
  converge.
• Step1 computation of the mean shift vector
  Mr(x).
• Step2 translation of the window Sr(x) by Mr(x).
• Iterations start for each pixel (5D point) and
  tipically converge in 2-3 steps.

55
Nonlinear image smoothingMean shift filtering

• Example1.

56
Nonlinear image smoothingMean shift filtering

• Detail of a 24x40
• window from the
• cameraman image
• a) Original data
• b) Mean shift paths for some points
• c) Filtered data
• d) Segmented data

57
Nonlinear image smoothingMean shift filtering

• Example 2

58
Nonlinear image smoothingMean shift filtering

• Comparison to bilateral filtering
• Both methods based on simultaneous processing of
  both the spatial and range domains
• While the bilateral filtering uses a static
  window, the mean shift window is dynamic, moving
  in the direction of the maximum increase of the
  density gradient.

59
REFERENCES

• D. Comaniciu, P. Meer Mean shift analysis and
  applications. 7th International Conference on
  Computer Vision, Kerkyra, Greece, Sept. 1999,
  1197-1203.
• D. Comaniciu, P. Meer Mean shift a robust
  approach toward feature space analysis. IEEE
  Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18.
• M. Elad On the origin of bilateral filter and
  ways to improve it. IEEE trans. on Image
  Processing Vol. 11, No. 10, October 2002,
  1141-1151.
• B. Jahne Digital image processing. Springer
  Verlag, Berlin 1995.
• M. Nagao, T. Matsuiama A structural analysis of
  complex aerial photographs. Plenum Press, New
  York, 1980.
• W.K. Pratt Digital image processing. John Wiley
  and sons, New York 1991
• C. Tomasi, R. Manduchi Bilateral filtering for
  gray and color images. Proc. Sixth Intl. Conf.
  Computer Vision , Bombay, 839-846.