Edge Preserving Smoothing Techniques

1
Edge Preserving Smoothing Techniques

• Prof. PhD. Vasile Gui
• Polytechnic University of Timisoara

2
Content

• Introduction
• Brief review of linear operators
• Linear image smoothing techniques
• Nonlinear image smoothing techniques

3
Introduction

• Why do we need image smoothing?
• What is image and what is noise?
• Frequency spectrum
• Statistical properties

4
Brief Review of Linear OperatorsPratt 1991

• Generalized 2D linear operator
• Separable linear operator
• Space invariant operator
• Convolution sum

5
Brief Review of Linear Operators

• Geometrical interpretation of 2D convolution

6
Brief Review of Linear Operators

• Circular convolution theorem
• Periodicity of f,h and g with N are assumed

7
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator
8
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Separable
• Can be computed recursively, resulting in roughly
  4 operations per pixel

9
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Optimality properties
• Signal and additive white noise
• Noise variance is reduced N times

10
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Unknown constant signal plus noise
• Minimize MSE of the estimation g

11
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• i.i.d. Gaussian signal with unknown mean.
• Given the observed samples, maximize
• Optimal solution arithmetic mean

12
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Frequency response
• Non-monotonically decreasing with frequency

13
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• An example

14
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator

• Image smoothed with 3?3, 5?5, 9?9
• and 11 ?11 box filters

15
Linear Image smoothing techniques Box filters.
Arithmetic mean L?L operator
Lena image filtered with 5x5 box filter
Original Lena image
16
Linear Image smoothing techniques Binomial
filters Jahne 1995

• Computes a weighted average of pixels in the
  window
• Less blurring, less noise cleaning for the same
  size
• The family of binomial filters can be defined
  recursively
• The coefficients can be found from (1x)n

17
Linear Image smoothing techniques Binomial
filters. 1D versions
As size increases, the shape of the filter is
closer to a Gaussian one
18
Linear Image smoothing techniques Binomial
filters. 2D versions
19
Linear Image smoothing techniques Binomial
filters. Frequency response
Monotonically decreasing with frequency
20
Linear Image smoothing techniques Binomial
filters. Example
Original Lena image
Lena image filtered with binomial 5x5 kernel
Lena image filtered with box filter 5x5
21
Linear Image smoothing techniques Binomial and
box filters. Edge blurring comparison

• Linear filters have to compromise smoothing with
  edge blurring

22
Nonlinear image smoothingThe median filter
Pratt 1991

• Block diagram

N is odd
23
Nonlinear image smoothingThe median filter

• Numerical example

24
Nonlinear image smoothingThe median filter

• Nonlinearity

median f1 f2 ? median f1 median
f2. However median c f c median f
, median c f c median f .

• The filter selects a sample from the window,
  does not average
• Edges are better preserved than with liner
  filters
• Best suited for salt and pepper noise

25
Nonlinear image smoothingThe median filter
Noisy image
5x5 median filtered
5x5 box filter
26
Nonlinear image smoothingThe median filter

• Optimality
• Grey level plateau plus noise. Minimize sum of
  absolute differences
• Result
• If 51 of samples are correct and 49 outliers,
  the median still finds the right level!

27
Nonlinear image smoothingThe median filter

• Caution points, thin lines and corners are
  erased by the median filter

28
Nonlinear image smoothingThe median filter

• Cross shaped window can correct some of the
  problems

29
Nonlinear image smoothingThe median filter

• Implementing the median filter
• Sorting needs O(N2) comparisons
• Bubble sort
• Quick sort
• Huang algorithm (based on histogram)
• VLSI median

30
Nonlinear image smoothingThe median filter

• VLSI median block diagram

31
Nonlinear image smoothingThe median filter

• Color median filter
• There is no natural ordering in 3D (RGB) color
  space
• Separate filtering on R,G and B components does
  not guarantee that the median selects a true
  sample from the input window
• Vector median filter, defined as the sample
  minimizing the sum of absolute deviations from
  all the samples
• Computing the vector median is very time
  consuming, although several fast algorithms exist
  

32
Nonlinear image smoothingThe median filter

• Example of color median filtering
• 5x5 pixels window
• Up original image
• Down filtered image

33
Nonlinear image smoothingThe weighted median
filter

• The basic idea is to give higher weight to some
  samples, according to their position with respect
  to the center of the window
• Each sample is given a weight according to its
  spatial position in the window.
• Weights are defined by a weighting mask
• Weighted samples are ordered as usually
• The weighted median is the sample in the ordered
  array such that neither all smaller samples nor
  all higher samples can cumulate more than 50 of
  weights.
• If weights are integers, they specify how many
  times a sample is replicated in the ordered array

34
Nonlinear image smoothingThe weighted median
filter

• Numerical example for the weighted median filter

35
Nonlinear image smoothingRank-order filters (L
filters)

• Block diagram

36
Nonlinear image smoothingRank-order filters (L
filters)

• Some examples

Percentile filters (rank selection) 0, 50 100
Particular cases of grey level morphological
filters
37
Nonlinear image smoothingRank-order filters (L
filters)

• Optimality

Minkovski distance
Case r 1 best estimator is median. Case r 2,
best estimator is arithmetic mean. Case r ? ?,
best estimator is mid-range.
38
Nonlinear image smoothingRank-order filters (L
filters)

• Alpha-trimmed mean filter

• ? (2Q 1) / N, defines de degree of averaging
• 1 corresponds to arithmetic mean
• ? 1 / N corresponds to the median filter

Properties in between mean and median
39
Nonlinear image smoothingRank-order filters (L
filters)

• Median of absolute differences trimmed mean
• Better smoothing than the median filter and good
  edge preservation

M2 is a robust estimator of the variance
40
Nonlinear image smoothingRank-order filters (L
filters)

• k nearest neighbour (kNN) median filter
• Median of the k grey values nearest by rank to
  the central pixel
• Aim same as above

41
Nonlinear image smoothingSelected area filtering
Nagao 1980

• Kuwahara type filtering
• Form regions Ri
• Compute mean, mi and variance, Si for each
  region.
• Result mi mi lt mj for all j different from i,
  i,e. the mean of the most homogeneous region
• Matsujama Nagao

42
Nonlinear image smoothingConditional mean

• Pixels in a neighbourhood are averaged only if
  they differ from the central pixel by less than a
  given threshold

L is a space scale parameter and th is a range
scale parameter
43
Nonlinear image smoothingConditional mean

• Example with L3, th32

44
Nonlinear image smoothingBilateral filter
Tomasi 1998

• Space and range are treated in a similar way
• Space and range similarity is required for the
  averaged pixels
• Tomasi and Manduchi 1998 introduced soft
  weights to penalize the space and range
  dissimilarity.

s() and r() are space and range similarity
functions (Gaussian functions of the Euclidian
distance between their arguments).
45
Nonlinear image smoothingBilateral filter

• The filter can be seen as weighted averaging in
  the joint space-range space (3D for monochromatic
  images and 5D x,y,R,G,B - for colour images)
• The vector components are supposed to be properly
  normalized (divide by variance for example)
• The weights are given by

46
Nonlinear image smoothingBilateral filter

• Example of Bilateral filtering
• Low contrast texture has been removed
• Yet edges are well preserved

47
Nonlinear image smoothingMean shift filtering
Comaniciu 1999, 2002

• Mean shift filtering replaces each pixels value
  with the most probable local value, found by a
  nonparametric probability density estimation
  method.
• The multivariate kernel density estimate obtained
  in the point x with the kernel K(x) and window
  radius r is
• For the Epanechnikov kernel, the estimated
  normalized density gradient is proportional to
  the mean shift

S is a sphere of radius r, centered on x and nx
is the number of samples inside the sphere
48
Nonlinear image smoothingMean shift filtering

• The mean shift procedure is a gradient ascent
  method to find local modes (maxima) of the
  probability density and is guaranteed to
  converge.
• Step1 computation of the mean shift vector
  Mr(x).
• Step2 translation of the window Sr(x) by Mr(x).
• Iterations start for each pixel (5D point) and
  tipically converge in 2-3 steps.

49
Nonlinear image smoothingMean shift filtering

• Example1.

50
Nonlinear image smoothingMean shift filtering

• Detail of a 24x40
• window from the
• cameraman image
• a) Original data
• b) Mean shift paths for some points
• c) Filtered data
• d) Segmented data

51
Nonlinear image smoothingMean shift filtering

• Example 2

52
Nonlinear image smoothingMean shift filtering

• Comparison to bilateral filtering
• Both methods based on simultaneous processing of
  both the spatial and range domains
• While the bilateral filtering uses a static
  window, the mean shift window is dynamic, moving
  in the direction of the maximum increase of the
  density gradient.

53
REFERENCES

• D. Comaniciu, P. Meer Mean shift analysis and
  applications. 7th International Conference on
  Computer Vision, Kerkyra, Greece, Sept. 1999,
  1197-1203.
• D. Comaniciu, P. Meer Mean shift a robust
  approach toward feature space analysis. IEEE
  Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18.
• M. Elad On the origin of bilateral filter and
  ways to improve it. IEEE trans. on Image
  Processing Vol. 11, No. 10, October 2002,
  1141-1151.
• B. Jahne Digital image processing. Springer
  Verlag, Berlin 1995.
• M. Nagao, T. Matsuiama A structural analysis of
  complex aerial photographs. Plenum Press, New
  York, 1980.
• W.K. Pratt Digital image processing. John Wiley
  and sons, New York 1991
• C. Tomasi, R. Manduchi Bilateral filtering for
  gray and color images. Proc. Sixth Intl. Conf.
  Computer Vision , Bombay, 839-846.