Digital Imaging 20062007

1
Digital Imaging2006/2007

• Lecture 2
• Image processing I

Ioannis Ivrissimtzis 05 - Mar - 2007
2
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

3
Spatial filters

• An common example of greyscale image processing
  through matrix
• manipulation is the convolution of a matrix by
  3x3 mask.

Matrix
Mask
4
Spatial filters

• The convolution processes each pixel of the image
  to create a new
• Image.

Centre
Original image
Processed image
5
Spatial Filters

• Why is it called spatial?
• It is performed directly on the pixels of the
  image, not on a
• transform of it.
• Why is it called linear?
• The mask operates on the image linearly, i.e.,
  component-wise multiplication and summation.
• Sometimes the mask is also called filter, kernel,
  template, window.

6
Spatial filters

• A simple example of convolution mask is the
  average filter
• Each pixel of the original image is replaced by
  the average of its
• neighbours.
• When applied to a greyscale image it smoothes it.

7
Example 1

• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 1 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0

.. .. .. .. .. .. .. .. 0
0 0 0 0 .. .. 0 1/9 1/9 1/9 0
.. .. 0 1/9 1/9 1/9 0 .. .. 0 1/9
1/9 1/9 0 .. .. 0 0 0 0 0
.. .. .. .. .. .. .. ..
Processed image
Original image
8
Example 2

• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 1 1 0 0 0
• 0 0 1 1 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0

.. .. .. .. .. .. .. .. 1/9
2/9 2/9 1/9 0 .. .. 2/9 4/9 4/9 2/9 0
.. .. 2/9 4/9 4/9 2/9 0 .. .. 1/9 2/9 2/9
1/9 0 .. .. 0 0 0 0 0 .. ..
.. .. .. .. .. ..
Processed image
Original image
9
Example 3
Original image
Filtered image
10
Example 4
Original image
10 iterations of filtering
Convolution is an operation that can be repeated
iteratively. Notice that near the boundary of
the filtered image became dark.
11
Boundary
What can we do near the boundary where the mask
does not fit into the image? ? ? ?
? 4 0 6 3 2 6 ? 8 3 7 2
7 3 5 4 7 5 3 5 5 3 6 8
7 3 9 3 0 7 7 3 4 1 4 6 0
9
12
Boundary
Possible solution expand it with zeros (default
in Matlab Image processing Tool). 0 0
0 0 0 0 0 0 0 4 0 6 3 2
6 0 0 8 3 7 2 7 3 0 0 5
4 7 5 3 5 0 0 5 3 6 8 7
3 0 0 9 3 0 7 7 3 0 0
4 1 4 6 0 9 0 0 0 0 0 0
0 0 0
13
Boundary

• The problem with this is that it creates a black
  border around the image.

14
Boundary
Another possible solution is to replicate the
outer pixels of the image 4 4 0 6
3 2 6 6 4 4 0 6 3 2 6 6
8 8 3 7 2 7 3 3 5 5 4 7
5 3 5 5 5 5 3 6 8 7 3
3 9 9 3 0 7 7 3 3 4 4 1
4 6 0 9 9 4 4 1 4 6 0 9
9
15
Boundary
Original image
10 iterations of filtering
The use of replication has removed the artifact
on the boundary.
16
Boundary

• Another possible solution is to replicate the
  whole image as a 2D
• periodic tile.

17
Boundary

• Another possible solution is to replicate the
  whole image as a 2D
• periodic tile.

18
Non-symmetric filters

• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 1 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0
• 0 0 0 0 0 0 0

.. .. .. .. .. .. .. .. 0
0 0 0 0 .. .. 0 9 8 7
0 .. .. 0 6 5 4 0 .. .. 0
3 2 1 0 .. .. 0 0 0 0
0 .. .. .. .. .. .. .. ..
Processed image
Original image
19
Non-symmetric filters

• We notice that in this example the mask is
  imprinted on the new image
• but it is by rotated 1800.
• In some application this might be a problem.
• In such cases, when we use a non-symmetric filter
  we first rotate it by
• 1800 and then convolve.
• Most filters are symmetric.

20
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

21
Standard linear filters

• The average filter we used in some of the
  previous examples is a blunt
• instrument for removing the noise from an image.
  It more blurs the
• image rather than smoothing it.
• The Gaussian filter
• is better in preserving the details in the
  smoothed image.

22
Gaussian filter

• The 5x5 mask of the Gaussian filter is

23
Example
Gaussian filter
24
Gaussian filter

• How are the filters coefficients computed?
• We can think of the images as discrete functions.
• We convolve one discrete function (the image)
  with another discrete
• function (the filter).

Gaussian function
25
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

26
Laplacian filter

• The mask of the Laplacian filter is

27
Laplacian filter

• The Laplacian filter makes the constant areas
  equal to zero.

Laplacian filter
28
Laplacian filter

• The Laplacian filter is used to sharpen the
  image.
• This is done by subtracting the filtered image
  from the original.

-

29
Laplacian filter

• How are the filters coefficients computed?
• First consider an 1D function instead of a 2D
  image
• The derivatives between a,b and b,c are
• (this is a discrete approximation of how fast the
  values of f change)

30
Laplacian filter

• Assuming a,b,c are uniformly placed (as the
  pixels of an image are) we
• can put
• a - b c - b 1
• and the derivatives become
• f(b) - f(a) and f(c) - f(b)

31
Laplacian filter

• Similarly, we approximate second derivatives as
  differences of the first
• derivarives
• ( f(c) - f(b) ) - ( f(b) - f(a) ) f(a) - 2f(b)
  f(c)

32
Laplacian filter

• Using the terminology of the filters
• f(a) - 2f(b) f(c)
• gives the mask 1 -2 1 which can
  be convolved with the 1D
• image f(a) f(b) f(c)

33
Laplacian filter

• In 2D images now, repeating this procedure in the
  horizontal and
• vertical direction we get the masks
• 0 0 0 0 1 0
• 1 -2 0 and 0 -2 0
• 0 0 0 0 1 0
• 0 1 0
• Adding the two masks we get the Laplacian filter
  1 -4 1
• 0 1 0

34
Laplacian filter

• In a mathematical notation, adding the second
  derivatives in the
• horizontal and vertical direction corresponds to
  the operator
• which is called the Laplace operator.
• The most commonly used Laplacian filter (shown on
  the first slide on
• Laplacian filters) is obtained from a different
  discretisation of the
• Laplacian.
• This discretisation takes into account the
  diagonal directions of the
• image as well.

35
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

36
Cropping

• A region of interest is a part of the image which
  we want to process
• separately.

37
Zoom 0-order

• We first describe a simple method for x2 zooming
  on an nxn image
• using spatial linear filters.
• The first step is to create a new (2n) x (2n)
  image such that
• The new image has the original image copied on
  its pixels with
• both coordinates even, and zeros everywhere else.
• 0 0 0 0 0 0 0 3 0 7
  0 8 0 0 0 0 0 0
• 3 7 8 0 7 0 5 0 4
• 7 5 4 0 0 0 0 0 0
• 4 3 1 0 4 0 3 0 1

38
Zoom 0-order

• The second step is to apply the 2x2 filter
  on the (2n) x (2n)
• image
• 0 0 0 0 0 0 1 1 7 7
  8 8 0 3 0 7 0 8 1 3 7
  7 8 8
• 0 0 0 0 0 0 7 7 5 5
  4 4
• 3 7 8 0 7 0 5 0 4 7
  7 5 5 4 4
• 7 5 4 0 0 0 0 0 0 4
  4 3 3 1 1
• 4 3 1 0 4 0 3 0 1 4
  4 3 3 1 1
• The pixels of the original image have been
  replicated into 2x2 blocks.
• This is called 0-order interpolation.

39
Zoom 0-order
The 0-order x2 zooming method described in the
previous slides.
40
Zoom 1-order

• Because of the pixel replication the previous
  method creates blocky
• images.
• To address this problem, instead of the 2x2
  filter
• we can apply the 3x3 filter
• This method is called 1-order or linear
  interpolation.

41
Zoom 1-order

• Linear interpolation also preserves the original
  pixels
• 0 0 0 0 0 0 (1/4)0 (1/2)0
  (1/4)0
• 0 3 0 7 0 8 (1/2)0 (1)7
  (1/2)0
• 0 0 0 0 0 0 (1/4)0 (1/2)0
  (1/4)0 7
• 0 7 0 5 0 4
• 0 0 0 0 0 0
• 0 4 0 3 0 1

42
Zoom 1-order

• Assuming zeros on the boundary, a new pixel
  between two old ones is
• the average of them
• 0 0 0 0 0 0 (1/4)0 (1/2)0
  (1/4)0
• 0 3 0 7 0 8 (1/2)7 (1)0
  (1/2)5
• 0 0 0 0 0 0 (1/4)0 (1/2)0
  (1/4)0 6
• 0 7 0 5 0 4
• 0 0 0 0 0 0
• 0 4 0 3 0 1

43
Zoom 1-order

• A new pixel between four old ones is again the
  average of them
• 0 0 0 0 0 0 (1/4)7 (1/2)0
  (1/4)5
• 0 3 0 7 0 8 (1/2)0 (1)0
  (1/2)0
• 0 0 0 0 0 0 (1/4)4 (1/2)0
  (1/4)3 19/4
• 0 7 0 5 0 4
• 0 0 0 0 0 0
• 0 4 0 3 0 1

44
Zoom 1-order
The 1-order x2 zoom described in the previous
slides.
45
Zoom

• We can repeat the process to obtain x4, x8, x16,
  , zooms of the
• original.

46
Other zooming methods

• Other more sophisticated algorithms use
• Higher order interpolation, e.g. cubic
  interpolation.
• Assume that the original pixels are on a cubic
  B-spline.
• Compute the pixels in between so that they also
  lie on the same spline.
• Nonlinear diffusion.
• Assume that the values of the original pixels
  correspond to a natural property of them, e.g.
  temperature.
• Solve a differential equation to compute the
  flow of the heat in the image and thus find the
  values of the pixels in between.

47
Summary of zoom methods

• 0-order and 1-order linear interpolation can be
  used to obtain zooms of
• an image.
• The results are generally poor, but the method is
  very fast and simple.
• Implementing these algorithms with spatial linear
  filters is better than a
• direct implementation
• It is always good to use general tools spatial
  linear filters have other uses than zooming.
• Spatial linear filters are very fast because
  they are supported by the GPU (Graphics
  Processing Unit).

48
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

49
Intensity transformation functions

• An intensity transformation function is a
  function T that transforms the
• intensity (that is, the value) of each pixel

50
Intensity transformation functions

• We can describe the transformation of the image
  by a function T(r) with
• range the set of intensity values 0,1.

51
Intensity transformation functions

• For a given T(r), the value of a pixel of the
  output image depends only
• on the value of the corresponding pixel of the
  input image.

52
Example

• The function
• T(r) 1 - r
• gives the negative of the original image.

T(r) 1 - r
53
Gamma correction

• The gamma correction changes the value of each
  pixel by
• x ? xgamma
• If gammalt1 the mapping is weighted toward higher
  (brighter) output
• values.
• If gammagt1 the mapping is weighted toward lower
  (darker) output
• values.
• If gamma 1 the mapping is linear.

54
Gamma correction

• If gammalt1 the mapping is weighted toward higher
  (brighter) output
• values.

55
Gamma correction

• A gamma value of 0.25 brightens the image.

56
Gamma correction

• If gammagt1 the mapping is weighted toward lower
  (darker) output
• values.

57
Gamma correction

• A gamma value of 2 darkens the image.

58
Gamma correction

• If gamma1 the mapping is linear.

59
Contrast stretching functions

• The plot of a contrast stretching transformation.

60
Contrast stretching functions
61
Contrast stretching functions

• The plot of a sharper contrast stretching
  transformation.

62
Contrast stretching functions
63
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

64
Histograms

• We already introduced several tools for image
  processing, images, i.e.
• linear filters and intensity transformation
  functions.
• In the selection of the tools we may rely on the
  visual inspection of the
• original image and our experience from
  experimentation with different
• processing tools.
• Histograms can be used for a more rigorous
  analysis of images.

65
Histograms

• Histograms convey in a graphical form statistical
  information about the
• distribution of the intensities.
• In the discrete case (e.g. when the intensity
  values are integers
• between 0 and 255) that means the number of
  pixels with that intensity.

66
Example

• Consider the following 5x5 image.
• The intensity of each pixel is an integer between
  1 and 8.

1 8 4 3 4 1 1 1 7 8 8 8 3
3 1 2 2 1 5 2 1 1 8 5 2
67
Example

• 1 8 4 3 4
• 1 1 1 7 8
• 8 8 3 3 1
• 2 2 1 5 2
• 1 1 8 5 2

68
Example
69
Example
70
Example
71
Histogram function

• The histogram function is defined over all
  possible intensity levels.
• For each intensity level, its value is equal to
  the number of the pixels
• with that intensity.

72
Histogram function
73
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

74
Normalized histogram function

• The normalized histogram function is the
  histogram function divided by
• the total number of the pixels of the image
• It gives a measure of how likely is for a pixel
  to have a certain intensity.
• That is, it gives the probability of occurrence
  the intensity.
• The sum of the normalized histogram function over
  the range of all
• intensities is 1.

75
Normalized histogram function
76
Histogram Equalization

• Some applications may require a uniform spread of
  the intensities
• over the whole range of the interval 0,1.
• We can use information from the histogram, in
  particular the normalized
• histogram function, to compute an intensity
  transformation function
• achieving this goal.
• The intensity transformation function is given by
  
• That is, we add the values of the normalized
  histogram function from 1
• to k to find where the intensity will be
  mapped.

77
Histogram Equalization
78
Histogram Equalization
79
Histogram Equalization

• The 32 of the pixels have intensity r1. We
  expect them to cover 32 of the possible
  intensities.
• The 48 of the pixels have intensity r2 or less.
  We expect them to cover 48 of the possible
  intensities.
• The 60 of the pixels have intensity r3 or less.
  We expect them to cover 60 of the possible
  intensities.

80
Example
81
Example
82
Summary of the lecture

• Spatial filters
• Gaussian filter - denoising
• Laplacian filter - enhancement
• Zooming
• Intensity transformation functions
• Histograms
• Histogram equalization
• Histogram specification

83
Histogram specification

• In the continuous case we think of the histogram
  function h as a
• continuous function.

84
Histogram specification

• The number of occurrences in a certain region
  corresponds to the area
• below the graph of the function and inside that
  region.
• In the normalized histogram function p, the area
  below the whole curve
• is equal to 1.

85
Histogram specification

• In the discrete case we used the sum of the
  values of the normalized
• histogram function between 1 and k.
• In the continuous case we use the integral of the
  function between 0
• and r

Frequency of occurrence
p
Intensities
0
1
r
86
Histogram specification

• Histogram equalization tries to make the
  distribution of the intensities
• uniform.
• Because of the irregular initial distribution,
  usually this is not possible.
• Moreover, sometimes histogram equalization
  introduces artifacts, (see
• the example with the photograph of the moon).

87
Histogram specification

• Instead, we can specify the function we want the
  histogram to
• approximate.

88
Histogram specification

• The intensity transformation function is now
  given by