ASAM Image Processing 20082009

1
ASAM - Image Processing2008/2009

• Lecture 4
• Spatial Linear Filtering II

Ioannis Ivrissimtzis 06-Nov-2008
2
Overview

• Boundary
• Sifting
• Application zooming

3
Boundary

• Consider the convolution of matrix by a mask

4
Boundary
Near the boundary the mask does not fit into the
image. How do we process the boundary pixels of
the image?
? ? ? ? ? ? ? ?
0.4 0.0 0.6 0.3 0.2 ? ? 0.8
0.3 0.7 0.2 0.7 ? ? 0.5 0.4
0.7 0.5 0.3 ? ? 0.5 0.3 0.6
0.8 0.7 ? ? 0.9 0.3 0.0 0.7
0.2 ? ? ? ? ? ? ?
?
5
Zero boundary

• Solution 1 The values of the pixels outside the
  boundary of the image
• are set to zero.

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.4 0.0 0.6 0.3 0.2 0.0 0.0 0.8
0.3 0.7 0.2 0.7 0.0 0.0 0.5 0.4
0.7 0.5 0.3 0.0 0.0 0.5 0.3 0.6 0.8
0.7 0.0 0.0 0.9 0.3 0.0 0.7 0.2
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
The zero boundary is convenient when we work in
the frequency domain (which will be introduced
later in the course).
6
Zero boundary

• Assuming a black border outside the image can
  create artifacts in
• spatial linear filtering.

7
Zero boundary
10 iterations of filtering
Original image
Filtered image
8
Replicate the boundary pixels

• Solution 2 Replicate the pixels at the boundary
  of the image.

0.4 0.4 0.0 0.6 0.3 0.2 0.2 0.4
0.4 0.0 0.6 0.3 0.2 0.2 0.8 0.8
0.3 0.7 0.2 0.7 0.7 0.5 0.5 0.4
0.7 0.5 0.3 0.3 0.5 0.5 0.3 0.6 0.8
0.7 0.7 0.9 0.9 0.3 0.0 0.7 0.2
0.2 0.9 0.9 0.3 0.0 0.7 0.2 0.2
9
Replicate the boundary pixels

• By replicating boundary pixels we can avoid
  artifacts in spatial linear
• filtering.

10 iterations of filtering
Original image
10
Periodic images

• Solution 3 Replicate the whole image as a 2D
  periodic tile.

11
Periodic images

• Solution 3 Replicate the whole image as a 2D
  periodic tile.

12
Periodic images

• Solution 3 Replicate the whole image as a 2D
  periodic tile.

0.2 0.9 0.3 0.0 0.7 0.2 0.9 0.2
0.4 0.0 0.6 0.3 0.2 0.4 0.7 0.8
0.3 0.7 0.2 0.7 0.8 0.3 0.5 0.4
0.7 0.5 0.3 0.5 0.7 0.5 0.3 0.6 0.8
0.7 0.5 0.2 0.9 0.3 0.0 0.7 0.2
0.9 0.2 0.4 0.0 0.6 0.3 0.2 0.4
13
Boundary

• The choice of pixel values outside the boundary
  depends on
• The properties of the image (periodicity,
  patterns)
• The type of the application (spatial filtering,
  frequency domain
• filtering)

14
Overview

• Boundary
• Sifting
• Application zooming

15
Sifting

• Consider the convolution of the matrix A by a
  mask M

16
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 .. ..
.. .. .. .. .. .. .. .. ..
.. .. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

17
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 ..
.. .. .. .. .. .. .. .. ..
.. .. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

18
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. .. .. .. .. .. .. ..
.. .. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

19
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 .. .. .. .. .. .. ..
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

20
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 5 .. .. .. .. .. ..
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

21
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 5 4 .. .. .. .. ..
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

22
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 5 4 .. .. 3 .. ..
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

23
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 5 4 .. .. 3 2 ..
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

24
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 5 4 .. .. 3 2 1
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

25
Sifting
Original image
Processed image
.. .. .. .. .. .. 9 8 7
.. .. 6 5 4 .. .. 3 2 1
.. .. .. .. .. ..

• 0 0 0 0 0
• 0 0 0 0 0
• 0 0 1 0 0
• 0 0 0 0 0
• 0 0 0 0 0

The mask has been imprinted on the image, but
rotated by 180. This phenomenon is called
sifting.
26
Sifting

• Sifting can create artifacts on the processed
  image.
• When we use a non-symmetric mask we first rotate
  it by 180 and then
• convolve.
• Most masks are symmetric and thus, invariant
  under rotation by 180.

27
Overview

• Boundary
• Shifting
• Application zooming

28
Region of interest (ROI)

• A region of interest (ROI) is a part of the image
  which we want to
• process separately.
• Spatial filters are often applied on regions of
  interests.

29
x2 zoom

• x2 zooming algorithm
• 1. Given an nxn image A, create a new (2n)x(2n)
  image A' such that
• The values of the pixels of A are copied on the
  pixels A' with both coordinates even.
• The values of the other pixels of A' are set to
  zero.
• 2. Smooth A' with a spatial linear filter.

30
x2 zoom
The values of the pixels of A are copied on the
pixels A' with both coordinates even. The other
pixels of A' have zero values.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.3 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
0.5 0.1 0.6 0.8 0.3 0.7 0.5 0.4 0.7
31
x2 zoom
The values of the pixels of A are copied on the
pixels of A' with both coordinates even. The
other pixels of A' have zero values.
32
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.2 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
.. .. .. .. .. .. ..
0.5 .. .. .. .. ..
.. .. .. .. .. .. ..
.. .. .. .. .. ..
.. .. .. .. .. .. ..
.. .. ..
33
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.2 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
.. .. .. .. .. .. ..
0.5 0.3 .. .. .. .. ..
.. .. .. .. .. ..
.. .. .. .. .. .. ..
.. .. .. .. .. ..
.. .. ..
34
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.2 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
.. .. .. .. .. .. ..
0.5 0.3 0.1 .. .. .. ..
.. .. .. .. .. .. ..
.. .. .. .. .. ..
.. .. .. .. .. .. ..
.. ..
35
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.2 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
.. .. .. .. .. .. ..
0.5 0.3 0.1 0.35 .. .. ..
.. .. .. .. .. .. ..
.. .. .. .. .. ..
.. .. .. .. .. .. ..
.. ..
36
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.2 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
.. .. .. .. .. .. ..
0.5 0.3 0.1 0.35 .. .. 0.65 ..
.. .. .. .. .. ..
.. .. .. .. .. .. ..
.. .. .. .. .. ..
.. ..
37
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.0
0.1 0.0 0.6 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.8 0.0 0.2 0.0 0.7 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.4 0.0
0.7
.. .. .. .. .. .. ..
0.5 0.3 0.1 0.35 .. .. 0.65 0.4
.. .. .. .. .. .. ..
.. .. .. .. .. ..
.. .. .. .. .. .. ..
..
38
x2 zoom
The second step is to smooth A' with a spatial
linear filter.
39
x2 zoom

• The zooming algorithm with the mask
• is called 1-order interpolation zooming.
• Each pixel of A' is the average of the pixels of
  A around it.
• 1-order interpolation zooming is simple and fast,
  but the results are
• generally poor.

40
x2 zoom

• 1-order interpolation zooming can be implemented
  directly.
• From an nxn image A, create a (2n)x(2n) image A'
  using the rules

41
x2 zoom

• The implementation of 1-order interpolation
  zooming with spatial linear filtering is
  generally better than the direct implementation.
• It is always good to use general tools spatial
  linear filters have many other uses.
• Spatial linear filters can be very fast because
  they are supported by the GPU (Graphics
  Processing Unit).

42
x4 zoom

• We can repeat the process to obtain x4, x8, x16,
  , zooms of the
• original.

x4
x2
x1
43
Other zooming methods

• Other zooming methods include higher order
  interpolation zooming,
• especially cubic interpolation.
• Assume that the original pixels are on a cubic
  surface.
• Compute the pixels in between so that they also
  lie on the same cubic surface.

44
Other zooming methods

• A more sophisticated zooming method is based on
  nonlinear diffusion.
• Assume that the values of the original pixels
  correspond to a
• natural property of them, e.g. temperature.
• Solve a differential equation to compute the flow
  of the heat
• between pixels to compute the values of the
  pixels in between.