Neighborhood Operations

1
Neighborhood Operations
2
Objectives

• Why are neighborhoods important?
• What is linear convolution?
• discrete
• templates, masks or filters
• algorithm mechanics
• graphical interpretation
• Describe non-linear operators
• maximum
• minimum
• median
• What is tiling?

3
Why are neighbourhoods important?
pixel
4
Because

• Provide context for individual pixels.
• Relationships between neighbors determine image
  features.
• Neighborhood operations
• noise reduction
• edge enhancement
• zooming

5
Noise reduction
Edge Enhancement
Zooming
6
Neighbourhood Operations

• Linear convolution ()
• ABCD BCDA .
• Non-linear operators
• median, max, min, ...

7
Convolution versus Spectral

• We learnt two methods of processing images
• Convolution
• Spectral
• We analyzed and demonstrated how to build a
  processor (systolic, pipelined, parallel,
  cellular automaton) for 1D convolution.
• 1D convolution is used in speech processing and
  in polynomial multiplication.
• We will use visualized animations now to show in
  more detail how 2D convolution works for images.
• This should convince you how important it is to
  do convolution quickly in modern Spectral
  Architectures, especially for 3D etc.

8
2D Convolution
We will show more examples of convolution now,
especially for 2D data

• Consists of filtering an image A using a filter
  (mask, template) B.
• Mask is a small image whose pixel values are
  called weights.
• Weights modify relationships between pixels.

9
Input image
Filter, mask or template
Convolved Image
A
C
B
?

2?? 2
3?? 3
4?? 4
10
C1,1
?
?
?
11
C1,2
?
?
?
12
C1,3
?
?
?
13
C2,1
?
?
?
14
(No Transcript)
15
Mathematical Notation
C1,1
?
?
?
16
Convolution
Input image
Filter, mask or template
Convolved Image
A
C
B
6
23
21
?

9
26
19
16
27
17
2?? 2
3?? 3
4?? 4
17
Convolution size
Typical Mask sizes 3?3, 5 ?5, 7?7, 9 ?9, 11?11
What is the convolved image size for a 128 ? ?
128 image and 7 ? 7 mask?
18


We convolve with 99 averaging filter
19
Nonlinear Neighbourhood Operations

• Maximum
• Minimum
• Median

We discussed already sorter architecture (three
variants pipelined, butterfly combinational and
sequential controller). It can be used for all
these operations, and also for other non-linear
operators
20
Max and Min Operations
C1,2
63max, 59min
21
Median Operation
9 8 7 6 5 4 3 2 1
C1,2
22
(No Transcript)
23
Edge Detection

• What do we mean by edge detection?
• What is an edge?

24
What is Edge Detection?

• Detects large intensity transitions between
  pixels
• Redraws the image with only the edges showing

0 0 0 33
0 0 45 78
0 45 23 33
0 0 42 76
0 0 0 38
25
What is an Edge?
26
What is an Edge?
Where is edge? Single pixel wide or multiple
pixels?
27
What is an Edge?
Noise is here
Noise have to distinguish noise from actual edge
28
What is an Edge?
Is this one edge or two?
29
What is an Edge?
Texture discontinuity
30
Edge Detection so what is an edge to be
detected?

• What is an edge
• A large change in image brightness of a short
  spatial distance
• Edge strength (I(x,y)-I(xdx,y))/dx

But this general definition still allows for many
theories, software implementation and hardware
architectures.
31

• Now we will discuss and illustrate various kinds
  of filter operators

32
Edge Detection Filters

• High - Pass Filtering Eliminates Uniform Regions
  (Low Frequencies)
• edge detection or enhancement

33
Edge Detection Filters
34
Edge Detection Filters
Edge Detection Continued

• Sum of Kernel Coefficients 0
• differences in signs emphasize differences in
  pixel values
• reduces average image intensity
• Negative pixel values in output?

35
Edge Direction
vertical
horizontal
diagonal
36
Directional High Pass Filters
37
Convolution Edge Detection using Sobel and
similar operators
38
Example of Sobel Operator
Sobel Operator
39
Sobel Edge Detection
40
Convolution Application Examples
We apply Sobel Operator
--Edge Detection
-1 -2 -1 0 0 0 1 2 1
-1 0 1 -2 0 2 -1 0 1
Column Mask
Row Mask
as mask to a sub-field of a picture
p0, p1, p2 p3, p4, p5 p6, p7, p8
-1 2 -1 0 0 0 1 2 1

(p6-p0)2(p7-p1)(p8-p2)
The final step of the convolution equation,
dividing by the weight , must be ignored

• We can learn from the result obviously
• The result of the above calculation for column
  mask is horizontal difference
• With Row Mask we will get vertical difference

41
Convolution Application Examples
--Edge Detection with Sobel Operator
The weight of a mask determines the grey level of
the image after convolution.
Like the weight of Sobel Mask W W (-1) (-2)
(-1) 0 0 0 1 2 1 0 The resulting image
lost its lightness to be dark.
42
Sobel Operator
43
Sobel Operator
S2
S1
Edge Magnitude
Edge Direction
44
Comparison of Edge Detection Algorithms
Sobel
Canny
Prewitt
Ticbetts
45
Edge Direction
Assymetric kernels detect edges from specific
directions
NorthEast 1 -1 -1 1 -2 -1 1 1 -1
East 1 1 -1 1 -2 -1 1 1 -1
North -1 -1 -1 1 -2 1 1 1 1
46
Robinson Operator
47
Robinson Compass Masks
Arrows show edge directions
48
Roberts Operator
49
Roberts Operator
or

• Does not return any information about the
  orientation of the edge

50
Prewitt Operator
P2
P1
Edge Magnitude
Edge Direction
51
Edge Detection Filters
Prewitt Row
52
Original and filtered cow
53
Edge Detection Filters compare Prewitt and Sobel
Edge Detection (continued)

• First Order (Gradient) Kernels
• Prewitt Row
• 1 0 -1
• 1 0 -1
• 1 0 -1
• Sobel Row
• 1 0 -1
• 2 0 -2
• 1 0 -1
• Combine Row and Column Operators

54
1D Laplacian Operator
first derivative
second derivative
55
2D Laplacian Operator
Convolution masks approximating a Laplacian
This is just one example of Laplacian, we can use
much larger window
56
Input Mask
Output
57
Image Processing Operations for Early
Vision Edge Detection
58
Reminder Effect of Filters
low
high
59
Edges
are the important part of images
simplest, least robust
intensity color edges textures contours condensati
on...
most difficult, most robust
There are many letters B occluded by black shape
here. How to find them?
60
Image Processing Operations

• Edge Detection
• Edges are curves in the image plane across which
  there is a significant change in image
  brightness.
• The goal of edge detection is the construction
  of an idealized line drawing

61
Pixels on edges
62
Edges found
63
Edge effects rarely ideal edges
Not all information is created equal...
64
Causes of edges

• Depth discontinuity
• One surface occludes another
• Surface orientation discontinuity
• the edge of a block
• reflectance discontinuity
• texture or color changes
• illumination discontinuity
• shadows

65
Edges causes
What are they? Why?
four physical events that cause image edges...
66
Edges causes
What are they? Why?
four physical events that cause image edges...

• surface color/intensity

• surface normal

discontinuities in

• depth

• lighting (specularities)

67
Edges causes
Edges are image locations with a local maximum in
image gradient in the direction of that gradient
(steepness)
68
Formal Model of Edge (cont)
Formal Model of Edge
69
Formal Model of Edge (cont)
Formal Model of Edge Roberts
70
Formal Model of Edge (cont)
Formal Model of Edge Laplacian and Marr-Hildreth
71
Formal Model of Edge (cont)
Formal Model of Edge
72
Formal Model of Edge (cont)
Formal Model of Edge
73
Thresholds
Thresholds are important, done before or during
edge detection.
original image
very high threshold
74
Thresholds
Thresholds
very high threshold
original image
75
Thresholds
very high threshold
original image
76
Thresholds
very high threshold
original image
reasonable
77
Thresholds
very high threshold
original image
too low !
reasonable
this all takes time...