Title: MATHEMATICS OF BINARY MORPHOLOGY
1MATHEMATICS OF BINARY MORPHOLOGY and
APPLICATIONS IN Vision
2(No Transcript)
3In general what is Morphology?
- The science of form and structure
- the science of form, that of the outer form,
inner structure, and development of living
organisms and their parts - about changing/counting regions/shapes
- Among other applications it is used to pre- or
post-process images - via filtering, thinning and pruning
- Smooth region edges
- create line drawing of face
- Force shapes onto region edges
- curve into a square
- Count regions (granules)
- number of black regions
- Estimate size of regions
- area calculations
4What is Morphology in computer vision ?
- Morphology generally concerned with shape and
properties of objects. - Used for segmentation and feature extraction.
- Segmentation used for cleaning binary objects.
- Two basic operations
- erosion (opening)
- dilation (closing)
5Morphological operations and algebras
- Different definitions in the textbooks
- Different implementations in the image processing
programs. - The original definition, based on set theory, is
made by J. Serra in 1982. - Defined for binary images - binary operations
(boolean, set-theoretical) - Can be used on grayscale images - multiple-valued
logic operations
6Morphological operations on a PC
- Various but slightly different implementations in
- Scion
- Paint Shop Pro
- Adope Photoshop
- Corel Photopaint
- mm
Try them, it is a lot of fun
7Mathematical Morphology - Set-theoretic
representation for binary shapes
8Binary Morphology
- Morphological operators are used to prepare
binary (thresholded) images for object
segmentation/recognition - Binary images often suffer from noise
(specifically salt-and-pepper noise) - Binary regions also suffer from noise (isolated
black pixels in a white region). Can also have
cracks, picket fence occlusions, etc. - Dilation and erosion are two binary morphological
operations that can assist with these problems.
9Goals of morphological operations
- 1. Simplifies image data
- 2. Preserves essential shape characteristics
- 3. Eliminates noise
- 4. Permits the underlying shape to be identified
and optimally reconstructed from their distorted,
noisy forms
10What is the mathematical morphology ?
- 1. An approach for processing digital image based
on its shape - 2. A mathematical tool for investigating
geometric structure in image - The language of morphology is set theory.
- Mathematical morphology is extension to set
theory.
11Importance of Shape in Processing and Analysis
- Shape is a prime carrier of information in
machine vision - For instance, the following directly correlate
with shape - identification of objects
- object features
- assembly defects
12Binary Morphology
13Shape Operators
- Shapes are usually combined by means of
- Set Difference based on Set intersection
(occluded objects)
Set difference
Set intersection
14Morphological Operations based on combining base
operations
- The primary morphological operations are dilation
and erosion - More complicated morphological operators can be
designed by means of combining erosions and
dilations
We will use combinations of union, complement,
intersection, erosion, dilation, translation...
Let us illustrate them and explain how to combine
15Libraries of Structuring Elements
- Application specific structuring elements created
by the user
16Notation
x
-2 -1 0 1 2
-2 -1 0 1 2
B
y
A special set the structuring element
Origin at center in this case, but not
necessarily centered nor symmetric
X
No necessarily compact nor filled
33 structuring element, see next slide how it
works
17Dilation
18Explanation of Dilation
Dilation x (x1,x2) such that if we center B
on them, then the so translated B
intersects X.
X
difference
dilation
B
19Notation for Dilation
Dilation x (x1,x2) such that if we center B
on them, then the so translated B
intersects X. How to formulate this definition ?
1) Literal translation
Mathematical definition of dilation
2) Better from Minkowskis sum of sets
Another Mathematical definition of dilation uses
the concept of Minkowskis sum
B is ingeneral not the same as B
20The Concept of Minkowski Sum
21Minkowskis Sum
Definition of Minkowskis sum of sets S and B
l
Minkowskis Sum
l
22Another View at Dilation
Dilation
l
Dilation
Dilation
23Comparison of Dilation and Minkowski sum
Dilation
Bx
x and b are points
Minkowski sum
24It is like dilation but we are not going around
, we go only to top and to right
25Dilation and Minkowski Set
Dilation and Minkowski Set are denoted by or by
? No unified notation
26Dilation is not the Minkowskis sum
Minkowskis Sum
l
27Dilation is not the Minkowskis sum
Dilation
l
Dilation
l
B is not the same as B
28Dilation with other structuring elements
29Dilation with other structuring elements
30Dilation vs SE
- Erosion shrinks
- Dilation expands binary regions
- Can be used to fill in gaps or cracks in binary
images
structuring Element ( SE )
- If the point at the origin of the structuring
element is set in the underlying image, then all
the points that are set in the SE are also set in
the image - Basically, its like ORing the SE into the image
31Dilation fills holes
- Fills in holes.
- Smoothes object boundaries.
- Adds an extra outer ring of pixels onto object
boundary, ie, object becomes slightly larger.
32Main Applications of Dilation
33Dilation example
34Possible problems with Morphological Operators
- Erosion and dilation clean image but leave
objects either smaller or larger than their
original size. - Opening and closing perform same functions as
erosion and dilation but object size remains the
same.
35 More Erode and Dilate Examples
Input Image Dilated Eroded
Made in Paint Shop Pro
36Dilation explained pixed by pixel
Denotes origin of B i.e. its (0,0)
Denotes origin of A i.e. its (0,0)
B
A
37Dilation explained by shape of A
Shape of A repeated without shift
B
Shape of A repeated with shift
A
38Properties of Dilation
objects are light (white in binary)
Dilation does the following
- 1. fills in valleys between spiky regions
- 2. increases geometrical area of object
- 3. sets background pixels adjacent to object's
contour to object's value - 4. smoothes small negative grey level regions
39Structuring Element for Dilation
Length 6
Length 5
40Structuring Element for Dilation
41Structuring Element for Dilation
Single point in Image replaced with this in
the Result
42Structuring Element for Dilation
43Definition of Dilation Mathematically
- Dilation is the operation that combines two sets
using vector addition of set elements. - Let A and B are subsets in 2-D space. A image
undergoing analysis, B Structuring element,
denotes dilation
44Dilation versus translation
- Let A be a Subset of and .
- The translation of A by x is defined as
- The dilation of A by B can be computed as the
union of translation of A by the elements of B
x is a vector
45Dilation versus translation, illustrated
Shift vector (0,1)
Element (0,0)
Shift vector (0,0)
B
46Dilation using Union Formula
Center of the circle
This circle will create one point
This circle will create no point
47Example of Dilation with various sizes of
structuring elements
Pablo Picasso, Pass with the Cape, 1960
48Mathematical Properties of Dilation
- Commutative
- Associative
- Extensivity
- Dilation is increasing
-
-
Illustrated in next slide
49Illustration of Extensitivity of Dilation
A
B
Replaced with
Here 0 does not belong to B and A is not included
in A? B
50More Properties of Dilation
- Translation Invariance
- Linearity
- Containment
- Decomposition of structuring element
51Dilation
- The dilation operator takes two pieces of data as
input - A binary image, which is to be dilated
- A structuring element (or kernel), which
determines the behavior of the morphological
operation - Suppose that X is the set of Euclidean
coordinates of the input image, and K is the set
of coordinates of the structuring element - Let Kx denote the translation of K so that its
origin is at x. - The DILATION of X by K is simply the set of all
points x such that the intersection of Kx with X
is non-empty
52Dilation
Example Suppose that the structuring element is
a 3x3 square with the origin at its center
(-1,-1), (0,-1), (1,-1), (-1,0), (0,0),
(1,0), ( 1,1), (0,1), (1,1)
K
X
53Dilation
Example Suppose that the structuring element is
a 3x3 square with the origin at its center
Note Foreground pixels are represented by a
color and background pixels are empty
54Dilation
Structuring element
Input
output
55Dilation
output
56Dilation
output
57Dilation
58Dilation
59Erosion
60Example of Erosion
Erosion x (x1,x2) such that if we center B on
them, then the so translated B is
contained in X.
difference
erosion
61Notation for Erosion
Erosion x (x1,x2) such that if we center B on
them, then the so translated B is
contained in X. How to formulate this definition
? 1) Literal translation
Erosion
2) Better from Minkowskis substraction of sets
Minkowskis substraction
62Notation for Erosion
BINARY MORPHOLOGY
Minkowskis substraction of sets
Erosion
63Erosion with other structuring elements
64Erosion with other structuring elements
Did not belong to X
?
?
When the new SE is included in old SE then a
larger area is created
65Common structuring elements shapes
origin
x
y
circle
disk
segments 1 pixel wide
Note that here
points
66Problem in BINARY MORPHOLOGY using Minkowski Sum
First we calculate the operation in parentheses
to obtain a diamond
67PROBLEM BINARY MORPHOLOGY
next we calculate the external operation to
obtain a hexagon
68ANOTHER EXAMPLE OF EROSION
Where d is a diameter
Problem
ltd/2
d/2
d
69Implementation of dilation very low
computational cost
0
1 (or gt0)
Logical or
70Implementation of erosion very low
computational cost
0
1
Logical and
71More on Erosion
72Erosion
- (Minkowski subtraction)
- The contraction of a binary region (aka, region
shrinking) - Use a structuring element on binary image data to
produce a new binary image - Structuring elements (SE) are simply patterns
that are matched in the image - It is useful to explain operation of erosion and
dilation in different ways.
73Typical Uses of Erosion
- Removes isolated noisy pixels.
- Smoothes object boundary.
- Removes the outer layer of object pixels, ie,
object becomes slightly smaller.
74Properties of Erosion
- Erosion removes spiky edges
- objects are light (white in binary)
- decreases geometrical area of object
- sets contour pixels of object to background value
- smoothes small positive grey level regions
75Erosion Example
76Erosion explained pixel by pixel
A
B
77Structuring Element in Erosion Example
78How It Works?
- During erosion, a pixel is turned on at the image
pixel under the structuring element origin only
when the pixels of the structuring element match
the pixels in the image - Both ON and OFF pixels should match.
- This example erodes regions horizontally from the
right.
79Structuring Element in Erosion Example
80Structuring Element in Erosion Example
81Structuring Element in Erosion Example
82Structuring Element in Erosion Example
83Structuring Element in Erosion Example
84Structuring Element in Erosion Example
85Mathematical Definition of Erosion
- Erosion is the morphological dual to dilation.
- It combines two sets using the vector subtraction
of set elements. - Let denotes the erosion of A by B
86Erosion in terms of other operations
- Erosion can also be defined in terms of
translation - In terms of intersection
Observe that vector here is negative
87Reminder - this was A
88Erosion illustrated in terms of intersection and
negative translation
Observe negative translation
Because of negative shift the origin is here
89Erosion formula and intuitive example
Center of B is here and adds a point
Center here will not add a point to the Result
90Example of Erosions with various sizes of
structuring elements
Structuring Element
Pablo Picasso, Pass with the Cape, 1960
91Properties of Erosion
- Erosion is not commutative!
- Extensivity
- Erosion is dereasing
- Chain rule
The chain rule is as sharp operator in Cube
Calculus used in logic synthesis. There are more
similarities of these algebras
92Properties of Erosion
- Translation Invariance
- Linearity
- Containment
- Decomposition of structuring element
93Duality Relationship between erosion and dilation
- Dilation and Erosion transformation bear a marked
similarity, in that what one does to image
foreground and the other does for the image
background. - , the reflection of B, is
defined as - Erosion and Dilation Duality Theorem
Observe negative value which is mirror image
reflection of B
Similar but not identical to De Morgan rule in
Boolean Algebra
94Erosion as Dual of Dilation
- Erosion is the dual of dilation
- i.e. eroding foreground pixels is equivalent to
dilating the background pixels.
95Duality Relationship between erosion and dilation
- Easily visualized on binary image
- Template created with known origin
- Template stepped over entire image
- similar to correlation
- Dilation
- if origin 1 -gt template unioned
- resultant image is large than original
- Erosion
- only if whole template matches image
- origin 1, result is smaller than original
Another look at duality
96One more view at Erosion with examples
- To compute the erosion of a binary input image by
the structuring element - For each foreground pixel superimpose the
structuring element - If for every pixel in the structuring element,
the corresponding pixel in the image underneath
is a foreground pixel, then the input pixel is
left as it is - Otherwise, if any of the corresponding pixels in
the image are background, however, the input
pixel is set to background value
97Erosion
98Erosion example with dilation and negation
We want to calculate this
We dilate with negation
99Erosion
.. And we negate the result
We obtain the same thing as from definition
100Morphological Operations in terms of more general
neighborhoods
This exists in Matlab
101Erode and Dilate in terms of more general
neighborhoods
Yet another loook at Duality Relationship between
erosion and dilation
102Edge detection or Binary Contour
103Boundary Extraction
104Erode and Binary Contour in Matlab
Erosion can be used to find contour Dilation can
be also used for it - think how?
105Edge detection
This subtraction is set theoretical
Dilate - original
Now you need to invert the image There are more
methods for edge detection
106Opening Closing
- Opening and Closing are two important operators
from mathematical morphology - They are both derived from the fundamental
operations of erosion and dilation - They are normally applied to binary images
107Open and Close
Close Dilate next Erode Open Erode next Dilate
Original image
eroded
dilated
dilated
eroded
Open
Close
108OPENING
109OPENING
Opening
also
OPENING
difference
- Supresses
- small islands
- ithsmus (narrow unions)
- narrow caps
110Opening with other structuring elements
111Open
- An erosion followed by a dilation
- It serves to eliminate noise
- Does not significantly change an objects size
112Comparison of Opening and Erosion
- Opening is defined as an erosion followed by a
dilation using the same structuring element - The basic effect of an opening is similar to
erosion - Tends to remove some of the foreground pixels
from the edges of regions of foreground pixels - Less destructive than erosion
- The exact operation is determined by a
structuring element.
113Opening Example
- What combination of erosion and dilation gives
- cleaned binary image
- object is the same size as in original
Original
114Opening Example Cont
- Erode original image.
- Dilate eroded image.
- Smooths object boundaries, eliminates noise
(isolated pixels) and maintains object size.
Dilate
Original
Erode
115One more example of Opening
- Erosion can be used to eliminate small clumps of
undesirable foreground pixels, e.g. salt noise - However, it affects all regions of foreground
pixels indiscriminately - Opening gets around this by performing both an
erosion and a dilation on the image
116CLOSING
117EXAMPLE OF CLOSING
Closing
also
- Supresses
- small lakes (holes)
- channels (narrow separations)
- narrow bays
118BINARY MORPHOLOGY
Closing previous image with other structuring
elements
With bigger rectangle like this
With smaller cross like this
119Application shape smoothing and noise filtering
Papilary lines recognition
120Application segmentation of microstructures
(Matlab Help)
121PROPERTIES IN BINARY MORPHOLOGY
- Properties
- all of them are increasing
- opening and closing are idempotent
122EXTENSIVE VERSUS ANTI-EXTENSIVE OPERATIONS
- dilation and closing are extensive operations
- erosion and opening are anti-extensive
operations
123DUALITIES OF MORPHOLOGICAL OPERATORS
- duality of erosion-dilation, opening-closing,...
124 Decomposition of structuring elements
operations with big structuring elements can be
done by a succession of operations with small
s.es
125HIT-OR-MISS
Hit-or-miss
Bi-phase structuring element
Hit part (white)
Miss part (black)
126HIT or MISS FOR ISOLATED POINTS
Looks for pixel configurations
background
foreground
doesnt matter
127ISOLATED POINTS
isolated points at4 connectivity
128More examples on Closing
129Close
- Dilation followed by erosion
- Serves to close up cracks in objects and holes
due to pepper noise - Does not significantly change object size
130More examples of Closing
- What combination of erosion and dilation gives
- cleaned binary image
- object is the same size as in original
Original
131 More examples of Closing cont
- Dilate original image.
- Erode dilated image.
- Smooths object boundaries, eliminates noise
(holes) and maintains object size.
Erode
Dilate
Original
132Closing as dual to Opening
- Closing, like its dual operator opening, is
derived from the fundamental operations of
erosion and dilation. - Normally applied to binary images
- Tends to enlarge the boundaries of foreground
regions - Less destructive of the original boundary shape
- The exact operation is determined by a
structuring element.
133Closing is opening in revers
- Closing is opening performed in reverse.
- It is defined simply as a dilation followed by an
erosion using the same
134One more example of Closing
135Mathematical Definitions of Opening and Closing
- Opening and closing are iteratively applied
dilation and erosion - Opening
-
- Closing
136Relation of Opening and Closing
Difference is only in corners
137Opening and Closing are idempotent
- Their reapplication has not further effects to
the previously transformed result
138Properties of Opening and Closing
- Translation invariance
- Antiextensivity of opening
-
- Extensivity of closing
- Duality
-
139Example of Openings with various sizes of
structuring elements
Pablo Picasso, Pass with the Cape, 1960
140Example of Closings with various sizes of
structuring elements
Example of Closing
141Thinning and Thickening
142Thinning and Thickening
Thinning
Thickenning
- Depending on the structuring elements (actually,
series - of them), very different results can be achieved
- Prunning
- Skeletons
- Zone of influence
- Convex hull
- ...
143Prunning at 4 connectivity
Prunning at 4 connectivity remove end points by
a sequence of thinnings
This point is removed with dark green neighbors
1 iteration
144IDEMPOTENCE shown as a result of thinning
1st iteration
2nd iteration
3rd iteration idempotence
145Other thinning operations
doesnt matter
background
foreground
146USING EROSION TO FIND CONTOURS
Contours of binary regions
erosion
difference
Contour found with larger mask
147CONTOURS with different connectivity patterns
Important for perimeter computation.
148Use of thickening Convex hull
ii. Convex hull union of thickenings, each up
to idempotence
Original shaper
Thickening with first mask
Union of four thickenings
149Example of using convex hull
150iii. Skeleton
Maximal disk disk centered at x, Dx, such that
Dx ? X and no other Dy contains it . Skeleton
union of centers of maximal disks.
151PROBLEMS with skeletons
- Problems
- Instability infinitessimal variations in the
border of X - cause large deviations of the skeleton
- not necessarily connex even though X connex
- good approximations provided by thinning with
- special series of structuring elements
152Example of iterative thinning with 8 masks
1st iteration
153Example of iterative thinning with 8 masks
result of 1st iteration
2nd iteration reaches idempotence
154Thinning with thickening
20 iterations of thinning color white
40 iterations Thickening color white
Some sort of region clustering
155Skeletonization for OCR
BINARY MORPHOLOGY
Application skeletonization for OCR by graph
matching
skeletonization
vectorization
156skeletonization
Application skeletonization for OCR by graph
matching
157Calculation of Geodesic zones of influence (GZI)
- X set of n connex components Xi, i1..n .
- The zone of influence of Xi , Z(Xi) , is the set
of points closer to some point of Xi than to a
point of any other component. - Also, Voronoi partition.
- Dual to skeleton.
158Calculating and using Geodesic Zones of Influence
159Calculating and using Geodesic Zones of Influence
(cont)
dist
160Skeleton by Maximal Balls
161Example Morphological Processing of Handwritten
Digits
thresholding
thinning
smoothing
opening
162PROGRAMMING OF MORPHOLOGICAL OPERATIONS
163USING LISP
164USING LISP
165USING LISP
166USING LISP
167USING LISP
168Morphological Filtering
169Morphological Filtering
- Main idea of Morphological Filtering
- Examine the geometrical structure of an image by
matching it with small patterns called
structuring elements at various locations - By varying the size and shape of the matching
patterns, we can extract useful information about
the shape of the different parts of the image and
their interrelations. - Combine set-theoretical and morphological
operations
170Example 1 Morphological filtering
- Noisy image will break down OCR systems
Noisy image
Clean original image
171Morphological filtering (MF)
Restored image
172Rank Filter Median
Input 1 operation 2 operations
173Postprocessing
- Opening followed by closing.
- Removes noise and smoothes boundaries.
174Postprocessing
- Opening followed by closing.
- Removes noise and smoothes boundaries.
175(No Transcript)
176 177(No Transcript)
178(No Transcript)
179(No Transcript)
180(No Transcript)
181(No Transcript)
182(No Transcript)
183(No Transcript)
184erosion
dilation
185(No Transcript)
186(No Transcript)
187Change of histogram as a result of dilation
188(No Transcript)
189(No Transcript)
190(No Transcript)
191(No Transcript)
192(No Transcript)
193(No Transcript)
194(No Transcript)
195Removal of Border Objects
- Marker is the border itself
196Summary on Morphological Approaches
- Mathematical morphology is an approach for
processing digital image based on its shape - The language of morphology is set theory
- The basic morphological operations are erosion
and dilation - Morphological filtering can be developed to
extract useful shape information - Methods can be extended to more values and more
dimensions - Nice mathematics can be formulated - non-linear
197Conclusion
- Segmentation separates an image into regions.
- Use of histograms for brightness based
segmentation. - Peak corresponds to object.
- Height of peak corresponds to size of object.
- If global image histogram is multimodal, local
image region histogram may be bimodal. - Local thresholds can give better segmentation.
198Conclusion
- Postprocessing uses morphological operators.
- Same as convolution only use Boolean operators
instead of multiply and add. - Erosion clears noise, makes smaller.
- Dilation fills in holes, makes larger.
- Postprocessing
- Opening and closing to clean binary images.
- Repeated erosion with special rule produces
skeleton.
199Problems 1 - 6
- 1. Write LISP or C program for dilation of
binary images - 2. Modify it to do erosions (few types)
- 3. Modify it to perform shift and exor operation
and shift and min operation - 4. Generalize to multi-valued algebra
- 5. Create a comprehensive theory of multi-valued
morphological algebra and its algorithms
(publishable). - 6. Write a program for inspection of Printed
Circuit Boards using morphological algebra.
200Problem 7.
- Electric Outlet Extraction has been done using a
combination of Canny Edge Detection and Hough
Transforms - Write a LISP program that will use only basic
morphological methods for this application.
201Image Processing for electric outlet, how?
- Currently there are many, many ways to approach
this problem - Segmentation
- Edge Detection
- DPC compression
- FFT
- IFFT
- DFT
- Thinning
- Growing
- Haar Transform
- Hex Rotate
Alpha filtering DPC compression Perimeter Fractal
Gaussian Filter Band Pass Filter Homomorphic
Filtering Contrast Sharper Least Square
Restoration Warping Dilation
202Image Processing, how?
- Create morphological equivalents of other image
processing methods. - New, publishable, use outlet problem as example
to illustrate
203Problem 8. Openings and Closings as examples.
- The solution here is to follow up one operation
with the other. - An opening is defined as an erosion operation
followed by dilation using the same structuring
element. - Similarly, a closing is dilation followed by
erosion. - Define and implement other combined operations.
204Problems 9 - 12.
- 9. Generalize binary morphological algebra from 2
dimensional to 3 dimensional images. What are the
applications. - 10. Write software for 9.
- 11. Generalize your generalized multi-valued
morphological algebra to 3 or more dimensions,
theoretically, find properties and theorems like
those from this lecture. - 12. Write software for 11.
205Problem 13
- Mathematical morphology uses the concept of
structuring elements to analyze image features. - A structuring element is a set of pixels in some
arrangement that can extract shape information
from an image. - Typical structuring elements include rectangles,
lines, and circles. - Think about other structuring elements and their
applications.
206(No Transcript)
207Morphological Operations Matlab
BWMORPH Perform morphological operations on
binary image. BW2 BWMORPH(BW1,OPERATION)
applies a specific morphological operation to the
binary image BW1. BW2 BWMORPH(BW1,OPERATION,N)
applies the operation N times. N can be Inf, in
which case the operation is repeated until the
image no longer changes. OPERATION is a string
that can have one of these values 'close'
Perform binary closure (dilation followed by
erosion) 'dilate' Perform dilation using the
structuring elementones(3) 'erode' Perform
erosion using the structuring elementones(3)
'fill' Fill isolated interior pixels (0's
surrounded by1's) 'open' Perform binary
opening (erosion followed bydilation) 'skel'
With N Inf, remove pixels on the boundariesof
objects without allowing objects to break apart
demos/demo9morph/
208Sources
D.A. Forsyth, University of New Mexico, Qigong
Zheng, Language and Media Processing Lab Center
for Automation Research University of Maryland
College Park October 31, 2000 John Miller Matt
Roach J. W. V. Miller and K. D. Whitehead The
University of Michigan-Dearborn Spencer
Lustor Light Works Inc. C. Rössl, L. Kobbelt,
H.-P. Seidel, Max-Planck Institute for, Computer
Science, Saarbrücken, Germany LBA-PC4 Howard
Schultz Shreekanth Mandayam ECE Department Rowan
University D.A. Forsyth, University of New Mexico
209More recent Sources
- Howie Choset
- G.D. Hager,
- Z. Dodds,
- Dinesh Mocha