Machine Vision lecture 2

1
Machine Visionlecture 2

• Topics
• Point processing
• Neighborhood processing

2
Fundamental Steps in Computer Vision

• Today
• Pre-processing
• Segmentation

3
Machine Vision

• Improve image quality
• Binarize image
• Remove noise
• Extract BLOBs
• Calculate features

Today
Feature vector 2,1,,3
. . .
. . .
Feature vector 4,7,,0
4
Point Processing

• What is point processing?
• Grey level mapping
• Histograms
• Segmentation using thresholding
• Computing with images

5
What is point processing?

• Only one pixel in the input has an effect on the
  output
• For example
• Changing the brightness, thresholding, histogram
  stretching

Input
Output
6
Point processing

• Grey level enhancement
• Process one pixel at a time independent of all
  other pixels
• For example used to correct Brightness and
  Contrast (remote control)

Too low contrast
Too high contrast
Too high brightness
Too low brightness
Correct
7
Brightness

• The brightness is the intensity
• Change brightness
• To each pixel is added the value b
• f(x,y) is the input image
• g(x,y) is the (enhanced) output image
• If bgt0 gt brighter image
• If blt0 gt less bright image

g(x,y) f(x,y) b
8
Contrast

• The contrast describes the level of details we
  can see
• Change contrast
• Each pixel is multiplied by a
• f(x,y) is the input image
• g(x,y) is the (enhanced) output image
• If agt1 gt more contrast
• If alt1 gt less contrast

g(x,y) a f(x,y)
9
Combining brightness and contrast
g(x,y) a f(x,y) b

• Both
• A straight line
• Greylevel mapping
• X-Axis Input Value
• Y-Axis Output Value
• This plot Identity
• Output equals Input a1 and b0
• Apply to each pixel!
• To save time the greylevel
• mapping can be written as a
• Lookup-Table

Output image g(x,y)
Input image f(x,y)
(Show bridge)
10
How to set the greylevel mapping
k

• Histogram processing a powerfull tool!

11
Histogram Types
Dark Image
Bright Image
Low Contrast
High Contrast

• How can a histogram help? (show PET_scan,
  TEM_filter)

12
Histogram processing

• Different types of mapping
• Piecewise linear

13
Histogram processing

• Non-linear, e.g., Logarithmic
• Arbitrary

14
Improving contrast

• Humans cannot tell the difference between
  greylevel values too close to each other
• So spread out the greylevel values
• This is called histogram stretching

15
Histogram stretching

• Algorithm
• Find maximum
• Find minimum
• Shift values so that minimum is 0
• Scale values so that maximum is 255 and write
  into output image g(x,y)

max0 //Check each pixel if f(x,y)gtmax maxf(x
,y)
//For each pixel g(x,y)f(x,y)-min
c255/(max-min) //For each pixel g(x,y)round(g(
x,y)c)
16
Histogram stretching
17
Something really different
18
Segmentation

• Until now Image processing (manipulation)
• Image analysis segmentation
• The task
• Information versus noise
• Foreground (object) versus background

19
Segmentation

• Use greylevel mapping and the histogram
• When two peaks (modes) of a histogram correspond
  to object and noise (Show AuPbSn40, bridge)
• Find a THRESHOLD value, T, that separates the two
  peaks. This process is called THRESHOLDING
• Algorithm
• If f(x,y) gt T then g(x,y) 1, else g(x,y) 0
• If f(x,y) lt T then g(x,y) 1, else g(x,y) 0
• Result a binary image where
• object pixels 1 and noise 0
• (Show AuPbSn40, bridge, 2Dgel, blobs)

20
Segmentation

• In Machine Vision, obtaining a bi-modal histogram
  is the sole purpose of the image acquisition
• Lighting
• Setup
• Camera
• Lins

Background
Object
Ideal
21
Segmentation

• How to define the Theshold?
• If we have a good setup gt the Threshold is
  static!
• Find it during training
• But the histogram is NEVER static!!
• Find it automatically
• Find the two peaks and set the threshold between
  them
• We want small variances and large distance
  between the means
• Try all values

22
Computing with images
23
Image Computing

• Arithmetic An image calculator - / etc.
• Logic operations (OR, AND, etc. )
• Image computation is done pixel-wise
• For example

a2 b2
a1 b1

a2
a1

b2
b1

a4 b4
a3 b3


a4
a3

b4
b3






a2 10
a1 10

a2
a1
10

a4 10
a3 10


a4
a3



24
Image Computing

• The size of the output image
• Sometimes a program requires the images to have
  the same sizes

25
Image Computing Caution!!

• Undesired effects may happen, when computing with
  images Overflow / Underflow
• Example Two Gray value images I1, I2 with pixel
  ranges 0,255. Their Pixel sum may be larger
  than 255!! For example 124202326!
• Subtraction results may be smaller than 0!!
• For example 124-202-78!!

26
Solution to the Overflow/Underflow Problem

• Use an intermediate image
• Pixel values are float values (32 bits/ 4 bytes)
• Can store almost any number
• Write computation results into intermediate image
• Rescale intermediate image to values 0255 and
  write results into normal Gray value image.

27
Gray Value Scaling and Shifting
Intermediate float image
Gray value image
28
Different operations

• Monadic (one image one constant)
• Add Im 17
• Sub Im - 17
• Mul Im 17
• Div Im / 17
• And Im AND 17
• Or Im OR 17
• Xor Im XOR 17
• Min Min(Im,17)
• Max Max(Im,17)
• xxx

• Dynadic (two images)
• Add AB
• Sub A-B
• Mul AB
• Div A/B
• And A AND B
• Or A OR B
• Xor A XOR B
• Min Min(A,B)
• Max Max(A,B)
• xxx

29
Measure noise in your setup

• Subtract two images of the same scene at
  different time instances gt Noise in scene
• Help you to judge the nature of the noise

After
Before
Noise
-

-

30
Image averaging remove noise
Noise added
Original

• Assume noise has zero mean
• The noise tends to cancel out if we average a
  number of images
• Example real value 100.
• Value in different images 103, 109, 98, 87, 95,
  112.
• Average (I1 I2 I3 I4 I5 I6 ) / 6
  104
• The more images the better
• Requirement nothing is moving (changing) in the
  scene!

8
16
64
128
31
Segmentation of the object

• Background subtraction
• Image containing object - Background image
  Object
• Advanced background subtraction used avg. bg.
  image

Object
Background
Image w. Obj.
-

-

32
Neighborhood processing
33
Neighborhood processing

• Several pixels in the input has an effect on the
  output

Input
Output
34
Convolution/correlation
35
Correlation (1D)
Filter coefficients
(mask, kernel, template, window)
Input Signal/Image-row
Filter
Output Signal/Image-row
Filter Response
36
Normalise filter response
Max values in image
C
B
Filter coefficients
A
Max filter response A 255 B 255 C 255
ltgt 255 (ABC) If max filter
response 255 (one byte) then New filter
response filter response / (ABC)
37
Correlation (1D)
38
Correlation (1D)
This process is called Correlation!!
39
Correlation on images

• The filter is now 2D
• Kernel (mask), kernel coefficients
• Size 3x3, 5x5, 7x7, .

Input
Output
40
Math. of 2D Correlation
Correlation
41
Applications of convolution/correlation

• Many many operations defined by the programmer.
  and some standard operations
• Object detection
• Template matching
• Smoothing filters
• Edge detection (next time)
• Morphology (next time)
• .

42
Simple Object Detection

• Finding a specific object in the image
• 1D example An object is given (known) as an
  image, e.g.,
• Task Find this object in an image

Input
Output
For images this is called corelation or template
matching!
43
Template Matching

• The filter is called a template or a mask
• The brighter the value in the output, the better
  the match
• Implemented as the correlation coefficient

Input image
Output
Template
44
Smoothing filters

• Also know as Smoothing kernel, Mean filter, Low
  pass filter
• The simplest filter
• Spatial low pass filter
• Another mask
• Gaussian filter

45
Applications of smoothing

• Blurring to remove
• identity or other details
• Degree of blurring kernel size

46
Applications of smoothing

• Preprocessing enhance objects
• Smooth Thresholding

47
Uneven illumination

• Improve segmentation, especially in case of
  uneven illumination
• Remove background
• Algorithm
• g(x,y) f(x,y) f(x,y)
• (show Dot_blots, mean 25, sub, TH, median, morph)

48
Applications of smoothing

• Remove noise

49
Neighbor processing, but not correlation
50
Rank Filters

• Aka order-statistics filters
• Not based on convolution but still neighborhood
  processing
• Principle
• Define a mask, e.g., 3x3
• Sort all pixel-values within the mask into
  ascending order
• Select a pixel-value according to the filter
  type Median, min., max., range,

51
Median Filter

• For an image, mask symmetric 3x3, 5x5, etc.

Sorted 0,0,1,1,1,2,2,2,4
Input
Output
1
52
Median Filter

• Median Filter
• Good for cleaning salt-and-pepper noise.
• Better than the mean filter as blurring is
  minimized and edges stay sharp

53
What to remember

• Point processing pixel-wise operations
• Histogram processing
• Segmentation Thresholding. Bimodal histogram
• An image calculator
• Judge noise in your setup
• Neighborhood processing
• Correlation (convolution)
• Kernel, mask, filter, template
• Template matching (object recognition)
• Filtering
• Mean filter blur, preprocessing
• Median filter (good at removing noise)
• Remove background
• Sharpening

54
Xtra
55
Convolution (1D)
Filter coefficients
(mask, kernel, template, window)
Input Signal/Image-row
Filter
Output Signal/Image-row
Filter Response
56
Math of convolution

• g(x) output, h filter, means convolution,
• f(x) input, n _ width of filter / 2 _
• _ _ rounds down, for example _ 1.7_ 1
• For example Filter (h)
• width 3 gt n1

h(-1)1
h(0)2
h(1)1
57
Correlation versus Convolution
Correlation
Convolution
In image processing we use CORRELATION but
(nearly) always call it CONVOLUTION!!!!! Note
When the filter is symmetric correlation
convolution!
58
Math. of 2D Convolution/Correlation
Convolution
Correlation
59
Problems at the borders

• Why is the output image smaller than the input?
• We are lacking information
• The bigger the kernel the bigger the problem
• Does it matter? Yes, if we are going to combine
  the images afterwards

Input
Output
60
Problems at the borders

• Solutions
• Add a value 0, 255, neighbor (input/output)
• Change histogram, very different value, new
  pattern, etc.
• Truncate kernel 3x3 gt for example 2x3
• Complex and not well-defined

Input
Output
61
Problems at the borders

• Solutions
• Complex and perhaps wrong

Circular indexing
3
1
Reflected (mirrored)
2
2
1
2
7
3
Input
Input
62
Combine two images

• Used when adding text etc. to an image
• Mixing two images by averaging
• But what if we want to put more weight on one
  image?
• Alpha blending (or weighted averaging)
• If then this operation is
  standard averaging
• Many programs (CVCG) support alpha blending
• Image representation RGBA (4 bytes) per pixel