3-D Computer Vision CSc 83020

1
3-D Computer VisionCSc 83020

• Image Processing I/Filtering

2
Image Processing I/Filtering

• Convolution (1-D)
• Linear Shift Invariant Systems
• Convolution (2-D)
• Application Noise
• Filtering Averaging, Smoothing, Median..

3
Convolution (Important!)

h
f
g
Used for Derivatives, Edges, Matching,
4
Convolution

h
f
g
5
Convolution
f(?)
x
?
h(?)
6
Convolution
f(?)
x
?
h(?)
h(-?)
7
Convolution
f(?)
x
?
h(?)
h(-?)
h(x-?)
?
x
8
Convolution
f(?)
h(x-?)
x
?
f(?) h(x-?)
x
?
g(x) area under curve
9
Convolution
f(?)
h(x-?)
x
?
f(?) h(x-?)
?
x
g(x) area under curve
Calculate g(x) for all x!!
10
Convolution
g(x)
x
f(x)
h(x)
Calculate g(x) for all x!! gt g(x) 1-D function
11
Example
a(?)
b(?)
1
1
?
?
1
-1
1
-1
c a b
?
12
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
13
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
xlt-2
14
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
x-2
15
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
-2ltxlt-1
16
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
-2ltxlt-1
17
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
x-1
18
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
-1ltxlt0
19
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
x0
20
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
0ltxlt1
21
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
x1
22
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
1ltxlt2
23
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
x2
24
Example
a(?)
b(-?)
1
1
?
?
1
-1
1
-1
a(?)
1
b(x-?)
?
1
-1
x
xgt2
25
Example
a(?)
b(?)
1
1
?
?
1
-1
1
-1
c a b
c(x)
1
x
1
-1
-2
2
26
Properties of Convolution

• Commutativity b a a b.
• Associativity (a b) c a (b c)
• Cascaded system

g
f
h1
h2
g
f
h1h2
Equivalent Systems
g
f
h2h1
27
Convolution Discrete
f(?)
m
?
m1
m2
h(?)
n2
n1
Discrete samples of continuous signal. Sampling
at regular intervals. Example Scanline
28
Convolution Discrete
f(?)
m
m-n1
m-n2
?
m1
m2
h(?)
n2
n1
29
One Scan Line 1-D discrete signal
f(?), m10, m2450.
0
450
30
One Scan Line 1-D discrete signal
f(?), m10, m2450.
1/9
0
450
fh ?
4
-4
h(?), n1-4, n24.
31
One Scan Line 1-D discrete signal
f(?)
1/9
h(m- ?)
m
m4
m-4
0
450
m
fh ?
32
One Scan Line 1-D discrete signal
f(?)
1/9
h(m- ?)
m
m4
m-4
0
450
m
fh ?
33
One Scan Line 1-D discrete signal
f(?), m10, m2450.
0
450
h filter or mask. fh filtered version of
f. In this case h spatially averages f in a
neighborhood of 9 samples.
34
Recap

• 1-D Convolution
• Continuous vs. Discrete.
• Finite vs. Infinite signals (spatial domain).
• Filtering.

35
Linear Shift Invariant Systems
f(x)
g(x)
Linearity
f1(x)
g1(x)
f2(x)
g2(x)
af1(x)bf2(x)
ag1(x)bg2(x)
36
Linear Shift Invariant Systems
f(x)
g(x)
Shift Invariance
f(x-a)
g(x-a)
g(x)
f(x)
x
x
g(x)
f(x-a)
x
x
a
a
37
Properties of Convolution

• Commutativity b a a b.
• Associativity (a b) c a (b c)
• Cascaded system

g
f
h1
h2
g
f
h1h2
Equivalent Systems
g
f
h2h1
38
Convolution
Used for Derivatives, Edges, Matching,
Convolution LINEAR SHIFT INVARIANT
f(x)
g(x)
h(x)
Also, any LSIS is doing a CONVOLUTION!
39
Properties of Convolution

• Commutativity b a a b.
• Associativity (a b) c a (b c)
• Cascaded system

g
f
h1
h2
g
f
h1h2
Equivalent Systems
g
f
h2h1
40
Example of LSIS
g
f
Defocused image g Processed version of Focused
image f. Ideal Lens
f(x)
g(x)
LSIS
Linearity Brightness Variations. Shift
Invariance Scene Movement. Note Not valid for
lenses with non-linear distortions
(aberrations). Study of LSIS leads to useful
algorithms for processing images!
41
System as a black box
g
f
h
Can we find h? What f will give us gh?
1/(2e)
d(x)
d(x)
d(x)
1/(2e)
1/(2e)
Decrease e
Decrease e
x
x
x
2e
2e
2e
42
System as a black box
d(x)
Unit Impulse Function
1/(2e)
x
1
-1
2e
Impulse Response
f(x)d(x)
h(x)
IMPULSE RESPONSE
43
Impulse Response
f(x)d(x)
h(x)
IMPULSE RESPONSE
44
Image Formation
Scene
Image
Optical System
Point Source d(x)
Point Spread Function h(x)
Optical System
In an ideal system h(x)d(x) Optical Systems are
never ideal!
Human Eye Point Spread Function.
45
2-D Convolution

g(x,y) Output Image
h(x,y) Filter
f(x,y) Input Image
46
Discrete Convolution




















1 1 1 1 1
1 2 2 2 1
1 2 3 2 1
1 2 2 2 1
1 1 1 1 1
y
(0,0)
g (larger than f)
x
h
f
47
Discrete Convolution




















1 1 1 1 1
1 2 2 2 1
1 2 3 2 1
1 2 2 2 1
1 1 1 1 1
1 1 1 1 1
1 2 2 2 1
1 2 3 2 1
1 2 2 2 1
1 1 1 1 1
f
h
48
Discrete Convolution 2D flip
1 1 1 1 1
4 3 2 2 1
4 5 3 2 1
4 5 5 3 1
4 4 4 4 1
1 4 4 4 4
1 3 5 5 4
1 3 3 5 4
1 3 3 3 4
1 1 1 1 1
49
Discrete Convolution
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1

g
f
h
50
Discrete Convolution
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
51
Discrete Convolution
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
52
Discrete Convolution
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
53
Discrete Convolution
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
54
Discrete Convolution
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
55
Discrete Convolution
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
56
Discrete Convolution
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
57
Discrete Convolution
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
58
Discrete Convolution
1 2 3 4 5 4 3
2 1 2 6 10 14 16 14
10 6 2 3 10 20 28 32
28 20 10 3 4 14 28 42
48 42 28 14 4 5 16 32
48 57 48 32 16 5 4 14
28 42 48 42 28 14 4 3
10 20 28 32 28 20 10 3
2 6 10 14 16 14 10 6
2 1 2 3 4 5 4 3
2 1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1

f
g
h
g is larger than f. Finite borders!
59
Discrete Convolution

h
g
f
60
Discrete Convolution
Commutativity

h
g
f
61
Gaussian Noise
62
Linear Filtering
m size of filter (odd number) m/2 integer
(i.e. if m5, m/22)
63
Linear Filtering
m3
64
Mean Filtering - Averaging
m3
65
Gaussian Filtering
Separable Kernel
66
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
y y y y y

I
gr
Ir
67
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
y y y y y

68
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
y y y y y

69
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
y y y y y

70
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
y y y y y

71
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1
y y y y y

72
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1

y y y y y
73
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
1 1 1 1 1 1 2 2
2 1 1 2 3 2 1 1
2 2 2 1 1 1 1 1
1

y y y y y
I
gr
Ir
74
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
y y y y y

Ir
gc
IG
75
Separable convolution
y y y y y
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x

76
Separable convolution
y y y y y
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x

77
Separable convolution
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x
x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x
x x x x x x x x x x
x x x x x x x x
y y y y y

Ir
gc
IG
78
Separable convolution
Two 1-D convolutions are more efficient than one
2-D convolution!
79
Gaussian Filtering
80
Constructing a Gaussian Filter
1-D Gaussian Mask g w width of mask (in
pixels) s continuous Gaussian kernel Relation
between w and s
81
Size of the mask
82
Noise Median Filter
Cannot implement with a convolution mask.
83
Noise Median Filter
Cannot implement with a convolution mask.