Mountain Region Arizona Engineering Capabilities

1
ASU MAT 591Image Processing Scienceand Robotic
VisionRod PickensPrincipal Research
EngineerLockheed Martin, Incorporated
2
Signals and Processing

• Signals
• Analog and discrete signals
• Dimensionality of signals
• 1-D signals
• Sounds (temporal), echocardiogram, seismic signal
• 2-D signals (this presentation)
• Images (spatial)
• 3-D signals
• Video sequences of images (spatial and temporal)
• Signal processing
• Synthesize and analyze signals
• Filter signals using low-pass, band-pass, and
  high-pass filter
• Modify signals such as warp, delay, stretch,
  rotate, shrink,
• Restore and enhance signals
• Recognize patterns and detect signals

3
Signal Processing Now
Animal
Robotic
Touch
Touch
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell
4
The Processing Analogy
5
Analysis and Synthesis of Light
6
Fourier Transforms are Inverse Functions
7
Inverse Functions
8
Filtering
White Light In
Filtering removes all but red colors
Red Light Out
9
Television
Television Stations 3, 5, 6, 13, 15,
Television
Filtering removes all but Channel 6
Channel 6
10
Television
Television Stations 3, 5, 6, 13, 15,
Television
Filtering removes all but Channel 15
Channel 15
11
Radio
Radio Stations
Radio Stations 91.5, 96.9, 100.7
Radio
Station 100.7
Filtering removes all but Station 100.7
12
Radio
Radio Stations
Radio Stations 91.5, 96.9, 100.7
Radio
Station 96.9
Filtering removes all but Station 96.9
13
Vision
Scene of a Room walls, books, desks, chairs,
windows,
Robot vision
Book
Filtering removes all but a book
14
Vision
Scene of a Room walls, books, desks, chairs,
windows,
Scene of a Room
Robot vision
Table
Filtering removes all but a table
15
Graphics to build a scene
16
Data compression
17
Data compression goal
Signal
Approximation of Signal
Filter that eliminates less important data.
18
An Example of a Processing Architecture
19
The Example Architecture
Format
Correct Errors
Preprocess
Restore
Format
Data
Recognize
Analyze
Descriptions
Will Discuss in more detail!
20
Preprocess
Format
Correct Errors
Preprocess
Restore
Preprocess
Data
Normalize Remove Noise Remove Distortions
Analyze
Recognize
Descriptions
21
Fourier Based Noise Filtering
Mostly Noise so is Zeroed
Mostly Signal
Fourier Transform
and Filter the Noise
From Jason Plumb at http//noisybox.net/weblog/
22
Filtering and Enhancing Data
Math to follow
From Mathworks homepage at http//www.mathworks.co
m/
23
Filtering Analysis
Image
Analysis
24
Filtering Removing Noise
Image
Filtering removes noise
25
Filtering Synthesis
Synthesis
Image
Enhanced
26
Filtering
Synthesis
Analysis
27
Enhancing the Data Linear map
28
Warping data
Suppose we have unwanted camera motion.
From Mathworks homepage at http//www.mathworks.co
m/
29
Warping data
We can correct motion errors if we know motion
model.
From Mathworks homepage at http//www.mathworks.co
m/
30
Warping data
From Mathworks homepage at http//www.mathworks.co
m/
31
Warping Correction is an Inverse Function
Warping Correction
Warping
32
Linear Algebra to Flip
33
Linear Algebra to Flip
34
Linear Algebra to Flip
y1
I(x1,y1)
y2
y2y1
x1
y1
x2
x1
x2- x1
35
Linear Algebra to Flip
y1
I(x1,y1)
y2
y2y1
x1
y1
x2
x1
x2- x1
I(x2,y2)
36
Linear Algebra to Flip
y1
I(x1,y1)
y2
y2y1
x1
y1
x2
x1
x2- x1
I(f(x1),g(y1))
37
Linear Algebra to Flip
y1
I(x1,y1)
y2
y2y1
x1
y1
x2
y2
y2
y2
y2
x1
x2- x1
x2
x2
x2
x2
I(x2,y2)I(f(x1),g(y1))
38
Linear Algebra to Flip
y1
y1
y1y2
x1
y2
x1
y2
x2
x1- x2
x2
I(x2,y2)
39
Linear Algebra to Flip
y1
I(f-1(x2), g-1(y2))
y1
y1y2
x1
y2
x1
y2
x2
x1- x2
x2
I(x2,y2)
40
Linear Algebra to Flip
y1
I (x1,y1)I(f-1(x2), g-1(y2))
y2
y2y1
x1
y1
x2
y2
x1
x2- x1
x2
I (x2,y2)
41
Linear Algebra to Flip and Shrink
y1
x1
y2
x2
42
Linear Algebra to Flip and Shrink
y1
y2
y2 -0.5 y1
x1
y1
x2
y2
x2 0.5 x1
x1
x2
43
Correcting warped data (camera motion)
From Mathworks homepage at http//www.mathworks.co
m/
44
Restoration
Format
Correct Errors
Preprocess
Restore
Restore
Data
Remove Sensor Effects
Recognize
Analyze
Descriptions
45
Restoring data for smear, optics,
From Mathworks homepage at http//www.mathworks.co
m/
Smear and optics can be viewed as filters that
can degrade an image!
Uses Linear Systems Theory
Next
46
Restoring data for smear, optics,
From Mathworks homepage at http//www.mathworks.co
m/
Uses Linear Systems Theory
Next
47
Restoration Analysis
Image
Analysis
48
Filtering Removing Smear
Image
Smr-1(wx,wy) is a filter that removes smear or
restores the original object.
49
Filtering Synthesis
Synthesis
Image
Object
50
Filtering
Smear inverted as a filter
Image
Image Restored to best look like original Object
51
Restoring data for smear, optics,
From Mathworks homepage at http//www.mathworks.co
m/
Uses Linear Systems Theory
Image(wx,wy)
Next
52
Restoring data for smear, optics,
From Mathworks homepage at http//www.mathworks.co
m/
Smr(wx,wy)Image(wx,wy)
Uses Linear Systems Theory
Image(wx,wy)
Next
53
Restoring data for smear, optics,
From Mathworks homepage at http//www.mathworks.co
m/
Smr(wx,wy)Image(wx,wy)
Uses Linear Systems Theory
Image(wx,wy)
54
Synthesis and Analysis
Format
Correct Errors
Preprocess
Restore
Data
Synthesize
Recognize
Analyze
Analyze
Descriptions
Decompose / Compose Signals - Transforms
Fourier, SVD, Wavelets - Statistical
Analysis parametric and non-parametric
55
Fourier Transform
56
Fourier Transform
From Wolfram homepage at http//documents.wolfram.
com
Magnitude
Phase
57
Radon Transform
From Mathworks homepage at http//www.mathworks.co
m/
58
Wavelet Transform
From Wolfram homepage at http//documents.wolfram.
com
59
Common Transforms

• Fourier
• Discrete fourier
• Cosine
• Sine
• Hough
• Hadamard
• Slant
• Karhunen-Loeve
• Fast KL
• SVD
• Sinusoidal

Many kinds of transforms
60
Statistics
From Mathworks homepage at http//www.mathworks.co
m/
61
Recognition
Format
Correct Errors
Preprocess
Restore
Data
Recognize
Recognize
Analyze
Descriptions
Label Signals - Signal Detection - Pattern
Recognition - Artificial Intelligence
62
Pattern Recognition
Features are mathematical measurements
Class 2 (rose)
Class 1 (daisy)
Feature 1
Feature 1
Class 3 (sun flower)
Feature 2
Feature 2
63
Mathematical Decisions
Class 1 is z
f2
z
z
z
z
z
z
o
How do we separate the classes?
z
z
o
z
z
z
z
o
z
o
z
o
z
o
o
o
o
o
o
f1
o
o
o
o
o
o
o
Class 2 is o
64
Mathematical Decisions
Class 1 is z
f2
z
z
z
z
z
z
o
z
z
o
z
z
z
Linear decision
z
o
z
o
z
o
z
o
o
o
o
o
o
f1
o
o
o
o
o
o
o
Class 2 is o
65
Mathematical Decision
Class 1 is z
f2
z
z
z
z
z
z
o
z
z
o
z
z
z
Linear decision
z
o
z
o
z
o
z
o
o
o
o
o
o
f1
o
o
o
o
o
o
o
Class 2 is o
66
Mathematical Decision
Class 1 is z
f2
z
z
z
z
z
z
o
z
z
o
z
z
z
Quadratic decision
z
o
z
o
z
o
z
o
o
o
o
o
o
f1
o
o
o
o
o
o
o
Class 2 is o
67
Mathematical Decision
Class 1 is z
f2
z
z
z
z
z
z
z
z
z
z
z
z
z
z
z
f1
68
Mathematical Decision
f2
o
o
o
o
o
o
o
o
o
o
f1
o
o
o
o
o
o
o
Class 2 is o
69
Mathematical Decision
Class 1 is z
f2
z
z
z
z
z
z
o
z
z
o
z
z
z
z
o
z
o
z
o
z
o
o
o
o
o
o
f1
o
o
o
o
o
o
o
Class 2 is o
70
Isolate Object Segmentation
From Mathworks homepage at http//www.mathworks.co
m/
71
Analyze Object Features
- Length - Width - Contour - Orientation

• - Edges
• Skeleton
• - Texture Details
• - Intensity

From Mathworks homepage at http//www.mathworks.co
m/
72
Matched Filtering (registration)
Input Image or Iin(x,y)
From Mathworks homepage at http//www.mathworks.co
m/
73
Matched Filtering (registration)
Input Image or Iin(x,y)
Exemplar (reference) or Iref(x,y)
From Mathworks homepage at http//www.mathworks.co
m/
74
Matched Filtering (registration)
Input Image or Iin(x,y)
Exemplar (reference) or Iref(x,y)
From Mathworks homepage at http//www.mathworks.co
m/
75
Matched Filtering (registration)
Input Image or Iin(x,y)
Exemplar (reference) or Iref(x,y)
Actually search form min of x,y simultaneously!
From Mathworks homepage at http//www.mathworks.co
m/
76
Image Processing Summary
Format
Correct Errors
Preprocess
Restore
Format
Data
Recognize
Analyze
Descriptions
77
References

• Fundamentals of Image Processing by Jain
• Digital Image Analysis by Gonzalez and Wintz
• Pattern Recognition by Fukunaga
• Pattern Recognition Principles Tou and Gonzalez
• Detection, Estimation, and Modulation Theory by
  Van Trees
• Pattern Classification by Duda and Hart
• Robot by Hans Moravec (graphics from
  www.amazon.com)

78
Signal Processing 50 years from now
Evolved
Robotic
Touch
Touch
Vision
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell
79
Signal Processing 50 years from now
Evolved
Robotic
Touch
Touch
Vision
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell
80
Signal Processing 50 years from now
Evolved
Robotic
Touch
Touch
I see, therefore, am I? Hmmm.
Vision
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell