Unsupervised Learning Networks

1
Unsupervised Learning Networks

• ??? ???

2
Content

• Introduction
• Important Unsupervised Learning NNs
• Hamming Networks
• Kohonens Self-Organizing Feature Maps
• Grossbergs ART Networks
• Counterpropagation Networks
• Adaptive BAN
• Neocognitron
• Conclusion

3
Unsupervised Learning Networks

• Introduction

4
What is Unsupervised Learning?

• Learning without a teacher.
• No feedback to indicate the desired outputs.
• The network must by itself discover the
  relationship of interest from the input data.
• E.g., patterns, features, regularities,
  correlations, or categories.
• Translate the discovered relationship into output.

5
A Strange World
6
Supervised Learning
7
Supervised Learning
Try Classification
8
The Probabilities of Populations
9
The Centroids of Clusters
10
The Centroids of Clusters
Try Classification
11
Unsupervised Learning
12
Unsupervised Learning
13
Clustering Analysis
Categorize the input patterns into several
classes based on the similarity among patterns.
14
Clustering Analysis
Categorize the input patterns into several
classes based on the similarity among patterns.
How many classes we may have?
15
Clustering Analysis
Categorize the input patterns into several
classes based on the similarity among patterns.
2 clusters
16
Clustering Analysis
Categorize the input patterns into several
classes based on the similarity among patterns.
3 clusters
17
Clustering Analysis
Categorize the input patterns into several
classes based on the similarity among patterns.
4 clusters
18
Unsupervised Learning Networks

• The Hamming Networks

19
The Nearest Neighbor Classifier

• Suppose that we have p prototypes centered at
  x(1), x(2), , x(p).
• Given pattern x, it is assigned to the class
  label of the ith prototype if
• Examples of distance measures include the Hamming
  distance and Euclidean distance.

20
The Nearest Neighbor Classifier
The Stored Prototypes
x(1)
x(2)
x(3)
x(4)
21
The Nearest Neighbor Classifier
x(1)
x(2)
? ?Class
x(3)
x(4)
22
The Hamming Networks

• Stored a set of classes represented by a set of
  binary prototypes.
• Given an incomplete binary input, find the class
  to which it belongs.
• Use Hamming distance as the distance
  measurement.
• Distance vs. Similarity.

23
The Hamming Net
MAXNET Winner-Take-All
Similarity Measurement
24
The Hamming Distance
y 1 ?1 1 1 ?1 1 1
x ?1 ?1 1 1 1 ?1 1
Hamming Distance ?
25
The Hamming Distance
y 1 ?1 1 1 ?1 1 1
x ?1 ?1 1 1 1 ?1 1
Hamming Distance 3
26
The Hamming Distance
y 1 ?1 1 1 ?1 1 1
Sum1
x ?1 ?1 1 1 1 ?1 1
?1 1 1 1 ?1 ?1 1
27
The Hamming Distance
28
The Hamming Distance
29
The Hamming Net
MAXNET Winner-Take-All
Similarity Measurement
30
The Hamming Net
MAXNET Winner-Take-All
WM?
Similarity Measurement
WS?
31
The Stored Patterns
MAXNET Winner-Take-All
WM?
Similarity Measurement
WS?
32
The Stored Patterns
Similarity Measurement
33
Weights for Stored Patterns
WS?
34
Weights for Stored Patterns
Similarity Measurement
WS?
35
The MAXNET
MAXNET Winner-Take-All
Similarity Measurement
36
Weights of MAXNET
y1
y2
yn?1
yn
MAXNET Winner-Take-All
1
1
2
n?1
n
37
Weights of MAXNET
y1
y2
yn?1
yn
0lt ? lt 1/n
??
MAXNET Winner-Take-All
1
1
2
n?1
n
38
Updating Rule
0lt ? lt 1/n
??
MAXNET Winner-Take-All
1
1
2
n?1
n
s1
s2
s3
sn
39
Updating Rule
0lt ? lt 1/n
??
MAXNET Winner-Take-All
1
1
2
n?1
n
s1
s2
s3
sn
40
Analysis ? Updating Rule
Let
If now
41
Analysis ? Updating Rule
Let
If now
42
Example
43
Unsupervised Learning Networks

• The Self-organizing Feature Map

44
Feature Mapping

• Map high-dimensional input signals onto a
  lower-dimensional (usually 1 or 2D) structure.
• Similarity relations present in the original data
  are still present after the mapping.

Dimensionality Reduction
Topology-Preserving Map
45
Somatotopic Map IllustrationThe Homunculus
The relationship between body surfaces and the
regions of the brain that control them.
46
Another Depiction of the Homunculus
47
Phonotopic maps
48
Phonotopic maps
humppila
49
Self-Organizing Feature Map

• Developed by professor Kohonen.
• One of the most popular neural network models.
• Unsupervised learning.
• Competitive learning networks.

50
The Structure of SOM
51
Example
52
Local Excitation, Distal Inhibition
53
Topological Neighborhood
Square
Hex
54
Size Shrinkage
55
Size Shrinkage
56
Learning Rule
Similarity Matching
Updating
57
Example
58
Example
59
Example
60
Example