Neural Networks Chapter 8

1
Neural NetworksChapter 8

• Joost N. Kok
• Universiteit Leiden

2
Unsupervised Learning

• No teacher, no feedback from the environment
• Discover patterns, features, regularities,
  correlations or categories in input data
• Redundancy provides knowledge

3
Unsupervised Learning

• Familiarity
• Principal Component Analysis
• Clustering
• Prototyping
• Encoding
• Feature Mapping

4
Unsupervised Learning

• Most networks consist of one layer
• Try plain Hebbian learning

5
Unsupervised Learning

• At equilibrium point weight changes average to 0

6
Unsupervised Learning

• All eigenvalues of C are positive or zero
• Direction of largest eigenvalue becomes dominant
• Weights w moves in direction of eigenvector with
  largest eigenvalue
• Trick renormalization of w after each update

7
Unsupervised Learning

• Ojas rule
• Properties
• Unit length
• Direction of maximal eigenvector of C
• Maximizes