Associative Learning

1
Associative Learning
2
Simple Associative Network
3
Banana Associator
4
Unsupervised Hebb Rule
5
Banana Recognition Example
6
Example
7
Problems with Hebb Rule

• Weights can become arbitrarily large
• There is no mechanism for weights to decrease

8
Hebb Rule with Decay
9
Example Banana Associator
10
Example
11
Problem of Hebb with Decay
12
Instar (Recognition Network)
13
Instar Operation
14
Vector Recognition
15
Instar Rule
16
Graphical Representation
17
Example
18
Training
19
Further Training
20
Kohonen Rule
21
Outstar (Recall Network)
22
Outstar Operation
23
Outstar Rule
24
Example - Pineapple Recall
25
Definitions
26
Iteration 1
27
Convergence
28
Boltzmann Learning

• Stochastic learning process with a recurrent
  structure
• State of a neuron is 1 or 1 and some neurons
  are free (adaptive state) and others are clamped
  (frozen state)
• Boltzmann machine is characterized by an energy
  function
• Free neurons change state with probability
• The learning rule is given by
• Where rkj is the correlation with neurons in
  clamped states and r-kj is the correlation with
  the neurons in a frozen state

Clamped