Artificial Neural Networks (ANNs) and the Error Backpropagation Procedure

1
Artificial Neural Networks (ANNs) and the Error
Backpropagation Procedure

• Prof. Carolina Ruiz
• Department of Computer Science
• Worcester Polytechnic Institute

2
A 2-layer feedforward ANN
Input hidden layer output
layer
-1
-1
-1
3
Error Backpropagation
A B Out
0 0 0
0 1 1
1 0 1
1 1 0
1. Initialize the weights to small random values
0.5
0.1
A
C
-0.2
-0.1
E
0.05
0.3
D
B
0.2
0.5
4
Error Backpropagation
A B Out
0 0 0
0 1 1
1 0 1
1 1 0
2. For each of the examples 2.1. Present
example to input layer 2.2. Propagate the
example forward
0.5
0
0.1
0.377
A
C
-0.2
-0.1
E
0.5094
0.05
0.3
D
0.377
B
0.2
0
0.5
5
Error Backpropagation
A B Out
0 0 0
0 1 1
1 0 1
1 1 0
2. For each of the examples 2.3. Compute
node errors for output layer 2.4. Compute
node errors for hidden layer
0.025
0.5
0
0.1
0.377
A
C
-0.2
-0.5094
-0.1
E
0.5094
0.05
0.3
D
0.377
B
0.2
0
-0.0382
0.5
6
Error Backpropagation
A B Out
0 0 0
0 1 1
1 0 1
1 1 0
2. For each of the examples 2.5. Compute
and record weight change for each
connection
0.025
0.5
0
0.1
0.377
A
C
-0.2
-0.5094
? A-gtC 0.0000
? A-gtD 0.0000
? B-gtC 0.0000
? B-gtD 0.0000
? C-gtE -0.0481
? D-gtE -0.0481
-0.1
E
0.5094
0.05
0.3
D
0.377
B
0.2
0
-0.0382
0.5
7
Error Backpropagation
A B Out
0 0 0
0 1 1
1 0 1
1 1 0
3. After processing all examples update weight 4.
Repeat process until obtaining good weights
0.025
0.5
0
0.1
0.377
A
C
-0.2
-0.5094
? A-gtC 0.0001
? A-gtD -0.0795
? B-gtC 0.0004
? B-gtD -0.0863
? C-gtE 0.3853
? D-gtE -0.049
-0.1
E
0.5094
0.05
0.3
D
0.377
B
0.2
0
-0.0382
0.5