Artificial Intelligence

1
Artificial Intelligence

• Lecture 19
• Artificial Neural Network

2
Overview

• Artificial Neural Network Introduction
• Perceptron
• Gradient Descent
• Multilayer Networks
• Sigmoid Unit
• Backpropagation Algorithm

3
ALVINN
4
Artificial Models

• Properties of artificial neural nets (ANN)
• Many neuron-like threshold switching units
• Many weighted interconnections among units
• Highly parallel, distributed process
• Emphasis on tuning weights automatically

5
When to Consider ANN

• Input is high-dimensional discrete or real-valued
  (e.g. raw sensor input)
• Output is discrete or real valued
• Output is a vector of values
• Possibly noisy data
• Form of target function is unknown
• Human readability of result is unimportant
• Examples
• Speech recognition
• Image classification
• Financial prediction
• Learning robot control strategies

6
Perceptron
7
Perceptron
8
Decision Surface of a Perceptron
9
Perceptron Training Rule
10
Perceptron Training Rule

• Can prove it will converge
• If training data is linearly separable
• And ? sufficiently small

11
Gradient Descent
12
Gradient Descent
13
The Delta Rule
The gradient specifies the direction that
produces the steepest increase in E.
14
(No Transcript)
15
Gradient Descent
16
Multilayer Networks of Sigmoid Units
17
Sigmoid Unit
18
Error Gradient for a Sigmoid Unit
19
Backpropagation
20
Backpropagation Algorithm
Propagate input forward through the network
Propagate error backward through the network
21
More on Backpropagation
22
Summary

• Gradient Descent
• Multilayer Networks
• Sigmoid Unit
• Backpropagation Algorithm