Artificial Neural Networks

1
Artificial Neural Networks

• An Introduction

2
What is a Neural Network?

• A human Brain
• A porpoise brain
• The brain in a living creature
• A computer program
• Simulates (at a very rudimentary level) a
  biological brain
• Limited connections

3
Artificial Neural Networks

• Artificial neural networks are information
  technology inspired by studies of the brain and
  nervous system
• ANNs are used to simulate the massively parallel
  processes that are effectively used in the brain
  for learning, and storing information and
  knowledge

4
Biological Neuron

• Dendrites
• Axon
• Soma
• Membrane
• Synapse
• Neurotransmitter
• Spikes

5
Simple Neuron Configuration
6
Threshold Logic Units

• Outputs are 0 or 1
• If the activation (accumulated weighted input) is
  larger than threshold the unit generates a signal

7
Sigmoidal Transfer function
Outputs are in the range from 0 to
1 y1/(1exp(-a)) Is differentiable
8
Neural Network Architecture

• In feedforward NN, neurons are grouped into
  layers
• The neurons on each layer are the same type
• There are different types of layers
• Input layer receive input from external sources
• Output layer communicate to user
• Hidden layer(s) neurons communicate only with
  other layers

9
Sample Network Configuration
Hidden layer
Output layer
Input layer
10
Some Characteristics of ANN

• Tolerance to noise
• Reliability
• Two layer networks are restricted to linearly
  separable problems
• Additional layers can solve more complicated
  problems
• Black Box. Why? Non-linearity
• Logic hidden in weights
• Universal approximators.

11
Learning Methods

• Supervised
• Error Backpropagation
• Counter-Propagation
• Unsupervised
• Hebbs rule
• Competitive Learning
• Reinforcement

12
Error Backpropagation Algorithm

• Generalized Delta Rule
• Allowed training multi-layer ANN
• Revived interest in ANN
• Error terms are propagated back through the
  network
• The weight coefficients are updated iteratively

13
Error Backpropagation Algorithm Drawbacks

• Local Minima
• Biologically implausible
• Possibility of network paralysis
• Slowness
• Oscillations.

14
Problems solved by ANN

• Classification
• Cluster Analysis
• Approximation
• Forecasting
• Association
• Data compression

15
Benefits of ANN

• Parallelism
• Learning
• Generalization
• NN can learn the characteristics of a general
  category of objects on specific examples from
  that category
• Robustness (reliability)
• Tolerance to noise
• Performance does not degrade appreciably if some
  of its neurons or interconnections are lost
  (Distributed memory)

16
Limitations of ANN

• Two-layer NN limited to linearly separable
  problems
• Local minima oscillations
• Number of hidden layers/units hard to determine
• Lack of transparency (perspicuity)

17
Sample of Applications

• Business
• Credit scoring
• Bankruptcy prediction
• Bond rating
• Security trading
• Technological processes
• Robotics
• Consumer electronics