Fainan May 2006 Pattern Classification and Machine Learning Course

1
The Use of NN in Classification
Architecture
Training

• Three layers Feedforward Neural Network (FFNN) is
  sufficient for realizing a broad class of
  input/output non-linear maps (Kolmogorovs
  theorem)
• Disadvantages
• number of neurons in the hidden layer cannot be
  determined
• number of neurons can be large implying
  expensive calculation

• Backpropagation Algorithm
• Disadvantages
• number of training epochs can not be determined
• local minima

2
Alternative NN Design Using Voronoi Diagrams

• Given two classes S1 and S2 and two features x1
  and x2
• S1 (4,0),(0,4) S2 (0,0),(4,4)

2 features ? two neurons at the first layer 2
classes ? two neurons at the output layer
Step 1 Draw convex hulls related to each class
Solution
3
x1-20
Step 2 Specify Hyperplanes
4-Veronoi cells ? 4 neurons at the hidden layer
x2-2 0

• Step 3
• Form a cluster corresponding to each class

C1 the cluster corresponding to class S1
C2 the cluster corresponding to class S2
4
Step 4 Now we are ready for the net synthesis
Number of neurons Activation function Bias vector Weight vector
Input layer The hyperplanes 2 Bipolar discrete (outputs either -1 or 1) -2 -2 Ones
Hidden layer AND function 4 Bipolar discrete (outputs either -1 or 1) -1.5 -1.5 -1.5 -1.5
Output layer OR function 2 Bipolar discrete (outputs either -1 or 1) 0.5 0.5 Ones
Specification
Layer
5
FFNN to solve non-linear classification problem
Ref. N. K. Bose, and A. K. Garga, Neural
Network Design Using Voronoi Diagrams, IEEE
trans. On Neural Networks, vol. 4, no. 5, Sept.
1993.