Neural Networks with Orthonormal Basis Dendrites

1
Neural Networks with Orthonormal Basis Dendrites

• Angelos Barmpoutis
• Computer Inf. Science Department
• University of Florida
• Gainesville, FL 32611, USA
• abarmpou_at_cise.ufl.edu

2
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

3
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

4
Motivation

• Dendrites, and not neurons, are the elementary
  computing devices of the brain, capable of
  implementing logical functions

5
Motivation

• Morphological Neural Networks (MNN)
• Solve some difficult non-linear problems
• Employed in applications for
• Face and object localization Grana2001,
  Raducanu2001
• Auto-Associative memories Ritter2004,
  Sussner2004
• Color images retrieval and restoration Yun2004,
  Zhan2003

6
Motivation

• Morphological Neural Networks (MNN)
• Solve some difficult non-linear problems
• Disadvantages
• Large size of neural networks formed
• Box patterns are visible in decision boundaries

7
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

8
Morphological Neurons with Dendrites

• How does a dendrite work?
• Lattice algebra operators max, min
• Computation performed by
• the kth dendrite

9
Morphological Neurons with Dendrites

• The diagram of a MNN with Dendrites

10
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

11
Orthonormal Basis Dendrites

• How does an Orthonormal Basis Dendrite work?

12
Orthonormal Basis Dendrites

• Computation performed by an OB dendr.
• Previously we had
• Local orientation information

13
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

14
Training

• Need to estimate for each dendrite
• Weights
• Orthonormal basis
• Find such Orthonormal basis that maximizes the
  volume of hyperbox
• Use any minimization algorithm

15
Training

• Training Algorithm

Algorithm I Input N training samples Xi, and N
outputs di 0 or 1 for class C1 or C2
respectively. i1,,N Output The number of
generated dendrites L, their weights Wj and their
orthonormal basis Rj. j1,,L Step 1 Find the
smallest possible hyperbox containing all the
samples of C1, and assign the appropriate values
W1 and R1 to the first dendrite. Set L1. Step
2 If there are misclassified points of C2, pick
arbitrarily a misclassified point ? and go to
step 3. Step 3 Find the biggest hyperbox that
contains ?, but it does not contain any point of
C1. Set LL1. Assign the appropriate values to
WL and RL. Go to step 2.
16
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

17
Experiments

• 2D spiral problem

Hyper boxes formed by regular Morphological
dendrites
Hyper boxes formed by Orthonormal Basis dendrites
18
Experiments

• 2D spiral problem

Decision boundaries formed by regular
Morphological dendrites
Decision boundaries formed by Orthonormal Basis
dendrites
19
Experiments

• 2D spiral problem
• Classification Errors and number of dendrites
  needed
• 1538 testing samples

20
Experiments

• Random points forming an ellipse

Ground truth decision boundary
MNN OBMNN Number of Dendrites
needed
21
Experiments

• Flower Irish Data base
• Classification Errors

22
Overview

• Motivation
• Morphological Neurons with Dendrites
• Orthonormal Basis Dendrites
• Training
• Experiments
• Conclusion

23
Conclusion

• Orthonormal basis Dendrites
• Different basis
• Better decision boundaries
• Smaller number of dendrites needed
• Local orientation information
• Further work
• Speed up the training process
• Fuzzy Orthonormal Basis Dendrites
• Use in face and object localization,
  Auto-Associative memories, color images retrieval
  and restoration etc

24
Neural Networks with Orthonormal Basis Dendrites
Thank you!

• Angelos Barmpoutis
• Computer Inf. Science Department
• University of Florida
• Gainesville, FL 32611, USA
• abarmpou_at_cise.ufl.edu