Neural Network Applications Using an Improved Performance Training Algorithm

1
Neural Network Applications Using an Improved
Performance Training Algorithm

• Annamária R. Várkonyi-Kóczy 1, 2,
• Balázs Tusor 2
• 1 Institute of Mechatronics and Vehicle
  Engineering, Óbuda University
• 2 Integrated Intelligent Space Japanese-Hungarian
  Laboratory
• e-mail varkonyi-koczy_at_uni-obuda.hu

2
Outline

• Introduction, Motivation for using SC Techniques
• Neural Networks, Fuzzy Neural Networks, Circular
  Fuzzy Neural Networks
• The place and success of NNs
• A new training and clustering algorithms
• Classification examples
• A real-world application fuzzy hand posture and
  gesture detection system
• Inputs of the system
• Fuzzy hand posture models
• The NN based hand posture identification system
• Results
• Conclusions

3
Motivation for using SC Techniques We need
something non-classical Problems

• Nonlinearity, never unseen spatial and temporal
  complexity of systems and tasks
• Imprecise, uncertain, insufficient, ambiguous,
  contradictory information, lack of knowledge
• Finite resources ? Strict time requirements
  (real-time processing)
• Need for optimization
• Need for users comfort
• New challenges/more complex tasks to be solved ?
  more sophisticated solutions needed

4
Motivation for using SC Techniques We need
something non-classical Intentions

• We would like to build MACHINES to be able to do
  the same as humans do (e.g. autonomous cars
  driving in heavy traffic).
• We always would like to find an algorithm leading
  to an OPTIMUM solution (even when facing too
  much uncertainty and lack of knowledge)
• We would like to ensure MAXIMUM performance
  (usually impossible from every points of view,
  i.e. some kind of trade-off e.g. between
  performance and costs)
• We prefer environmental COMFORT (user friendly
  machines)

5
Need for optimization

• Traditionally
• optimization precision
• New definition (L.A. Zadeh)
• optimization cost optimization
• But what is cost!?
• precision and certainty also carry a cost

6
Users comfort
Human language Modularity, simplicity,
hierarchical structures Aims of the processing
preprocessing
processing
improving the performance of the
algorithms giving more support to the processing
(new)
aims of preprocessing
image processing / computer vision
noise smoothing feature extraction (edge, corner
detection) pattern recognition, etc. 3D
modeling, medical diagnostics, etc. automatic 3D
modeling, automatic ...
preprocessing
processing
7
Motivation for using SC Techniques We need
something non-classical Elements of the
Solution

• Low complexity, approximate modeling
• Application of adaptive and robust techniques
• Definition and application of the proper cost
  function including the hierarchy and measure of
  importance of the elements
• Trade-off between accuracy (granularity) and
  complexity (computational time and resource need)
• Giving support for the further processing
• These do not cope with traditional and AI
  methods, only with Soft Computing Techniques and
  Computational Intelligence

8
What is Computational Intelligence?

• Computer Intelligence

Increased computer facilities
Added by the new methods
L.A. Zadeh, Fuzzy Sets 1965 In traditional
hard computing, the prime desiderata are
precision, certainty, and rigor. By contrast, the
point of departure of soft computing is the
thesis that precision and certainty carry a cost
and that computation, reasoning, and decision
making should exploit whenever possible the
tolerance for imprecision and uncertainty.
9
What is Computational Intelligence?

• CI can be viewed as a consortium of methodologies
  which play important role in conception, design,
  and utilization of information/intelligent
  systems.
• The principal members of the consortium are
  fuzzy logic (FL), neuro computing (NC),
  evolutionary computing (EC), anytime computing
  (AC), probabilistic computing (PC), chaotic
  computing (CC), and (parts of) machine learning
  (ML).
• The methodologies are complementary and
  synergistic, rather than competitive.
• What is common Exploit the tolerance for
  imprecision, uncertainty, and partial truth to
  achieve tractability, robustness, low solution
  cost and better rapport with reality.

10
Soft Computing Methods (Computational
Intelligence) fulfill all of the five
requirements(Low complexity, approximate
modelingapplication of adaptive and robust
techniquesDefinition and application of the
proper cost function including the hierarchy and
measure of importance of the elementsTrade-off
between accuracy (granularity) and complexity
(computational time and resource need)Giving
support for the further processing)
11
Methods of Computational Intelligence

• fuzzy logic low complexity, easy build in of the
  a priori knowledge into computers, tolerance for
  imprecision, interpretability
• neuro computing - learning ability
• evolutionary computing optimization, optimum
  learning
• anytime computing robustness, flexibility,
  adaptivity, coping with the temporal
  circumstances
• probabilistic reasoning uncertainty, logic
• chaotic computing open mind
• machine learning - intelligence

12
Neural Networks

• It mimics the human brain
• (McCullogh Pitts, 1943, Hebb, 1949)
• Rosenblatt, 1958 (Perceptrone)
• Widrow-Hoff, 1960 (Adaline)

13
Neural Networks

• Neural Nets are parallel, distributed information
  processing tools which are
• Highly connected systems composed of identical or
  similar operational units evaluating local
  processing (processing element, neuron) usually
  in a well-ordered topology
• Possessing some kind of learning algorithm which
  usually means learning by patterns and also
  determines the mode of the information processing
• They also possess an information recall algorithm
  making possible the usage of the previously
  learned information

14
Application areas where NNs are successfully used

• One and multi-dimensional signal processing
  (image processing, speech processing, etc.)
• System identification and control
• Robotics
• Medical diagnostics
• Economical features estimation
• Associative memory content addressable memory

15
Application area where NNs are successfully used

• Classification system (e.g. Pattern recognition,
  character recognition)
• Optimization system (the usually feedback NN
  approximates the cost function) (e.g. radio
  frequency distribution, A/D converter, traveling
  salesman problem)
• Approximation system (any input-output mapping)
• Nonlinear dynamic system model (e.g. Solution of
  partial differential equation systems,
  prediction, rule learning)

16
Main features

• Complex, non-linear input-output mapping
• Adaptivity, learning ability
• distributed architecture
• fault tolerant property
• possibility of parallel analog or digital VLSI
  implementations
• Analogy with neurobiology

17
Classical neural nets

• Static nets (without memory, feedforward
  networks)
• One layer
• Multi layer
• MLP (Multi Layer Perceptron)
• RBF (Radial Basis Function)
• CMAC (Cerebellar Model Articulation Controller)
• Dynamic nets (with memory or feedback recall
  networks)
• Feedforward (with memory elements)
• Feedback
• Local feedback
• Global feedback

18
Feedforward architectures
One layer architectures Rosenblatt perceptron
19
Feedforward architectures
One layer architectures
Input
Output
Tunable parameters (weighting factors)
20
Feedforward architectures
Multilayer network (static MLP net)
21
Approximation property

• universal approximation property for some kinds
  of NNs
• Kolmogorov Any continuous real valued N
  variable function defined over the 0,1N compact
  interval can be represented with the help of
  appropriately chosen 1 variable functions and sum
  operation.

22
Learning

• Learning structure parameter estimation
• supervised learning
• unsupervised learning
• analytic learning
• Convergence??
• Complexity??

23
Supervised learning
estimation of the model parameters by x, y, d
n (noise)
x
d
Input
CC(e)
y
Parameter tuning
24
Supervised learning

• Criteria function
• Quadratic
• ...

25

• Minimization of the criteria
• Analytic solution (only if it is very simple)
• Iterative techniques
• Gradient methods
• Searching methods
• Exhaustive
• Random
• Genetic search

26
Parameter correction

• Perceptron
• Gradient methods
• LMS (least means square algorithm)
• ...

27
Fuzzy Neural Networks

• Fuzzy Neural Networks (FNNs)
• based on the concept of NNs
• numerical inputs
• weights, biases, outputs fuzzy numbers

28
Circular Fuzzy Neural Networks (CFNNs)

• based on the concept of FNNs
• topology realigned to a circular shape
• connection between the hidden and input layers
  trimmed
• the trimming done depends on the input data
• e.g., for 3D coordinates, each coordinate can be
  connectedto only 3 neighboring hidden layer
  neurons
• dramatic decrease in the required
• training time

29
Classification

• Classification the most important unsupervised
  learning problem it deals with finding a
  structure in a collection of unlabeled data
• Clustering assigning a set of objects into
  groups whose members are similar in some way and
  are dissimilar to the objects belonging to
  other groups (clusters)
• (usually iterative) multi-objective optimization
  problem
• Clustering is a main task of explorative data
  mining, statistical data analysis used in machine
  learning, pattern recognition, image analysis,
  information retrieval, bioinformatics, etc.
• Difficult problem multi-dimensional spaces,
  time/data complexity, finding an adequate
  distance measure, non-unambiguous interpretation
  of the results, overlapping of the clusters, etc.

30
The Training and Clustering Algorithms

• Goal
• To further increase the speed of the training of
  the ANNs used for classification
• Idea
• During the learning phase, instead of directly
  using the training data the data should be
  clustered and the ANNs should be trained by using
  the centers of the obtained clusters

• u input
• u centers of the appointed clusters
• y output of the model
• d desired output
• c value determinedby the criteria function

31
The Algorithm of the Clustering Step (modified
K-means alg.)
32
The ANNs

• Feedforward MLP, BP algorithm
• Number of neurons 2-10-2
• learning rate 0.8
• momentum factor 0.1
• Teaching set 500 samples, randomly chosen from
  the clusters
• Test set 1000 samples, separately generated

33
Examples Problem 1

• Easily solvable problem
• 4 classes, no overlapping

34
The Resulting Clusters and Required Training Time
in the First Experiment with Clustering Distances
A 0.05, B 0.1, and C 0.25
Clustering distance Clustering distance Time Spent on Training (minsec) Quantity of Appointed Clusters Quantity of Appointed Clusters Quantity of Appointed Clusters Quantity of Appointed Clusters S
Unclustered Unclustered 207 (100) 113 127 127 133 500
Clustered A1 200 (94,5) 30 30 8 14 82
Clustered B1 153 (89) 11 13 3 4 31
Clustered C1 053 (41,7) 3 2 1 1 7

• (First experiment)

35
Comparison between the Results of the Training
using the Clustered and the Cropped Datasets of
the 1st Experiment
Clustering distance Clustering distance Accuracy of the Training Accuracy of the Training Decrease in quality Decrease in Required Time
Clustered A1 1000/1000 100 no decrease 5.5
Clustered B1 1000/1000 100 no decrease 11
Clustered C1 1000/1000 100 no decrease 58.3
Cropped A1 1000/1000 100 no decrease 18
Cropped B1 1000/1000 100 no decrease 62.99
Cropped C1 965/1000 96.5 3.5 decrease 63.78
36
Examples Problem 2

• Moderately hard problem
• 4 classes, slight overlapping

37
The Resulting Clusters and Required Training Time
in the Second Experiment with Clustering
Distances A 0.05, B 0.1, and C 0.25
Clustering distance Clustering distance Time Spent on Training (hourminsec) Quantity of Appointed Clusters Quantity of Appointed Clusters Quantity of Appointed Clusters Quantity of Appointed Clusters S
Unclustered Unclustered 33802 (100) 127 125 137 111 500
Clustered A2 04451 (20,57) 28 31 14 2 78
Clustered B2 01135 (5,31) 11 10 5 2 28
Clustered C2 00300 (1,38) 2 3 1 1 7
38
Comparison between the Results of the Training
using the Clustered and Cropped Datasets of the
2nd Experiment
Clustering distance Clustering distance Accuracy of the Training Accuracy of the Training Decrease in Accuracy Decrease in Required Time
Clustered A2 997/1000 99.7 0.3 79.43
Clustered B2 883/1000 88.3 11.7 94.69
Clustered C2 856/1000 85.6 14.4 98.62
Cropped A2 834/1000 83.4 16.6 96.32
Cropped B2 869/1000 86.9 13.1 96.49
Cropped C2 834/1000 83.4 16.6 96.68
39
Comparison of the Accuracy and Training Time
Results of the Clustered and Cropped Cases of the
2nd Experiment
Group Decrease in Accuracy Decrease in Accuracy Decrease in Required Time Decrease in Required Time
Group Clustered Cropped Clustered Cropped
A2 A2 0.3 16.6 79.43 96.32
B2 B2 11.7 13.1 94.69 96.49
C2 C2 14.4 16.6 98.62 96.68
40
Examples Problem 3

• Hard problem
• 4 classes, significant overlapping

41
The Resulting Clusters and Required Training Time
in the Third Experiment with Clustering Distances
A 0.05, B 0.1, and C 0.2
Clustering distance Clustering distance Time Spent on Training (minsec) Quantity of Appointed Clusters Quantity of Appointed Clusters Quantity of Appointed Clusters Quantity of Appointed Clusters S
Unclustered Unclustered N/A 127 125 137 111 500
Clustered 0.05 5229 28 30 33 6 97
Clustered 0.1 2413 12 10 12 3 37
Clustered 0.2 735 3 4 4 1 12
42
Comparison between the Results of the Training
using the Clustered and Cropped Datasets of the
3rd Experiment
Clustering distance Clustering distance Accuracy of the Training Accuracy of the Training Decrease in quality
Clustered A3 956/1000 95.6 4.4
Clustered B3 858/1000 85.8 14.2
Clustered C3 870/1000 87 13
Cropped A3 909/1000 90.9 9.1
Cropped B3 864/1000 86.4 13.6
Cropped C3 773/1000 77.3 22.7
43
Comparison of the Accuracy Results of the
Clustered and Cropped Cases of the 3rd Experiment
Group Decrease in quality Decrease in quality
Group Clustered Cropped
A3 A3 4.4 9.1
B3 B3 14.2 13.6
C3 C3 13 22.7
44
Examples Problem 4

• easy problem
• 4 classes, no overlapping

d 0.2 0.1
0.05

The original dataset
The trained networks classifying ability
45
Clustering Distance Clustering Distance Accuracy on the test set Number of samples Required time for training Relative speed increase
Original Original 100 500 2 minutes 38 seconds -
Clustered 0.2 89.8 7 6 seconds 96
Clustered 0.1 95.6 21 22 seconds 86
Clustered 0.05 99.7 75 44 seconds 72
Cropped 0.2 96.8 7 5 seconds 96.8
Cropped 0.1 97.1 21 11 seconds 93
Cropped 0.05 98.7 75 23 seconds 85
46
Clustering Distance Clustering Distance Accuracy of the training on the original training set Accuracy in percentage Accuracy of the training on the test set Accuracy in percentage
Clustered 0.2 450/500 90 898/1000 89.8
Clustered 0.1 481/500 96.2 956/1000 95.6
Clustered 0.05 499/500 99.8 997/1000 99.7
Cropped 0.2 447/500 89.4 898/1000 89.8
Cropped 0.1 488/500 97.6 971/1000 97.1
Cropped 0.05 498/500 99.6 987/1000 98.7
47
Accuracy/training time
Clustering Distance Clustered Cropped Clustered to cropped relation
0.2 89.8 89.8 equals
0.1 95.6 97.1 1.5 better
0.05 99.7 98.7 1 better
Clustering Distance Clustered Cropped Clustered to cropped relation
0.2 6 seconds 5 seconds 16.6 slower
0.1 22 seconds 11 seconds 50 slower
0.05 44 seconds 23 seconds 47.7 slower
48
Examples Problem 5

• Moderately complex problem
• 3 classes, with some overlapping
• The network could not learn the original training
  data with the same options

d 0.2 0.1
0.05

The original dataset
49
Clustering Distance Clustering Distance Accuracy on the original training set Number of clusters Required time for training
Clustered 0.2 80.6 16 35 seconds
Clustered 0.1 91 44 1 minute 47 seconds
Clustered 0.05 95.2 134 17 minutes 37 seconds
Cropped 0.2 80.2 16 32 seconds
Cropped 0.1 93.4 44 1 minute 20 seconds
Cropped 0.05 91.4 134 1 hour 50 minutes 9 seconds
50
Clustering Distance Clustering Distance Accuracy of the training on the original training set Accuracy in percentage Accuracy of the training on the test set Accuracy in percentage
Clustered 0.2 403/500 80.6 888/1000 88.8
Clustered 0.1 455/500 91 977/1000 97.7
Clustered 0.05 476/500 95.2 971/1000 97.1
Cropped 0.2 401/500 80.2 884/1000 88.4
Cropped 0.1 467/500 93.4 974/1000 97.4
Cropped 0.05 457/500 91.4 908/1000 90.8
51
Clustering Distance Clustered Cropped Clustered to cropped relation
0.2 35 seconds 32 seconds 8.6 slower
0.1 1 minute 47 seconds 1 minute 20 seconds 25 slower
0.05 17 minutes 37 seconds 1 hour 50 minutes 9 seconds 625 faster
52
A Real-World Application Man-machine cooperation
in ISpace

• Man-machine cooperation in ISpace using visual
  (hand posture and gesture based) communication
• Stereo-camera system
• Recognition of hand gestures/ hand tracking and
  classification of hand movements
• 3D computation of feature points /3D model
  building
• Hand model identification
• Interpretation and execution of instructions

53
The Inputs The 3D coordinate model of the
detected hand

• The method uses two cameras
• From two different viewpoint
• The method works in the following way
• It locates the areas in the pictures of the two
  cameras where visible human skin can be detected
  using histogram back projection
• Then it extracts the feature points in the back
  projected picture considering curvature extrema
• peaks and
• valleys
• Finally, the selected feature points are matched
  in a stereo image pair.

The results The 3D coordinate model of the hand,
15 spatial points
54
Fuzzy Hand Posture Models

• describing the human hand by fuzzy hand feature
  sets
• theoretically 314 different hand postures

• 1st set four fuzzy features describing the
  distance between the fingertips of
• each adjacent finger (How far are finger X
  and finger Y from each other?)
• 2nd set five fuzzy features describing the
  bentness of each finger
• (How big is the angle between the lowest joint
  of finger W and the plane of the palm?)
• 3rd set five fuzzy features describing the
  relative angle between the bottom finger
• joint and the plane of the palm of the given
  hand (How bent is finger Z?)

55
Fuzzy Hand Posture Models
Example Victory
Feature group Feature Value
Relative distance between adjacent fingers a Large
Relative distance between adjacent fingers b Medium
Relative distance between adjacent fingers c Small
Relative distance between adjacent fingers d Small
Relative angle between the lowest joint of each finger and the plane of the palm A Medium
Relative angle between the lowest joint of each finger and the plane of the palm B Small
Relative angle between the lowest joint of each finger and the plane of the palm C Small
Relative angle between the lowest joint of each finger and the plane of the palm D Large
Relative angle between the lowest joint of each finger and the plane of the palm E Large
Relative bentness of each finger A Medium
Relative bentness of each finger B Large
Relative bentness of each finger C Large
Relative bentness of each finger D Small
Relative bentness of each finger E Small
56
Fuzzy Hand Posture and Gesture Identification
System

• ModelBase
• GestureBase
• Target Generator
• Circular Fuzzy Neural Networks (CFNNs)
• Fuzzy Inference Machine (FIM)
• Gesture Detector

57
Fuzzy Hand Posture and Gesture Identification
System

• ModelBase

Stores the features of the models as linguistic
variables

• GestureBase

Contains the predefined hand gestures as
sequences of FHPMs
58
Fuzzy Hand Posture and Gesture Identification
System

• Target Generator

Input parameters
Calculates the target parameters for the CFNNs
and the FIM.
d - identification value (ID) of the model in
the ModelBase. SL - linguistic variable for
setting the width of the triangular fuzzy sets
59
Fuzzy Hand Posture and Gesture Identification
System

• Fuzzy Inference Machine (FIM)
• Max (Min(ßi)) ßi - intersection
  of the fuzzy feature sets
• Gesture Detector

Identifies the detected FHPMs by using fuzzy
min-max algorithm
Searches predefined hand gesture patterns in the
sequence of detected hand postures
60
Circular Fuzzy Neural Networks (CFNNs)

• 3 different NNs for the 3 feature groups
• 15 hidden layer neurons
• 4/5 output layer neurons
• 45 inputs ( 15 coordinate triplets)
• but only 9 inputs connected to each
• hidden neuron

Convert the coordinate model to a FHPM
61
The Experiments

• Six hand models
• Separate training and testing sets
• Training parameters
• Learning rate 0.8
• Coefficient of the momentum method 0.5
• Error threshold 0.1
• SL small
• 3 experiments
• First and second experiments compare the speed of
  the training using the clustered and the original
  unclustered data and the accuracy of the trained
  system
• for given clustering distance (0.5)
• Third experiment compares the necessary training
  time and the accuracy of the trained system for
  different clustering distances

• The first two experiments have been conducted on
  an average PC (Intel Pentium 4 CPU 3.00 GHz, 1
  GB RAM, Windows XPSP3 operating system), while
  the third experiment has been conducted on
  another PC (Intel CoreTM 2 Duo CPU T5670 1.80
  GHz, 2 GB RAM, Windows 7 32-bit operating
  system).

62
Experimental Results The Result in Required
Training Time
Network type Time Required for Error Threshold Intervals Time Required for Error Threshold Intervals Time Required for Error Threshold Intervals Time Required for Error Threshold Intervals
Network type 0.5-0.25 0.25-0.2 0.2-0.15 0.15-0.12
Unclustered A 28 mins 39 minutes 1 hour17 minutes 2 hour 24 minutes
Unclustered B 50 mins 2 hours 14 minutes 2 hours 14 minutes 2 hour 28 minutes
Unclustered C 53 mins 52 minutes 52 minutes 2 hour 40 minutes
Clustered A 16 minutes (42.86) 25 minutes (35.9) 1 hour 14 minutes (3.9) 1 hour 18 minutes (45.8)
Clustered B 32 minutes (36) 1 hour 3 minutes (52.9) 1 hour 3 minutes (52.9) 1 hour 1 minutes (58.8)
Clustered C 31 minutes (41.5) 46 minutes (11.5) 46 minutes (11.5) 58 minutes (63.75)

• (First experiment)

63
Experimental Results Another Training Session
with only One Session
Network type Error Threshold Intervals Speed Increase
Network type 0.5-0.12 Speed Increase
Unclustered A 4 hours and 27 minutes 51.6
Clustered A 2 hours and 9 minutes 51.6
Unclustered B 3 hours and 8 minutes 27.1
Clustered B 2 hour and 22 minutes 27.1
Unclustered C 4 hours and 5 minutes 18
Clustered C 3 hours and 21 minutes 18

• (Second experiment)

64
Experimental Results Comparative Analysis of the
Result of the Trainings of the Two Sessions
Measured attribute Input data for the training Input data for the training Difference in ratio
Measured attribute Unclustered Clustered Difference in ratio
First Experiment Total time spent on Training 14 hours and 38 minutes 8 hours and 8 minutes 44.4 decrease
First Experiment Classification accuracy 98.125 95.2 2.9 decrease
Second Experiment Total time spent on training 11 hours and 41 minutes 7 hour and 52 minutes 32.5 decrease
Second Experiment Classification accuracy 98.125 95.83 2.3 decrease
65
Experimental Results The quantity of Clusters
Resulting from Multiple Clustering Steps for
Different Clustering Distances
Clustering distance Number of clusters for each hand type Number of clusters for each hand type Number of clusters for each hand type Number of clusters for each hand type Number of clusters for each hand type Number of clusters for each hand type Number of clusters for each hand type
Clustering distance Open hand Fist Three Point Thumb-up Victory S
Unclustered 20 20 20 20 20 20 120
d 0.5 10 13 4 7 4 5 42
d 0.4 13 16 5 9 5 8 55
d 0.35 13 17 5 12 10 8 65

• (Third experiment)

66
Experimental Results Comparative Analysis about
the Characteristics of the Differently Clustered
Data Sets
Measured attribute Measured value Difference in ratio
Unclustered Total time spent on training 6 hours and 30 minutes -
Unclustered Average classification accuracy 97 -
d 0.5 Total time spent on training 3 hour and 57 minutes 39 decrease
d 0.5 Average classification accuracy 95.2 1.8 decrease
d 0.4 Total time spent on training 4 hour and 22 minutes 32.8 decrease
d 0.4 Average classification accuracy 97 0 decrease
d 0.35 Total time spent on training 5 hour and 46 minutes 11.1 decrease
d 0.35 Average classification accuracy 97 0 decrease

• (Third experiment)

67
Experimental Results Clustered Data Sets
Hand posture type Clustering distance Clustering distance Clustering distance Clustering distance
Hand posture type UC 0.5 0.4 0.35
Open hand 77/80 76/80 76/80 76/80
Fist 72/80 77/80 76/80 76/80
Three 78/80 74/80 79/80 80/80
Point 80/80 77/80 78/80 79/80
Thumb-up 80/80 78/80 80/80 78/80
Victory 79/80 75/80 77/80 77/80
Average (in ratio) 97 95.2 97 97
Number of correctly classified samples / number
of all samples

• (Third experiment)

68
References to the examples

• Tusor, B. and A.R. Várkonyi-Kóczy, Reduced
  Complexity Training Algorithm of Circular Fuzzy
  Neural Networks, Journal of Advanced Research in
  Physics, 2012.
• Tusor, B., A.R. Várkonyi-Kóczy, I.J. Rudas, G.
  Klie, G. Kocsis, An Input Data Set Compression
  Method for Improving the Training Ability of
  Neural Networks, In CD-ROM Proc. of the 2012
  IEEE Int. Instrumentation and Measurement
  Technology Conference, I2MTC2012, Graz, Austria,
  May 13-16, 2012, pp. 1775-1783.
• Tóth, A.A., Várkonyi-Kóczy, A.R., A New Man-
  Machine Interface for ISpace Applications,
  Journal of Automation, Mobile Robotics
  Intelligent Systems, Vol. 3, No. 4, pp. 187-190,
  2009.
• Várkonyi-Kóczy, A.R., B. Tusor, Human-Computer
  Interaction for Smart Environment Applications
  Using Fuzzy Hand Posture and Gesture Models,
  IEEE Trans. on Instrumentation and Measurement,
  Vol. 60, No 5, pp. 1505-1514, May 2011.

69
Conclusions

• SC and NN based methods can offer solution for
  many unsolvable cases however with a burden of
  convergence and complexity problems
• New training and clustering procedures which can
  advantageously be used in the supervised training
  of neural networks used for classification
• Idea reduce the quantity of the training sample
  set in a way that does little (or no) impact on
  its training ability
• Clustering based on the k-means method with the
  main difference in the assignment step, where the
  samples are assigned to the first cluster that is
  near enough.
• As a result, for classification problems, the
  complexity of the training algorithm (and thus
  the training time) of neural networks can
  significantly be reduced
• Open questions
• dependency of the decrease of classification
  accuracy and training time of different types of
  ANNs
• optimal clustering distance
• generalization of the method towards other types
  of NNs, problems, etc.