Optimizing classrelated thresholds with Particle Swarm Optimization

About This Presentation

Title:

Optimizing classrelated thresholds with Particle Swarm Optimization

Description:

Optimizing class-related thresholds ... 1 Pontifical Catholic University of Parana, Curitiba, BRAZIL. 2 cole de Technologie Sup rieure, Montreal, CANADA. ... – PowerPoint PPT presentation

Number of Views:130

Avg rating:3.0/5.0

Slides: 18

Provided by: luiz95

Category:

more less

Transcript and Presenter's Notes

Title: Optimizing classrelated thresholds with Particle Swarm Optimization

1
Optimizing class-related thresholdswith Particle
Swarm Optimization
International Joint Conference on Neural Networks
2005 July 31-August 4, 2005 -Montreal, Canada

Luiz S. Oliveira1, Alceu S. Britto Jr1., Robert
Sabourin2
1 Pontifical Catholic University of Parana,
Curitiba, BRAZIL.
2 École de Technologie Supérieure, Montreal,
CANADA.

2
Introduction

Cascading classifiers have been quite used to
solve pattern recognition problems
Improvement of classification.
Reduction of the complexity.
The majority of patterns can be explained by a
simple rule
Well behaved patterns.

3
Introduction

Easy cases
They can be classified using a relatively small
portion of the features available.
Difficult cases
They use more sophisticated classifiers,
therefore more expensive.
More features or complex classifiers.

4
Cascading Classifier
Inputs rejected by the first stage are handled
by the next ones
Reject-option
5
Reject-option

The most used error-reject trade-off was given by
Chow (1970)

Rejected
Accepted
6
Reject-option

Chows rules provides the optimal error-reject
trade-off only if the posteriori probabilities
are known.
Which does not happen in most of cases.
Affected by significant estimate errors.
Multiple reject thresholds for the different data
classes Fumera et al

7
Reject-option with multiple thresholds.
Rejected

It has been demonstrated that the values of
thresholds T1Tn exist that the corresponding
accuracy A(T1Tn) is equal or higher than A(T)

Accepted
8
Optimization problem

Find N thresholds that maximize the accuracy for
a fixed error rate.
It can be formulated in terms of constrained
maximization problem as follows
PSO to solve this problem
Better results than those achieved by Fumera et
al.

9
A Cascading System

Generated through a feature selection algorithm
IWFHR04.
Base classifier
MLP, trained with 132 features extracted from
concavities and contour.
Handwritten digits (10 classes)
NIST SD19
195k, 28k, and 30k for training, validation, and
testing.
99.13 on test set at zero-rejection level.

10
A Cascading System

Feature selection method generated five
classifiers

Base 132 99.13
11
Performance

Using Fumeras algorithm to define the
thresholds. Error fixed at 0.5

78 of the patterns are classified in the
first level of the cascade
12
Performance

The computation effort can be measured in terms
of the number of feature-values
where n is the number of classifiers, mi is the
number of features used by the classifier i and
xi is the number of instances classified by the
classifier i

13
Performance

By computing TVF, we observe that the cascade can
reduce this index in about 75 compared to the
base classifier.
Performance at the same level
99.38 vs 99.13

14
Searching thresholds with PSO

Conventional PSO using the sociometric principle
gbest
Maximum velocity constraint.
10-Dimensional space.
Values ranging from 0 to 1
Size of the swarm 20.
c1 and c2 2
The inertia weight w was set to 0.9 and decreased
over the iterations to allow exploitation.

15
Performance using PSO

For an error rate fixed at 0.5

With PSO, 82 of the patterns were classified in
the first level of the cascade
16
Conclusions

Cascading classifier system is a very interesting
approach to reduce complexity.
It can perform even better when the reject option
is properly fine-tuned.
We have demonstrated trhough experimentation that
PSO is an efficient approach to deal with this
kind of problem.