Title: Classification
1Classification
- Heejune Ahn
- SeoulTech
- Last updated 2015. May. 03
2Outline
- Introduction
- Purpose, type, and an example
- Classification design
- Design flow
- Simple classifier
- Linear discriminant functions
- Mahalanobis distance
- Bayesian classification
- K-means clustering unsupervised learning
31.Pupose
- Purpose
- For decision making
- Topics of Pattern recognition (in artificial
intelligence) - Model
- Automation and Human intervention
- Task specification what classes, what features
- Algorithm to used
- Training tuning algorithm parameters
Classifier (classification rules)
Features (patterns, structures)
classes
Images
42. Supervised vs unsupervised
- Supervised (classification)
- trained by examples (by humans)
- Unsupervised (clustering)
- only by feature data
- using the mathematical properties (statistics) of
data set
53. An example
Pine-nuts Lentils Pumpkin seeds
Features (circularity, line-fit-error)
Classifier (classification rules)
pine nut
lentil
pumpkin seed
6- Observations
- What if a single features used?
- What for the singular points?
- Classification
- draw boundaries
-
7 84. Design Flow
95. Prototypes min-distance classifier
- Prototypes
- mean of training samples in each class
10(No Transcript)
116. Linear discriminant
- Linear discriminant function
- g(x1,x2) ax1 bx2 c 0
-
- Ex 11.1 Fig11.6
12(No Transcript)
138. Mahalanobis distance
- Problems In min-dist.
- mean-value only, no distribution considered
- e.g. (right figure)
- std(class 1) ltlt std(class 2)
- Mahalanobis dist.
Variance considered. (larger variance, less
distance)
149. Bayesian classification
- Idea
- To assign each data to the most-probable class,
based on apriori-known probability - Assumption
- Priors (probability for class) are known.
- Bayes theorem
1510. Bayes decision rule
Intuitively
Bayes Theorem
Class-conditional probability density function
Prior probability
Total probability Not used in classification
decision
16- Interpretation
- Need to know priors and class-conditional pdf
often not available - MVN (multivariate normal) distribution model
- Practically quite good approximation
- MVN
- N-D Normal distribution with
1712. Bayesian classifier for M-varirates
taking log( ) It is monotonic increasing
function
18- Case 1 identical independent
- Linear Machine the decision region is
hyper-plane (linears) - Note when same prob(w), then Minimum distance
criterion
19- Case 2 all covariance is same
- Matlab
- class, err classify(test, training, group,
type, prior) - training and test
- Type DiagLinear for naïve Baysian
20wrong priors
correct priors
2113. Ensemble classifier
- Combining multiple classifiers
- Utilizing diversity, similar to ask multiple
experts for decision. - AdaBoost
- Weak classifier change (1/2) lt accuracy ltlt 1.0
- weighting mis-classified training data for next
classifiers
22- AdaBoost in details
- Given
- Initialize weight
- For t 1, . . ., T
- WeakLearn, which return the weak classifier
with minimum
error w.r.t. distribution Dt - Choose
- Update
-
- Where Zt is a normalization factor chosen
so that Dt1 is a distribution - Output the strong classifier
2314. K-means clustering
- K-means
- Unsupervised classification
- Group data to minimize
- Iterative algorithm
- (re-)assign Xis to class
- (re-)calculate ci
- Demo
- http//shabal.in/visuals/kmeans/3.html
24- Issues
- Sensitive to initial centroid values.
- Multiple trials needed gt choose the best one
- K ( of clusters) should be given.
- Trade-off in K (bigger) and the objective
function (smaller) - No optimal algorithm to determine it.
- Nevertheless
- used in most of un-supervised clustering now.
25- Ex11.4 F11.10
- kmeans function
- classIndexes, centers kmeans(data, k,
options) - k of clusters
- Options Replicates', Display