Classification - PowerPoint PPT Presentation

About This Presentation
Title:

Classification

Description:

Title: PowerPoint Presentation Last modified by: emcom Created Date: 1/1/1601 12:00:00 AM Document presentation format: (4:3) – PowerPoint PPT presentation

Number of Views:73
Avg rating:3.0/5.0
Slides: 26
Provided by: ackr
Category:

less

Transcript and Presenter's Notes

Title: Classification


1
Classification
  • Heejune Ahn
  • SeoulTech
  • Last updated 2015. May. 03

2
Outline
  • Introduction
  • Purpose, type, and an example
  • Classification design
  • Design flow
  • Simple classifier
  • Linear discriminant functions
  • Mahalanobis distance
  • Bayesian classification
  • K-means clustering unsupervised learning

3
1.Pupose
  • Purpose
  • For decision making
  • Topics of Pattern recognition (in artificial
    intelligence)
  • Model
  • Automation and Human intervention
  • Task specification what classes, what features
  • Algorithm to used
  • Training tuning algorithm parameters

Classifier (classification rules)
Features (patterns, structures)
classes
Images
4
2. Supervised vs unsupervised
  • Supervised (classification)
  • trained by examples (by humans)
  • Unsupervised (clustering)
  • only by feature data
  • using the mathematical properties (statistics) of
    data set

5
3. An example
  • Classifying nuts

Pine-nuts Lentils Pumpkin seeds
Features (circularity, line-fit-error)
Classifier (classification rules)
pine nut
lentil
pumpkin seed
6
  • Observations
  • What if a single features used?
  • What for the singular points?
  • Classification
  • draw boundaries

7
  • Terminalogy

8
4. Design Flow
9
5. Prototypes min-distance classifier
  • Prototypes
  • mean of training samples in each class

 
 
10
(No Transcript)
11
6. Linear discriminant
  • Linear discriminant function
  • g(x1,x2) ax1 bx2 c 0
  • Ex 11.1 Fig11.6

12
(No Transcript)
13
8. Mahalanobis distance
  • Problems In min-dist.
  • mean-value only, no distribution considered
  • e.g. (right figure)
  • std(class 1) ltlt std(class 2)
  • Mahalanobis dist.

Variance considered. (larger variance, less
distance)
14
9. Bayesian classification
  • Idea
  • To assign each data to the most-probable class,
    based on apriori-known probability
  • Assumption
  • Priors (probability for class) are known.
  • Bayes theorem

15
10. Bayes decision rule
  • Classification rule

Intuitively
Bayes Theorem
Class-conditional probability density function
Prior probability
Total probability Not used in classification
decision
16
  • Interpretation
  • Need to know priors and class-conditional pdf
    often not available
  • MVN (multivariate normal) distribution model
  • Practically quite good approximation
  • MVN
  • N-D Normal distribution with

17
12. Bayesian classifier for M-varirates
taking log( ) It is monotonic increasing
function
18
  • Case 1 identical independent
  • Linear Machine the decision region is
    hyper-plane (linears)
  • Note when same prob(w), then Minimum distance
    criterion

19
  • Case 2 all covariance is same
  • Matlab
  • class, err  classify(test, training, group,
    type, prior)
  • training and test
  • Type DiagLinear for naïve Baysian

20
  • Ex11.3

wrong priors
correct priors
21
13. Ensemble classifier
  • Combining multiple classifiers
  • Utilizing diversity, similar to ask multiple
    experts for decision.
  • AdaBoost
  • Weak classifier change (1/2) lt accuracy ltlt 1.0
  • weighting mis-classified training data for next
    classifiers

22
  • AdaBoost in details
  • Given
  • Initialize weight
  • For t 1, . . ., T
  • WeakLearn, which return the weak classifier
    with minimum
    error w.r.t. distribution Dt
  • Choose
  • Update


  • Where Zt is a normalization factor chosen
    so that Dt1 is a distribution
  • Output the strong classifier

23
14. K-means clustering
  • K-means
  • Unsupervised classification
  • Group data to minimize
  • Iterative algorithm
  • (re-)assign Xis to class
  • (re-)calculate ci
  • Demo
  • http//shabal.in/visuals/kmeans/3.html

24
  • Issues
  • Sensitive to initial centroid values.
  • Multiple trials needed gt choose the best one
  • K ( of clusters) should be given.
  • Trade-off in K (bigger) and the objective
    function (smaller)
  • No optimal algorithm to determine it.
  • Nevertheless
  • used in most of un-supervised clustering now.

25
  • Ex11.4 F11.10
  • kmeans function
  • classIndexes, centers kmeans(data, k,
    options)
  • k of clusters
  • Options Replicates', Display
Write a Comment
User Comments (0)
About PowerShow.com