Discriminative%20Training%20of%20Chow-Liu%20tree%20Multinet%20Classifiers

1
Discriminative Training of Chow-Liu tree Multinet
Classifiers

• Huang, Kaizhu
• Dept. of Computer Science and Engineering,
• CUHK

2
Outline

• Background
• Classifiers
• Discriminative classifiers
• Generative classifiers
• Bayesian Multinet Classifiers
• Motivation
• Discriminative Bayesian Multinet Classifiers
• Experiments
• Conclusion

3
Discriminative Classifiers

• Directly maximize a discriminative function

4
Generative Classifiers

• Estimate the distribution for each class, and
  then use Bayes rule to perform classification

5
Comparison
Example of Missing Information
From left to right Original digit, Cropped and
resized digit, 50 missing digit, 75 missing
digit, and occluded digit.
6
Comparison (Continue)

• Discriminative Classifiers cannot deal with
  missing information problems easily.
• Generative Classifiers provide a principled way
  to handle missing information problems.
• When is missing, we can use Marginalized P1
  and P2 to perform classification

7
Handling Missing Information Problem
SVM
TJT a generative model
8
Motivation

• It seems that a good classifier should combine
  the strategies of discriminative classifiers and
  generative classifiers
• Our work trains the one of the generative
  classifier the generative Bayesian Multinet
  classifier in a discriminative way

9
Roadmap of our work

10
How our work relates to other work?
Jaakkola and Haussler NIPS98
Difference Our method performs a reverse
process From Generative classifiers to
Discriminative classifiers
Beaufays etc., ICASS99, Hastie etc., JRSS 96
Difference Our method is designed for Bayesian
Multinet Classifiers, a more general classifier.
11
(No Transcript)
12
Problems of Bayesian Multinet Classifiers
Comments This framework discards the divergence
information between classes.
13
Our Training Scheme
14
Mathematic Explanation

• Bayesian Multinet Classifiers (BMC)
• Discriminative Training of BMC

15
Mathematic Explanation
16
Finding P1 and P2
17
Finding P1 and P2
18
Experimental Setup

• Datasets
• 2 benchmark datasets from UCI machine learning
  repository
• Tic-tac-toe
• Vote
• Experimental Environments
• PlatformWindows 2000
• Developing tool Matlab 6.5

19
Error Rate
20
Convergence Performance
21
Conclusion

• A discriminative training procedure for
  generative Bayesian Multinet Classifiers is
  presented
• This approach improves the recognition rate for
  two benchmark datasets significantly
• The theoretic exploration on the convergence
  performance of this approach is on the way.