CSE P573 Applications of Artificial Intelligence Bayesian Learning

1
CSE P573Applications of Artificial
IntelligenceBayesian Learning

• Henry Kautz
• Autumn 2004

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
Classify instance D as
11
(No Transcript)
12
Naive Bayes Classifier

• Important special, simple of a Bayes optimal
  classifier, where
• hypothesis classification
• all attributes are independent given the class

class
attrib. 1
attrib. 3
attrib. 2
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
Expectation-Maximization

• Consider learning a naïve Bayes classifier using
  unlabeled data. How can we estimate e.g. P(AC)?
• Initialization randomly assign numbers to P(C),
  P(AC), P(BC)
• repeat
• E-step Compute P(CA,B)
• M-step Re-compute maximum likelihood
  estimation of P(C), P(AC), P(BC)
• Calculate log likelihood of data
• until (likelihood of data not improving)

23
Expectation-Maximization

• Initialization randomly assign numbers to P(C),
  P(AC), P(BC).

24
Expectation-Maximization

• E-step Compute P(CA,B)

25
Expectation-Maximization

• M-step Re-compute maximum likelihood estimation
  of P(C), P(AC), P(BC)

26
Expectation-Maximization

• Calculate log likelihood of data

27
EM Demo
28
(No Transcript)
29
(No Transcript)