Learning Bayesian Networks:

1
Learning Bayesian Networks

• Huajie Zhang

2
Definition

• Bayesian network (BN) directed graph and
  conditional probability tables
• nodes random variables
• arcs direct influence
• conditional probability tables quantifying
  the effects the parents have on the node.

3
Definition

• Bayesian network (BN) directed graph and
  conditional probability tables.

B E
.95 .94 .29 .01
4
Applications of Bayesian Networks

• Fault diagnosis
• Microsoft troubleshooting wizard
• Medical expert systems
• Pathfinder lymph-node disease
• Enterprise and business
• CRM
• NASA Vista
• Hand-crafted from domain experts.

5
Learning BN initial state
Alarm
Burglary
MaryCall
Earthquake
JohnCall
E A J M B
T F T F T F
T F F F F
T T F T

6
Structure learning
7
Parameter learning (1)
P(B) .001
P(E) .002
Burglary
Earthquake
B E
P(A)
.95 .94 .29 .01
T T T F F T F F
Alarm
A P(J) T .90 F .05
A P(M)
MaryCall
JohnCall
T .70 F .01
8
Parameter Learning (2)

• Learning parameters with incomplete data

E A J M B
T F ? F T F
? F F F F
T T ? T

• Learning parameters for continuous variables

9
Naïve Bayes
10
Tree-Augmented Naïve Bayes (TAN)
11
Tree-Augmented Naïve Bayes (TAN)
12
General Augmented Naïve Bayes
13
Why TAN ?

• Learning Naïve Bayes is trivial.
• Learning BN is NP-hard.
• TAN is a practical choice.
• Algorithms
• ChowLiu Friedman et al. 1997
• Superparents Koegh and Pazzani 1999

14
A Greedy Exhaustive Search Strategy for TAN

• Initial structure Naïve Bayes
• Add one arc at one time
• Evaluate current network by accuracy
• Time complexity n3

15
The Algorithm SuperParent (1)

• Initialize being Naïve Bayes

16
The Algorithm SuperParent (2)

• Find best SuperParent (1)

17
The Algorithm SuperParent (3)

• Find best SuperParent (2)

18
The Algorithm SuperParent (4)

• Find the best son for the SuperParent

19
The Algorithm SuperParent (5)

• Repeat the previous steps

20
The Algorithm SuperParent (6)
Time complexity n2
21
AUC-based SuperParent

• Extend SuperParent
• Find the best SuperParent and the best child in
  terms of AUC, instead of accuracy