PGM: Tirgul 10 Learning Structure I

1
PGM Tirgul 10Learning Structure I
2
Benefits of Learning Structure

• Efficient learning -- more accurate models with
  less data
• Compare P(A) and P(B) vs. joint P(A,B)
• Discover structural properties of the domain
• Ordering of events
• Relevance
• Identifying independencies ? faster inference
• Predict effect of actions
• Involves learning causal relationship among
  variables

3
Why Struggle for Accurate Structure?
Adding an arc
Missing an arc

• Increases the number of parameters to be fitted
• Wrong assumptions about causality and domain
  structure

• Cannot be compensated by accurate fitting of
  parameters
• Also misses causality and domain structure

4
Approaches to Learning Structure

• Constraint based
• Perform tests of conditional independence
• Search for a network that is consistent with the
  observed dependencies and independencies
• Pros Cons
• Intuitive, follows closely the construction of
  BNs
• Separates structure learning from the form of the
  independence tests
• Sensitive to errors in individual tests

5
Approaches to Learning Structure

• Score based
• Define a score that evaluates how well the
  (in)dependencies in a structure match the
  observations
• Search for a structure that maximizes the score
• Pros Cons
• Statistically motivated
• Can make compromises
• Takes the structure of conditional probabilities
  into account
• Computationally hard

6
Likelihood Score for Structures

• First cut approach
• Use likelihood function
• Recall, the likelihood score for a network
  structure and parameters is
• Since we know how to maximize parameters from now
  we assume

7
Likelihood Score for Structure (cont.)

• Rearranging terms
• where
• H(X) is the entropy of X
• I(XY) is the mutual information between X and Y
• I(XY) measures how much information each
  variables provides about the other
• I(XY) ? 0
• I(XY) 0 iff X and Y are independent
• I(XY) H(X) iff X is totally predictable given
  Y

8
Likelihood Score for Structure (cont.)

• Good news
• Intuitive explanation of likelihood score
• The larger the dependency of each variable on its
  parents, the higher the score
• Likelihood as a compromise among dependencies,
  based on their strength

9
Likelihood Score for Structure (cont.)

• Bad news
• Adding arcs always helps
• I(XY) ? I(XY,Z)
• Maximal score attained by fully connected
  networks
• Such networks can overfit the data ---
  parameters capture the noise in the data

10
Avoiding Overfitting

• Classic issue in learning.
• Approaches
• Restricting the hypotheses space
• Limits the overfitting capability of the learner
• Example restrict of parents or of parameters
• Minimum description length
• Description length measures complexity
• Prefer models that compactly describes the
  training data
• Bayesian methods
• Average over all possible parameter values
• Use prior knowledge

11
Bayesian Inference

• Bayesian Reasoning---compute expectation over
  unknown G
• Assumption Gs are mutually exclusive and
  exhaustive
• We know how to compute P(xM1G,D)
• Same as prediction with fixed structure
• How do we compute P(GD)?

12
Posterior Score
Using Bayes rule P(D) is the same for all
structures G Can be ignored when comparing
structures
Prior over structures
Marginal likelihood
Probability of Data
13
Marginal Likelihood

• By introduction of variables, we have that
• This integral measures sensitivity to choice of
  parameters

14
Marginal Likelihood Binomial case

• Assume we observe a sequence of coin tosses.
• By the chain rule we have
• recall that
• where NmH is the number of heads in first m
  examples.

15
Marginal Likelihood Binomials (cont.)

• We simplify this by using
• Thus

16
Binomial Likelihood Example

• Idealized experiment with P(H) 0.25

-0.6
-0.7
-0.8
-0.9
(log P(D))/M
-1
Dirichlet(.5,.5)
-1.1
Dirichlet(1,1)
-1.2
Dirichlet(5,5)
-1.3
0
5
10
15
20
25
30
35
40
45
50
M
17
Marginal Likelihood Example (cont.)

• Actual experiment with P(H) 0.25

-0.6
-0.7
-0.8
-0.9
(log P(D))/M
-1
Dirichlet(.5,.5)
-1.1
Dirichlet(1,1)
-1.2
Dirichlet(5,5)
-1.3
0
5
10
15
20
25
30
35
40
45
50
M
18
Marginal Likelihood Multinomials

• The same argument generalizes to multinomials
  with Dirichlet prior
• P(?) is Dirichlet with hyperparameters ?1,,?K
• D is a dataset with sufficient statistics N1,,NK
• Then

19
Marginal Likelihood Bayesian Networks

• Network structure determines form ofmarginal
  likelihood

20
Marginal Likelihood Bayesian Networks

• Network structure determines form ofmarginal
  likelihood

21
Idealized Experiment

• P(X H) 0.5
• P(Y HX H) 0.5 p P(Y HX T) 0.5 - p

-1.3
-1.35
-1.4
-1.45
(log P(D))/M
-1.5
-1.55
-1.6
Independent
-1.65
P 0.05
P 0.10
-1.7
P 0.15
-1.75
P 0.20
-1.8
1
10
100
1000
M
22
Marginal Likelihood for General Network

• The marginal likelihood has the form
• where
• N(..) are the counts from the data
• ?(..) are the hyperparameters for each family
  given G

Dirichlet Marginal Likelihood For the sequence of
values of Xi when Xis parents have a particular
value
23
Priors

• We need prior counts ?(..) for each network
  structure G
• This can be a formidable task
• There are exponentially many structures

24
BDe Score

• Possible solution The BDe prior
• Represent prior using two elements M0, B0
• M0 - equivalent sample size
• B0 - network representing the prior probability
  of events

25
BDe Score

• Intuition M0 prior examples distributed by B0
• Set ?(xi,paiG) M0 P(xi,paiG B0)
• Note that paiG are not the same as the parents of
  Xi in B0.
• Compute P(xi,paiG B0) using standard inference
  procedures
• Such priors have desirable theoretical properties
• Equivalent networks are assigned the same score

26
Bayesian Score Asymptotic Behavior

• Theorem If the prior P(? G) is well-behaved,
  then
• Proof
• For the case of Dirichlet priors, use Stirlings
  approximation to ?( )
• General case, defer to incomplete data section

27
Asymptotic Behavior Consequences

• Bayesian score is consistent
• As M ?? the true structure G maximizes the
  score (almost surely)
• For sufficiently large M, the maximal scoring
  structures are equivalent to G
• Observed data eventually overrides prior
  information
• Assuming that the prior assigns positive
  probability to all cases

28
Asymptotic Behavior

• This score can also be justified by the Minimal
  Description Length (MDL) principle
• This equation explicitly shows the tradeoff
  between
• Fitness to data --- likelihood term
• Penalty for complexity --- regularization term

29
Scores -- Summary

• Likelihood, MDL, (log) BDe have the form
• BDe requires assessing prior network.It can
  naturally incorporate prior knowledge and
  previous experience
• BDe is consistent and asymptotically equivalent
  (up to a constant) to MDL
• All are score-equivalent
• G equivalent to G ? Score(G) Score(G)

30
Optimization Problem

• Input
• Training data
• Scoring function (including priors, if needed)
• Set of possible structures
• Including prior knowledge about structure
• Output
• A network (or networks) that maximize the score
• Key Property
• Decomposability the score of a network is a sum
  of terms.

31
Learning Trees

• Trees
• At most one parent per variable
• Why trees?
• Elegant math
• we can solve the optimization problem
  efficiently(with a greedy algorithm)
• Sparse parameterization
• avoid overfitting while adapting to the data

32
Learning Trees (cont.)

• Let p(i) denote the parent of Xi, or 0 if Xi has
  no parents
• We can write the score as
• Score sum of edge scores constant

33
Learning Trees (cont)

• Algorithm
• Construct graph with vertices 1, 2,
• Set w(i?j) be Score( Xj Xi ) - Score(Xj)
• Find tree (or forest) with maximal weight
• This can be done using standard algorithms in
  low-order polynomial time by building a tree in a
  greedy fashion(Kruskals maximum spanning tree
  algorithm)
• Theorem This procedure finds the tree with
  maximal score
• When score is likelihood, then w(i?j) is
  proportional to I(Xi Xj) this is known as the
  Chow Liu method

34
Learning Trees Example
Not every edge in tree is in the the
original network Tree direction is arbitrary ---
we cant learn about arc direction

• Tree learned fromalarm data
• correct arcs
• spurious arcs

35
Beyond Trees

• When we consider more complex network, the
  problem is not as easy
• Suppose we allow two parents
• A greedy algorithm is no longer guaranteed to
  find the optimal network
• In fact, no efficient algorithm exists
• Theorem Finding maximal scoring network
  structure with at most k parents for each
  variables is NP-hard for k gt 1

36
Heuristic Search

• We address the problem by using heuristic search
• Define a search space
• nodes are possible structures
• edges denote adjacency of structures
• Traverse this space looking for high-scoring
  structures
• Search techniques
• Greedy hill-climbing
• Best first search
• Simulated Annealing
• ...

37
Heuristic Search (cont.)

• Typical operations

Add C ?D
Reverse C ?E
Delete C ?E
38
Exploiting Decomposability in Local Search

• Caching To update the score of after a local
  change, we only need to re-score the families
  that were changed in the last move

39
Greedy Hill-Climbing

• Simplest heuristic local search
• Start with a given network
• empty network
• best tree
• a random network
• At each iteration
• Evaluate all possible changes
• Apply change that leads to best improvement in
  score
• Reiterate
• Stop when no modification improves score
• Each step requires evaluating approximately n new
  changes

40
Greedy Hill-Climbing Possible Pitfalls

• Greedy Hill-Climbing can get struck in
• Local Maxima
• All one-edge changes reduce the score
• Plateaus
• Some one-edge changes leave the score unchanged
• Happens because equivalent networks received the
  same score and are neighbors in the search space
• Both occur during structure search
• Standard heuristics can escape both
• Random restarts
• TABU search

41
Equivalence Class Search

• Idea
• Search the space of equivalence classes
• Equivalence classes can be represented by PDAGs
  (partially ordered graph)
• Benefits
• The space of PDAGs has fewer local maxima and
  plateaus
• There are fewer PDAGs than DAGs

42
Equivalence Class Search (cont.)

• Evaluating changes is more expensive
• These algorithms are more complex to implement

Original PDAG
New PDAG
Add Y---Z
Consistent DAG
Score
43
Learning in Practice Alarm domain
2
True Structure/BDe M' 10
Unknown Structure/BDe M' 10
1.5
1
KL Divergence
0.5
0
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
M
44
Model Selection

• So far, we focused on single model
• Find best scoring model
• Use it to predict next example
• Implicit assumption
• Best scoring model dominates the weighted sum
• Pros
• We get a single structure
• Allows for efficient use in our tasks
• Cons
• We are committing to the independencies of a
  particular structure
• Other structures might be as probable given the
  data

45
Model Averaging

• Recall, Bayesian analysis started with
• This requires us to average over all possible
  models

46
Model Averaging (cont.)

• Full Averaging
• Sum over all structures
• Usually intractable---there are exponentially
  many structures
• Approximate Averaging
• Find K largest scoring structures
• Approximate the sum by averaging over their
  prediction
• Weight of each structure determined by the Bayes
  Factor

The actual score we compute
47
Search Summary

• Discrete optimization problem
• In general, NP-Hard
• Need to resort to heuristic search
• In practice, search is relatively fast (100 vars
  in 10 min)
• Decomposability
• Sufficient statistics
• In some cases, we can reduce the search problem
  to an easy optimization problem
• Example learning trees