Local structures; Causal Independence, Context-sepcific independance

1
Local structuresCausal Independence,Context-sep
cific independance

• COMPSCI 276
• Fall 2007

2
Reducing parameters of families

• Determinizm
• Causal independence
• Context-specific independanc
• Continunous variables

3
(No Transcript)
4
Causal Independence

• Event X has two possible causes A,B. It is hard
  to elicit P(XA,B) but it is easy to determine
  P(XA) and P(XB).
• Example several diseases causes a symptom.
• Effect of A on X is independent from the effect
  of B on X
• Causal Independence, using canonical models
• Noisy-O, Noisy AND, noisy-max

A
B
X
5
Binary OR
A
B
X
A
B
P(X0A,B)
P(X1A,B)
0
0
1
0
0
1
0
1
1
0
0
1
1
1
0
1
6
Noisy-OR

• noise is associated with each edge
• described by noise parameter ? ? 0,1
• Let q b0.2, qa 0.1
• P(x0a,b) (1-?a) (1-?b)
• P(x1a,b)1-(1-?a) (1-?b)

A
B
?a
?b
X
A
B
P(X0A,B)
P(X1A,B)
0
0
1
0
0
1
0.1
0.9
qiP(X0A_i1,else 0)
1
0
0.2
0.8
1
1
0.02
0.98
7
Noisy-OR with Leak

• Use leak probability ?0 ? 0,1 when both parents
  are false
• Let ?a 0.2, ?b 0.1, ?0 0.0001
• P(x0a,b) (1-?0)(1-?a)a(1-?b)b
• P(x0a,b)1-(1-?0)(1-?a)a(1-?b)b

A
B
?a
?b
X
A
B
P(X0A,B)
P(X1A,B)
0
0
0.9999
0.0001
0
1
0.1
0.9
1
0
0.2
0.8
1
1
0.02
0.98
8
Formal Definition for Noisy-Or

• Definition 1
• Let Y be a binary-valued random variable with k
  binary-valued parents X1,,Xk.
• The CPT P(YX1,Xk) is a noisy-or if there are
  k1 noise parameters ?0, ?1, ?k such that
• P(y0 X1,,Xk) (1- ?0) ? i,Xi1 (1- ?i)

9
Closed Form Bel(X) - 1
Given noisy-or CPT P(xu) noise parameters
?i Tu i Ui 1 Define qi 1 - ?I, Then
q_i is the probability that the inhibitor for
u_i is active while the
10
Closed Form Bel(X) - 2
Using Iterative Belief Propagation
Set piix pix (uk1). Then we can show that
11
Closed Form Bel(X) - 2
Using Iterative Belief Propagation
Set piix pix (uk1). Then we can show that
12
Causal Influence Defined
X1
X1
X1

• Definition 2
• Let Y be a random variable with k parents
  X1,,Xk.
• The CPT P(YX1,Xk) exhibits independence of
  causal influence (ICI) if it is described via a
  network fragment of the structure shown in on the
  left where CPT of Z is a deterministic functions
  f.

Z0
Z1
Z2
Zk
Z
Y
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Context Specific Independence

• When there is conditional independence in some
  specific variable assignment

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
The impact during inference

• Causal independence in polytrees is linear during
  inference
• Causal independence in general can sometime be
  exploited but not always
• CSI can be exploited by using operation (product
  and summation) over trees.

24
Representing CSI

• Using decision trees
• Using decision graphs

25
A students example
Intelligence
Difficulty
Grade
SAT
Apply
Letter
Job
26
Tree CPD

• If the student does not apply, SAT and L are
  irrelevant
• Tree-CPD for job

A
a0
a1
S
(0.8,0.2)
s1
s0
L
(0.1,0.9)
l1
l0
(0.9,0.1)
(0.4,0.6)
27
Definition of CPD-tree

• A CPD-tree of a CPD P(Zpa_Z) is a tree whose
  leaves are labeled by P(Z) and internal nodes
  correspond to parents branching over their values.

28
Captures irrelevant variables
C
c1
c2
L2
L1
l21
l20
l11
l10
(0.1,0.9)
(0.8,0.2)
(0.3,0.7)
(0.9,0.1)
29
Multiplexer CPD

• A CPD P(YA,Z1,Z2,,Zk) is a multiplexer iff
  Val(A)1,2,k, and
• P(YA,Z1,Zk)Z_a

30
Rule-based representation

• A CPD-tree that correponds to rules.

A
a0
a1
B
C
b1
b0
c1
c0
C
(0.1,0.9)
B
(0.2,0.8)
c1
c0
b1
b0
(0.3,0.7)
(0.4,0.6)
(0.3,0.7)
(0.5,0.5)
31
Continuous Variables

• ICS 275b
• 2002

32
Gaussian Distribution
33
(No Transcript)
34
(No Transcript)
35
Multivariate Gaussian

• Definition
• Let X1,,Xn. Be a set of random variables. A
  multivariate Gaussian distribution over X1,,Xn
  is a parameterized by an n-dimensional mean
  vector ? and an n x n positive definitive
  covariance matrix ?. It defines a joint density
  via

36
Linear Gaussian Distribution

• Definition
• Let Y be a continuous node with continuous
  parents X1,,Xk. We say that Y has a linear
  Gaussian model if it can be described using
  parameters ?0, ,?k and ?2 such that
• P(y x1,,xk)N (?0 ?1x1 ,?kxk ?2 )

37
(No Transcript)
38
(No Transcript)
39
Linear Gaussian Network

• Definition
• Linear Gaussian Bayesian network is a Bayesian
  network all of whose variables are continuous and
  where all of the CPTs are linear Gaussians.
• Linear Gaussian BN ? Multivariate Gaussian
• gtLinear Gaussian BN has a compact representation

40
Hybrid Models

• Continuous Node, Discrete Parents (CLG)
• Define density function for each instantiation of
  parents
• Discrete Node, Continuous Parents
• Treshold
• Sigmoid

41
Continuous Node, Discrete Parents

• Definition
• Let X be a continuous node, and let
  UU1,U2,,Un be its discrete parents and
  YY1,Y2,,Yk be its continuous parents. We say
  that X has a conditional linear Gaussian (CLG)
  CPT if, for every value u?D(U), we have a a set
  of (k1) coefficients au,0, au,1, , au,k1 and a
  variance ?u2 such that

42
CLG Network

• Definition
• A Bayesian network is called a CLG network if
  every discrete node has only discrete parents,
  and every continuous node has a CLG CPT.

43
Discrete Node, Continuous ParentsThreshold Model
44
Discrete Node, Continuous ParentsSigmoid
Binomial Logit
Definition Let Y be a binary-valued random
variable with k continuous-valued parents X1,Xk.
The CPT P(YX1Xk) is a linear sigmoid (also
called binomial logit) if there are (k1) weights
w0,w1,,wk such that
45
(No Transcript)
46
References

• Judea Pearl Probabilistic Reasoning in
  Inteeligent Systems, section 4.3
• Nir Friedman, Daphne Koller Bayesian Network and
  Beyond