Reasoning Under Uncertainty: Belief Networks

1
Reasoning Under Uncertainty Belief
Networks Computer Science cpsc322, Lecture
27 (Textbook Chpt 10.3) March, 17, 2008
2
Lecture Overview

• Recap
• Belief Networks
• Def and Example
• Intro Inference, Compactness, Semantics
• Construction
• More Examples

3
Recap

• Probability is a rigorous formalism for uncertain
  knowledge
• For nontrivial domains, we must find a way to
  reduce the joint distribution size
• Representation, reasoning and learning are
  exponential
• Solution Exploit Conditional independence

4
Lecture Overview

• Recap
• Belief Networks
• Def and Example
• Intro Inference, Compactness, Semantics
• Construction
• More Examples

5
Bayesian networks Def.

• A graphical notation for conditional independence
  assertions
• (and hence for compact specification of full
  joint distributions)
• Syntax
• a set of nodes, one per random variable
• a directed, acyclic graph (link "directly
  influences")
• a set of conditional probability tables for each
  variable given its parents
• P (Xi Parents (Xi))

6
Bayesian networks Def (cont)

• Topology of network encodes conditional
  independence assertions each variable is
  conditionally independent of its non-descendants,
  given its parent nodes

7
Bnets Burglary Example

• I have an anti-burglar alarm in my house
• I have an agreement with two of my neighbors,
  John and Mary, that they call me if they hear the
  alarm go off when I am at work
• Sometime they call me at work for other reasons
• Sometimes the alarm is set off by minor
  earthquakes.
• Variables
• Lets examine the network topology which reflects
  "causal" knowledge
• A burglar can set the alarm off
• An earthquake can set the alarm off
• The alarm can cause Mary to call
• The alarm can cause John to call

8
Burglary Example contd.
9
Lecture Overview

• Recap
• Belief Networks
• Def and Example
• Intro Inference, Compactness, Semantics
• Construction
• More Examples

10
Burglary Example Bnets inference

• (A) I'm at work,
• neighbor John calls to say my alarm is ringing,
• neighbor Mary doesn't call.
• No news of any earthquakes.
• Is there a burglar?
• (B) I'm at work,
• Receive message that neighbor John called ,
• News of minor earthquakes.
• Is there a burglar?

11
Bayesian Networks Inference Types
Update algorithms exploit dependencies to reduce
the complexity of probabilistic inference
12
Bnets Compactness

• A CPT for Boolean Xi with k Boolean parents has
  2k rows for the combinations of parent values
• Each row requires one number pi for Xi
  true(the number for Xi false is just 1-pi )
• If each variable has no more than k parents, the
  complete network requires O(n 2k) numbers
• For kltlt n, this is a substantial improvement,
• the numbers required grow linearly with n, vs.
  O(2n) for the full joint distribution
• For burglary net, 1 1 4 2 2 10
• numbers (vs. 25-1 31)

13
Bnets Semantics

• The full joint distribution is defined as the
  product of the local conditional distributions
• P (X1, ,Xn) pi 1 P(Xi X1, ,Xi-1)
  (chain rule)
• pi 1 P
  (Xi Parents(Xi))
• Because each variable is conditionally
  independent of all its ancestors, given its
  parent nodes
• e.g., P(j ? m ? a ? ?b ? ?e)
• P (j a) P (m a) P (a ?b, ?e) P (?b) P
  (?e)

n
n
14
Lecture Overview

• Recap
• Belief Networks
• Def and Example
• Intro Inference, Compactness, Semantics
• Construction
• More Examples

15
Constructing a belief network

• Given a set of random variables, a belief network
  can be constructed as follows
• Totally order the variables of interest X1, ,Xn
• Theorem of probability theory (chain rule)

• The parents pXi of Xi are those predecessors of
  Xi that render Xi independent of the other
  predecessors. That is, pXi ? X1, ,Xi-1 and
• P(Xi pXi) P(Xi X1, ,Xi-1)
• So

16
Construction Example Fire Diagnosis

• Suppose you want to diagnose whether there
• is a fire in a building
• you receive a noisy report about whether everyone
  is leaving the building.
• if everyone is leaving, this may have been
  caused by a fire alarm.
• if there is a fire alarm, it may have been caused
  by a fire or by tampering
• if there is a fire, there may be smoke

17
Construction Example Choose Vars

• First you choose the variables. In this case,
  all are Boolean
• Tampering is true when the alarm has been
  tampered with
• Fire is true when there is a fire
• Alarm is true when there is an alarm
• Smoke is true when there is smoke
• Leaving is true if there are lots of people
  leaving the building
• Report is true if the sensor reports that people
  are leaving the building

18
Construction Example Establish Independencies

• Next, you order the variables Fire Tampering
  Alarm Smoke Leaving Report.
• Now evaluate which variables are conditionally
  independent given their parents
• is Tampering independent of Fire (Would learning
  that one is true change your beliefs about the
  probability of the other?)
• Is Alarm conditionally independent on Fire
  given Tampering ? Or vice versa?

19
Construction Example Establish Independencies

• Order of the variables Fire Tampering Alarm
  Smoke Leaving Report.
• Is Smoke independent of Tampering and Alarm
  given whether there is a Fire?
• We assume Leaving is only caused by Alarm, and
  thus is independent of the other variables given
  Alarm.
• Report is caused by Leaving, and thus is
  independent of the other variables given Leaving.

20
Example Fire Diagnosis

• This corresponds to the following belief network

Of course, we're not done until we also come up
with conditional probability tables for each node
in the graph.
21
Lecture Overview

• Recap
• Belief Networks
• Def and Example
• Intro Inference, Compactness, Semantics
• Construction
• Electrical System Bnet

22
Example Electrical System as a Bnet

• The belief network specifies
• The domain of the variables, including
• W0,,W6 ? live,dead
• S1_pos, S2_pos, and S3_pos have domain
  up,down
• L1_st , L2_st have domain ok, intermittent,
  broken.
• S1_st has ok, upside_down, short, intermittent,
  broken.
• .
• Conditional independence assertions
• Conditional probabilities

23
Conditional Independence assertions Examples

• Whether l1 is lit (L1_lit) depends only on the
  status of the light (L1_st) and whether there is
  power in wire w0. Thus, L1_lit is independent of
  the other variables given L1_st and W0. In a
  belief network, W0 and L1_st are parents of
  L1_lit.

Similarly, W0 depends only on whether there is
power in w1, whether there is power in w2, and
the position of switch s2 (S2_pos)
24
Example Circuit Diagnosis

• The power network can be used in a number of
  ways
• Conditioning on the circuit breakers, whether
  there is outside power and the position of the
  switches, you can simulate the lighting.
• Given values for the switches, the outside power,
  and whether the lights are lit, you can determine
  the posterior probability that circuit breaker is
  ok or not.
• Given some switch positions and some outputs and
  some intermediate values, you can determine the
  probability of any other variable in the network.

25
Example Liver Diagnosis Source Onisko et al.,
1999
26
Belief network summary

• A belief network is a directed acyclic graph
  (DAG) that effectively expresses independence
  assertions among random variables.
• The parents of a node X are those variables on
  which X directly depends.
• Consideration of causal dependencies among
  variables typically help in constructing a Bnet

27
Next Class

• Bayesian Networks Inference