Reasoning Under Uncertainty: Bnet Inference

1
Reasoning Under Uncertainty Bnet Inference
(Variable elimination) Computer Science cpsc322,
Lecture 29 (Textbook Chpt 10.4) March, 26, 2008
2
Lecture Overview

• Recap Bnets
• Bnets Inference
• Intro
• Factors
• Variable elimination Algo

3
Recap Bnets

• Why?
• How? Simplification relies on

4
Recap Bnets Example
5
Recap Bnets Entailed (in)dependencies
Indp(R,S,F)?
Indp(T,F,A)?
Indp(A,L,R)?
Indp(R,T,L)?
6
Lecture Overview

• Recap Bnets
• Bnets Inference
• Intro
• Factors
• Variable elimination Algo

7
Bnet Inference

• Our goal compute probabilities of variables in a
  belief network
• What is the posterior distribution over one or
  more variables, conditioned on one or more
  observed variables?

8
Bnet Inference Evidence

• If we want to compute the posterior probability
  of Z given evidence Y1 v1 ? ? Yj vj

• So the computation reduces to the probability of
  a conjunction
• P(Z,Y1 v1, ,Yj vj).
• We normalize at the end.

9
Lecture Overview

• Recap Bnets
• Bnets Inference
• Intro
• Factors
• Variable elimination Algo

10
Factors

• A factor is a representation of a function from a
  tuple of random variables into a number.
• We will write factor f on variables X1, ,Xj as
  f(X1, ,Xj).
• A factor denotes a distribution over the given
  tuple of variables in some (unspecified) context
• e.g., P(X1, X2) is a factor f(X1, X2)
• e.g., P(X1, X2, X3 v3) is a factor f(X1, X2)
• e.g., P(X1, X3 v3 X2) is a factor f(X1, X2)

11
Manipulating Factors

• We can make new factors out of an existing factor
• Our first operation we can assign some or all of
  the variables of a factor.
• f(X1 v1, X2, , Xj ), where v1 ? dom(X1), is a
  factor on X2, ,Xj.
• f(X1 v1, X2 v2, , Xj vj) is a number
  that is the value of f when each Xi has value
  vi
• The former is also written as f(X1, X2, , Xj)X1
  v1

12
Example of assignments
13
Factors Summing out variables

• Our second operation we can sum out a variable,
  say X1 with domain v1, ,vk , from factor
  f(X1, ,Xj), resulting in a factor on X2, ,Xj
  defined by

14
Summing out a variable example
15
Multiplying factors

• Our third operation factors can be multiplied
  together.
• The product of factor f1(X?, Y?) and f2(Y?,
  Z?), where Y? are the variables in common, is the
  factor (f1 f2)(X?, Y?, Z?) defined by

Note it's defined on all X?, Y?, Z? triples,
obtained by multiplying together the appropriate
pair of entries from f1 and f2 .
16
Multiplying factors example
17
Factors Summary

• A factor is a representation of a function from a
  tuple of random variables into a number.
• f(X1, ,Xj).
• We have defined three operations on factors
• Assigning one or more variables
• f(X1v1, X2, ,Xj) is a factor on X2, ,Xj ,
  also written as f(X1, , Xj)X1v1
• Summing out variables
• (?X1 f)(X2, ,Xj) f(X1v1, ,Xj) f(X1vk,
  ,Xj)
• Multiplying factors
• (f1 f2)(X?, Y?, Z?) f1(X?, Y?) f2 (Y?, Z?)

18
Lecture Overview

• Recap Bnets
• Bnets Inference
• Intro
• Factors
• Intro Variable elimination Algo

19
Variable Elimination Intro

• Suppose the variables of the belief network are
  X1,,Xn.
• Z is the query variable
• Y1v1, , Yjvj are the observed variables (with
  their values)
• Z1, ,Zk are the remaining variables
• What we want to compute

Example
20
Variable Elimination Intro

• The posterior distribution over one or more
  variables, conditioned on one or more observed
  variables can be computed as

This can be framed in terms of operations
between factors (that satisfy the semantics of
probability)
21
Variable Elimination Intro

• If we consider the joint as a factor, we can
  compute P(Z,Y1v1, ,Yjvj) by assigning Y1v1,
  , Yjvj and summing out the variables Z1, ,Zk
• We sum out these variables one at a time
• the order in which we do this is called our
  elimination ordering.

Are we done?
22
Variable Elimination Intro

• Using the chain rule and the definition of a
  belief network, we can write P(X1, , Xn) as
• Thus

Consider each CP as a factor Inference in
belief networks thus reduces to computing the
sums of products.
23
Lecture Summary
Computing P(Z Y1v1 ? ? Yjvj ) can be
reduced to computing P(Z ? Y1v1 ? ? Yjvj )
The computation of P(Z ? Y1v1 ? ? Yjvj )
can be expressed in terms of factors and basic
operations between factors

• Bnet inference as variable elimination
  (preliminary)
• Construct a factor for each conditional
  probability.
• In each factor assign the observed variables to
  their observed values.
• Multiply the factors and for each of the other
  variables Zi ? Z1, , Zk , sum out Zi

24
Next Class

• Finish Variable Elimination
• Simplify the computation
• Example