Baynesian Networks

1
Baynesian Networks

• J. M. Akinpelu

2
Conditional Probability

• If E and F are independent, then
• Law of Total Probability

3
Conditional Probability

• Bayes Theorem
• Chain Rule

4
Conditional Independence

• Let E, F, and G be events. E and F are
  conditionally independent given G if
• An equivalent definition is

5
Baynesian Networks

• A Baynesian network (BN) is a probabilistic
  graphical model that represents a set of
  variables and their independencies
• Formally, a BN is a directed acyclic graph (DAG)
  whose nodes represent variables, and whose arcs
  encode the conditional independencies between the
  variables

6
Baynesian Network - Example
family-out (fo)
bowel-problem (bp)
dog-out (do)
light-on (lo)
hear-bark (hb)
From Charniak
7
Bayesian Networks

• Over the last few years, a method of reasoning
  using probabilities, variously called belief
  networks, Bayesian networks, knowledge maps,
  probabilistic causal networks, and so on, has
  become popular within the AI community - from
  Chaniak
• Applications include medical diagnosis, map
  learning, language understanding, vision, and
  heuristic search. In particular, this method is
  playing an increasingly important role in the
  design and analysis of machine learning
  algorithms.

8
Baynesian Networks

• Two interpretations
• Causal
• BNs are used to model situations where causality
  plays a role, but our understanding is
  incomplete, so that we must describe things
  probabilistically
• Probabilistic
• BNs allow us to calculate the conditional
  probabilities of the nodes in a network given
  that some of the values have been observed.

9
Probabilities in BNs

• Specifying the probability distribution for a BN
  requires
• The prior probabilities of all the root nodes
  (nodes without parents)
• The conditional probabilities of all non-root
  nodes given all possible combinations of their
  direct parents
• BN representation can yield significant savings
  in the number of values needed to specify the
  probability distribution
• If variables are binary, then 2n?1 values are
  required for the complete distribution, where n
  is the number of variables

10
Probabilities in BNs - Example
P(fo) .15
P(bp) .01
family-out
bowel-problem
P(do fo bp) .99 P(do fo bp) .90 P(do
fo bp) .97 P(do fo bp) .3
dog-out
light-on
P(lo fo) .6 P(lo fo) .05
hear-bark
P(hb do) .7 P(hb do) .01
From Charniak
11
Calculating Probabilities - Example
What is the probability that the lights are
out? P(lo) P(lo fo) P(fo) P(lo fo)
P(fo) .6 (.15) .05 (.85)
0.1325
12
Calculating Probabilities - Example
What is the probability that the dog is
out? P(do) P(do bp fo) P(bp fo) P(do bp
fo) P(bp fo) P(do bp fo)
P(bp fo) P(do bp fo) P(bp fo)
P(do bp fo) P(bp) P(fo) P(do bp fo)
P(bp) P(fo) P(do bp fo) P(bp)
P(fo) P(do bp fo) P(bp) P(fo)
.99(.15)(.01) .90(.15)(.99) .97(.85)(.01)
.3(.85)(.99) 0.4
13
Types of Connections in BNs
14
Independence Assumptions

• Linear connection The two end variables are
  usually dependent on each other. The middle
  variable renders them independent.
• Converging connection The two end variables are
  usually independent of each other. The middle
  variable renders them dependent.
• Divergent connection The two end variables are
  usually dependent on each other. The middle
  variable renders them independent.

15
Inference in Bayesian Networks

• A basic task for BNs is to compute the posterior
  probability distribution for a set of query
  variables, given values for some evidence
  variables. This is called inference or belief
  updating.
• The input to a BN inference evaluation is a set
  of evidences e.g.,
• E hear-bark true, lights-on
  true
• The outputs of the BN inference evaluation are
  conditional probabilities
• P(Xi v
  E)
• where Xi is a variable in the network.

16
Inference in Bayesian Networks

• Types of inference
• Diagnostic
• Causal
• Intercausal (Explaining Away)
• Mixed

17
Diagnostic Inference

• Inferring the probability of a cause based on
  evidence of an effect
• Also known as bottom up reasoning

18
Example Diagnostic Inference

• Given that the dog is out, whats the probability
  that the family is out? That the dog has a bowel
  problem? Whats the probable cause of the dog
  being out?

19
Causal Inference

• Inferring the probability of an effect based on
  evidence of a cause
• Also known as top down reasoning

20
Example Causal Inference
What is the probability that the dog is out given
that the family is out? P(do fo) P(do fo
bp) P(bp) P(do fo bp) P(bp)
.99 (.01) .90 (.99)
0.90 What is the probability that the dog is out
given that he has a bowel problem? P(do bp)
P(do bp fo) P(fo) P(do bp fo) P(fo)
.99 (.15) .97 (.85)
0.973
21
Intercausal Inference (Explaining Away)

• Involves two causes that "compete" to "explain"
  an effect
• The causes become conditionally dependent given
  that their common effect is observed, even though
  they are marginally independent.

22
Explaining Away - Example
What is the probability that the family is out
given that the dog is out and has a bowel problem?
Evidence of the bowel problem explains away the
fact that the dog is out.
23
Mixed Inference

• Combines two or more diagnostic, causal, or
  intercausal inferences

24
Example A Lecturers Life

• Dr. Ann Nicholson spends 60 of her work time in
  her office. The rest of her work time is spent
  elsewhere. When Ann is in her office, half the
  time her light is off (when she is trying to hide
  from students and get some real work done). When
  she is not in her office, she leaves her light on
  only 5 of the time. 80 of the time she is in
  her office, Ann is logged onto the computer.
  Because she sometimes logs onto the computer from
  home, 10 of the time she is not in her office,
  she is still logged onto the computer.
• Draw the corresponding BN.
• Specify the conditional probabilities associated
  with each node.
• Suppose a student checks Dr. Nicholsons login
  status and sees that she is logged on. What
  effect does this have on the students belief
  that Dr. Nicholsons light is on.

25
Example A Lecturers Life
in office
P(io) .6
light on
computer on
P(co io) .8 P(co io) .1
P(lo io) .5 P(lo io) .05
26
Example A Lecturers Life
27
Example A Lecturers Life
28
Example Medical Diagnosis

• A patient presents to a doctor with shortness of
  breath. The doctor considers that possible causes
  are tuberculosis, lung cancer and bronchitis.
  Other additional information that is relevant is
  whether the patient has recently visited Asia
  (where tuberculosis is more prevalent), whether
  or not the patient is a smoker (which increases
  the chances of cancer and bronchitis). A positive
  X-ray would indicate either TB or lung cancer.
  (Example from (Lauritzen, 1988).)
• Draw the corresponding BN.
• Construct the probability of tuberculosis given
  that the patient has shortness of breath and a
  positive X-ray.
• Construct the probability that the patient has a
  positive X-ray given he is a smoker.

29
Example Medical Diagnosis
smoker
visited Asia
tuberculosis
bronchitis
lung cancer
positive X-ray
shortness of breath
30
Example Medical Diagnosis
31
Example Medical Diagnosis
32
Evaluating BNs

• Computation in Bayesian networks is NP-hard. All
  algorithms for computing the probabilities are
  exponential to the size of the network.
• There are two ways around the complexity barrier
• Algorithms for special subclass of networks,
  e.g., singly connected networks. (A singly
  connected network is one in which the underlying
  undirected graph has no more than one path
  between any two nodes.)
• Approximate algorithms.
• The computation for a singly connected network is
  linear to the size of the network.

33
In-class Exercise

• You have a new burglar alarm installed. It is
  reliable for detecting burglaries, but also
  responds to minor earthquakes. Two neighbors
  (John, Mary) promise to call you at work when
  they hear the alarm. John almost always calls
  when he hears the alarm, but confuses the alarm
  with phone ringing (and calls then too). Mary
  likes loud music and sometimes misses the alarm!
• Draw the BN.
• Using the information on the next chart
• Estimate the probability of an alarm.
• Given a burglary, estimate the probability of i.)
  a call from John ii.) a call from Mary.
• Estimate the probability of a burglary, given i.)
  a call from John ii.) a call from Mary.

34
In-class Exercise

• P(B) 0.01
• P(E) 0.02
• P(A B, E) 0.95
• P(A B, E) 0.94
• P(A B, E) 0.29
• P(A B, E) 0.001

• P(J A) 0.90
• P(J A) 0.05
• P(M A) 0.70
• P(M A) 0.01

35
References

• Eugene Charniak, Bayesian Networks without Tears,
  www.cs.ubc.ca/murphyk/Bayes/Charniak_91.pdf.
• BN Lecture, www.csse.monash.edu.au/annn/443/L2-4.
  ps.
• BN Lecture, www.cs.cmu.edu/awm/381/lec/bayesinfer
  /bayesinf.ppt.
• Judea Pearl, Causality Models, Reasoning, and
  Inference, Cambridge University Press, 2000.