Bayesian Nets and Applications

1
Bayesian Nets and Applications

• Todays Reading C. 14
• Next class machine learning
• C. 18.1, 18.2
• Questions on the homework?

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Why is this useful?

• Useful for assessing diagnostic probability from
  causal probability
• P(causeeffect) P(effectcause)P(cause)
  P(effect)
• Let M be meningitus, S be stiff
  neckP(ms)P(sm)P(m) 0.8 X 0.0001 0.0008
  P(s) 0.1
• Note posterior probability of meningitus is
  still very small!

9
Naïve Bayes

• What happens if we have more than one piece of
  evidence?
• If we can assume conditional independence
• Overslept and trafficjam are independent, given
  late
• P(lateoverslept ? trafficjam) aP(overslept ?
  trafficjam)late)P(late) aP(overslept)late)
  P(trafficjamlate)P(late)
• Naïve Bayes where a single cause directly
  influences a number of effects, all conditionally
  independent
• Independence often assumed even when not so

10
Bayesian Networks

• A directed acyclic graph in which each node is
  annotated with quantitative probability
  information
• A set of random variables makes up the network
  nodes
• A set of directed links connects pairs of nodes.
  If there is an arrow from node X to node Y, X is
  a parent of Y
• Each node Xi has a conditional probability
  distributionP(XiParents(Xi) that quantifies the
  effect of the parents on the node

11
Example

• Topology of network encodes conditional
  independence assumptions

12
Hard working
Smart
Good test taker
Understands material
Exam Grade
Homework Grade
13
Smart Smart
True False
.5 .5
Hard Working Hard Working
True False
.7 .3
Hard working
Smart
Good test taker
Understands material
S Good Test Taker Good Test Taker
S True False
True .75 .25
False .25 .75
S HW UM UM
S HW True False
True True .95 .05
True False .6 .4
False True .6 .4
False False .2 .8
Exam Grade
Homework Grade
14
Conditional Probability Tables
Smart Smart
True False
.5 .5
Hard Working Hard Working
True False
.7 .3
S Good Test Taker Good Test Taker
S True False
True .75 .25
False .25 .75
S HW UM UM
S HW True False
True True .95 .05
True False .6 .4
False True .6 .4
False False .2 .8
GTT UM Exam Grade Exam Grade Exam Grade Exam Grade Exam Grade
GTT UM A B C D F
True True .7 .25 .03 .01 .01
True False .3 .4 .2 .05 .05
False True .4 .3 .2 .08 .02
False False .05 .2 .3 .3 .15
Homework Grade Homework Grade Homework Grade Homework Grade Homework Grade
UM A B C D F
True .7 .25 .03 .01 .01
False .2 .3 .4 .05 .05
15
Compactness

• A CPT for Boolean Xi with k Boolean parents has
  2k rows for the combinations of parent values
• Each row requires one number p for Xitrue (the
  number for Xifalse is just 1-p)
• If each variable has no more than k parents, the
  complete network requires O(nx2k) numbers
• Grows linearly with n vs O(2n) for the full joint
  distribution
• Student net 11225511 numbers (vs. 26-1)31

16
Conditional Probability
A general version holds for joint distributions
P(PlayerWins,HostOpensDoor1)P(PlayerWinsHostOpe
nsDoor1)P(HostOpensDoor1)
17
Global Semantics/Evaluation

• Global semantics defines the full joint
  distribution as the product of the local
  conditional distributionsP(x1,,xn)?in1P(xi
  Parents(Xi))e.g.,
• P(EGA?GT?UM?S?HW)

18
Global Semantics

• Global semantics defines the full joint
  distribution as the product of the local
  conditional distributionsP(X1,,Xn)?in1P(XiP
  arents(Xi))e.g., ObservationsS, HW, not UM,
  will I get an A?
• P(EGA?GT?UM?S?HW) P(EGAGT
  ?UM)P(GTS)P(UM HW ?S)P(S)P(HW)

19
Conditional Independence and Network Structure

• The graphical structure of a Bayesian network
  forces certain conditional independences to hold
  regardless of the CPTs.
• This can be determined by the d-separation
  criteria

20
a
c
Converging
a
b
b
b
Diverging
Linear
c
c
a
21
D-separation (opposite of d-connecting)

• A path from q to r is d-connecting with respect
  to the evidence nodes E if every interior node n
  in the path has the property that either
• It is linear or diverging and is not a member of
  E
• It is converging and either n or one of its
  decendents is in E
• If a path is not d-connecting (is d-separated),
  the nodes are conditionally independent given E

22
Hard working
Smart
Good test taker
Understands material
Exam Grade
Homework Grade
23

• S and EG are not independent given GTT
• S and HG are independent given UM

24
Medical Application of Bayesian
NetworksPathfinder
25
Pathfinder

• Domain hematopathology diagnosis
• Microscopic interpretation of lymph-node biopsies
• Given 100s of histologic features appearing in
  lymph node sections
• Goal identify disease type malignant
  or benign
• Difficult for physicians

26
Pathfinder System

• Bayesian Net implementation
• Reasons about 60 malignant and benign diseases of
  the lymph node
• Considers evidence about status of up to 100
  morphological features presenting in lymph node
  tissue
• Contains 105,000 subjectively-derived
  probabilities

27
(No Transcript)
28
Commercialization

• Intellipath
• Integrates with videodisc libraries of
  histopathology slides
• Pathologists working with the system make
  significantly more correct diagnoses than those
  working without
• Several hundred commercial systems in place
  worldwide

29
Sequential Diagnosis
30
Features

• Structured into a set of 2-10 mutually exclusive
  values
• Pseudofollicularity
• Absent, slight, moderate, prominent
• Represent evidence provided by a feature as
  F1,F2, Fn

31
Value of information

• User enters findings from microscopic analysis of
  tissue
• Probabilistic reasoner assigns level of belief to
  different diagnoses
• Value of information determines which tests to
  perform next
• Full disease utility model making use of life and
  death decision making
• Cost of tests
• Cost of misdiagnoses

32
(No Transcript)
33
(No Transcript)
34
Group Discrimination Strategy

• Select questions based on their ability to
  discriminate between disease classes
• For given differential diagnosis, select most
  specific level of hierarchy and selects questions
  to discriminate among groups
• Less efficient
• Larger number of questions asked

35
(No Transcript)
36
(No Transcript)
37
Other Bayesian Net Applications

• Lumiere Who knows what it is?

38
Other Bayesian Net Applications

• Lumiere
• Single most widely distributed application of BN
• Microsoft Office Assistant
• Infer a users goals and needs using evidence
  about user background, actions and queries
• VISTA
• Help NASA engineers in round-the-clock monitoring
  of each of the Space Shuttles orbiters subsystem
• Time critical, high impact
• Interpret telemetry and provide advice about
  likely failures
• Direct engineers to the best information
• In use for several years
• Microsoft Pregnancy and Child Care
• What questions to ask next to diagnose illness of
  a child

39
Other Bayesian Net Applications

• Speech Recognition
• Text Summarization
• Language processing tasks in general