FUZZY BELIEF NETWORKS

1
FUZZY BELIEF NETWORKS

• APARNA PHADNIS
• SARA NASSER

2
FUZZY BELIEF NETWORKS

• A Presentation based on the paper,
• INFERENCE VIA FUZZY BELIEF NETWORKS,
• by Dr.Carl G. Looney Lily R.Liang

3
Overview of presentation

• Belief Networks and Applications
• Types of Belief Networks
• Bayesian Belief Networks and concepts
• Neighborhood updates
• Drawbacks of BBNs
• Fuzzy belief Networks and concepts
• Inferencing via Fuzzy Beliefs
• Running a Example
• Future work and conclusions

4
Why belief?
Invest money
Demand for Products
Establishes a belief Whether investing Would
earn him profits
Market
Earlier profits
5
Belief Networks

• What are belief networks?
• A combination of Probability theory and graph
  theory.
• Belief-uncertainty

6
Applications of Belief Networks

• Probabilistic modeling
• Space exploration
• Economics-Business Segmentation in Bank
• AI
• Speech Recognition
• Genetics
• Image Modeling
• Decision Making System
• Medical Expert system
• Data mining
• Path Finding
• Specific Known Uses
• Microsoft office text help
• Intel processor fault diagnosis (Intel)
• Generator monitoring expert system (General
  Electric)
• Troubleshooting (Microsoft)

7
Types of belief networks

• Bayesian Belief Networks (Bayes Nets)
• Fuzzy Belief Networks
• Neural-Logic Belief Network

8
Bayesian Belief Networks

• What are Bayesian Belief Networks?
• An acyclic directed graph, where nodes
  represent variables and arrows influences.
• Prior information (to infer anything from data)
• Involve some levels of uncertainty
• Probability theory is used to represent
  uncertainty

9
Graphical Representation
Parents (Root variables)- probability
distribution are independent of any other
variable states.
A
I
Parent
Immediate Dependence
Direct Influence
B
C
Child
Child
Child (dependant variables)-probability
distribution are dependent on some other
variables.
H
G
F
E
10
Concept of Bayesian Networks

• Each variable has a state, and the network state
  ensemble of nodal states
• The jpdf of the overall variables is calculated
  using conditional probability tables and prior
  probability tables
• Conditional Probability P(E A) P(E e, A
  a) / P(A a)
• 
  The probability of Ee given Aa.
• Joint Probability P (A, B) ? P (A a AND B
  b)
• The
  probability that both A a and B b
• Product rule of Probability P (A, B)
  P(Aa)P(Bb)
• 
  Variables A and B are independence each other

11
An Example
Prior Tables
Ta Fa 0.4 0.6
Tb Fb 0.8 0.2
B
A
Conditional tables
E TaTb TaFb FaTb FaFb Te 0.2 0.4
0.6 0.8 Fe 0.8 0.6 0.4 0.2
E
C
D
D Te Fe Td 0.2 0.6 Fd 0.8 0.4
C Te Fe Tc 0.7 0.3 Fc 0.3 0.7
Ta represents true of A and Fa represents false
of A
12
Example cont.

• jpdf of the example
• PA,B,C,D,EPA PB PEA,B PCE PDE
• Eg
• PTaFbFcTdTePTaPFbPTeTaFbPFcTePTdTe
  
• (0.4)(0.2)(0.4)(0.3)(0.2)0.000192

13
Inferencing as queries

• To query ETe ,Based on CTc
• Bayes Rule
• PETeCTcPCTcETePETe/PCTc
• Where PCTcETe0.7
• Marginal Probabilities of PCTc and PETe
  must be
• computed. To compute PETe we sum the
  probabilities of all 16 combinations of outcomes
  of ABCD where ETe is fixed.

14
Neighborhood Updates

• Deleted 1-nbhd
• 2-nbhd
• Prior probabilities
• Posterior probabilities
• Posterior updates
• Conditionally dependent (d-connected) and
  conditionally independent
• d-separated

15
neighborhood
If M is observed then the parents of M
are Conditionally dependent through M, while the
children are conditionally independent (blocked
by M).
B
A
E
2 nbhd
Prior Probabilities At any node are
conditionally depend on its parents only
D
C
Observed Node
Posterior updates due to a change at node N
must be done first at The 1-nbhd then 2-nbhd
So on.
Any posterior update on any link can only be in a
single direction
1st nbhd
The posterior probabilities at any Node N given
the observations are Conditionally dependent on
their 1-nbhd
16
Example

• Observations at C and D
• Update their 1-nbhd E,
• which depends conditionally on C and D
• Now can you update C D thru E?
• Next we can update the 2-nbhd of C,D ie A,B (we
  cant do the reverse)
• D and B are d-separated by E, i.e.,they are
  blocked by E( When E is observed B and D do not
  influence each other)
• When E is observed, A and B are conditionally
  dependent via E (d-connected)

17
Drawbacks of BBN

• NP-Hard
• Cannot be solved in polynomial time
• Exponential time For N nodes with say each
  having 2 states it takes 2N
• For the example we had 5 nodes so we had to
  compute the jpdf for a total of 2532 BN states

18
Why Fuzzy?

• Fuzzy sets and fuzzy logic were developed as a
  means for representing, manipulating, and
  utilizing uncertain information, and to provide a
  framework for handling uncertainties and
  imprecision in real-world applications.

19
Inferencing via Fuzzy Belief Networks

20
The Fuzzy Concept

• Fuzzy Logic Inferencing via FBNs
• -Based on Probabilistic Belief Rules
• -Fuzzy Influences
• -Fuzzy Processing
• -Fuzzy Belief Networks
• -Fuzzy Propagation

21
The Fuzzy Concept

• Fuzzy Logic Inferencing via FBNs
• -Based on Probabilistic Belief Rules
• -Fuzzy Influences
• -Fuzzy Processing
• -Fuzzy Belief Networks
• -Fuzzy Propagation

22
Fuzzy Logic

• Fuzzy Logic
• An approach to
• computing based on
• degrees of truth,
• rather than the usual
• true or false

• Characteristics
• Axiomatic
• Intuitive
• Allows simple common sense reasoning
• Influences can be tuned to fit the real world
  data

23
Probabilistic Belief Rules

• What are Probabilistic Belief Rules?
• Rules of the form Agt B that are modelled
  probabilistically.
• Axiomatic Structure

Empirical Evidence
Models of Equally Likely Elementary Outcomes
Basis of Fuzzy Belief Networks
24
Inferencing via FBN

• What is inferencing?
• Inferencing, is the process of
• updating fuzzy beliefs of outcomes based upon
• the relationships in the model
• the evidence known about the situation at hand.

25
Fuzzy Influences

• How do they work?
• Same as Bayesian probabilities,
• - can work in forward and backward
• direction to adjust the beliefs of the
• query when given the observed
• states.

26
Fuzzy Processing

• How are the influences processed using fuzzy
  logic?
• Fuzzy Processing establishes
• conditional belief relationships
• By Observing Nodes
• By Updating the fuzzy beliefs of their k-nbhds
  with fuzzy conditional influences(new evidence)

27
Fuzzy Belief Network

• Emulates a non-Bayesian
• probability measure
• Provides a mechanism to propagate uncertainty in
  the belief network, and a formalism to combine
  evidence to determine the belief in a node.

28
Features of a FBN

• Single Combined Influences
• Bounded
• Two State variables their beliefs
• Modelling of extent of influence based on FSMF
• Fuzzy Propagation

29
Single Combined Influences

• Influences from
• Single observations

• Influences from
• Combined observations

AgtE
AgtE
BgtE
E
E
SINGLE UPDATE
SINGLE UPDATE
30
A FSMF Model

• Example
• A Tsukamoto FSMF
• ref An approach to fuzzy reasoning
  method in Advances in Fuzzy Set Theory And
  Applications, Y Tsukamoto
• Shows Fuzzy Propagation of a belief influence
  from a variable X to a Parent or Child

31
f0 fxy(x0) 1/1exp-a(x0-b)

• f0 The output fuzzy confidence level(belief)
• fXY The propagating belief function from X to Y
  given by the sigmoid
• with center b and rate of influence a.
• x0 The input, 0 x 1

32
Fuzzy Propagation Observation of one
node,D Update to node H given by fH
afDH(fD) (1- a ) fH where, fH The
new updated fuzzy belief at H fDH(fD) FSMF
for influences from D to H fD The
observed fuzzy confidence (belief)
of the state D fH The previous belief for
H
33
Observation of two nodes, D E (Simultaneous
Updation) Update to node H given by fH
afDH(fD) ßfEH(fE) ?fH , (aß?1) where, fH
The new updated fuzzy belief at H fDH(fD)
FSMF for influences from D to H fEH(fE) FSMF
for influences from E to H fD The observed fuzzy
confidence (belief) of the state D fE The
observed fuzzy confidence (belief) of the
state E fH The previous belief for H
34
The Fuzzy belief Inferencing System
A, A- probabilistic prior and conditional beliefs
?EC0.3 ?EC0.7 ?EC0.6 ?EC0.4
?B0.4 ?B0.6
?A0.7 ?A0.3
B
B
E
E
A
A
?DB0.7 ?DB0.3 ?DB0.2 ?DB0.8
D
D
C
C
?CA,B0.2 ?CA,B0.8 ?CA,B0.3 ?CA,B
0.7 ?CA,B0.8 ?CA,B0.2 ?CA,B0.6
?CA,B0.4
G
G
H
H
?H0.2 ?H0.8
?GD0.2 ?GD0.8 ?GD0.4 ?GD0.6
F
F
?FD0.2 ?FD0.8 ?FD0.7 ?FD0.3
35
FBN version of referencing
0.4
Observe 0.86
2
5
1
0.5
0.7
0.7
Observe 0.74
4
3
0.4
7
8
0.2
6
0.3
0.3
fD afGD(fD) (1- a )fD , (0lta lt1)
36
Computer Run

• Observation of nodes3,4
• The k-nbhds and fuzzy influence rules
• 4gt7,4gt83,4gt23gt13gt68gt5
• The observed nodes 3,4 are flagged
• Ist nbhd 7,8,1,6,2
• 2nd nbhd 5
• a3.0 and b0.5 in
• f0 fxy(x0) 1/1exp-a(x0-b)1/1exp-3.0(x0
  -0.5)
• f2 af32(f3) ßf42(f4) ?f2 , (a ß?1)

37

• Results of fuzzy inferencing from 3,4
• Node1 Start belief0.7 Final Belief0.686
• Node2 Start belief04 Final Belief0.511
• Node3 Start belief0.74 Final Belief0.74
• Node4 Start belief0.86 Final Belief0.86
• Node5 Start belief0.5 Final Belief0.426
• Node6 Start belief0.3 Final Belief0.419
• Node7 Start belief0.3 Final Belief0.468
• Node8 Start belief0.2 Final Belief0.402

38
Influences Forward and reverse

• Influence 17gt4 Reverse 1 4gt7
• Influence 28gt4 Reverse 1 4gt8
• Influence 32gt3,4 Reverse 1 3,4gt2
• Influence 41gt3 Reverse 1 3gt1
• Influence 56gt3 Reverse 1 3gt6
• Influence 65gt8 Reverse 1 8gt5

39
Initial and Oberserved Beliefs

• Node1 Start belief0.7 Flag0
• Node2 Start belief04 Flag0
• Node3 Start belief0.74 Flag1(observed)
• Node4 Start belief0.86 Flag1(observed)
• Node5 Start belief0.5 Flag0
• Node6 Start belief0.3 Flag0
• Node7 Start belief0.3 Flag0
• Node8 Start belief0.2 Flag0

40
Lets Run an Example

• http//ultima.cs.unr.edu/fzBN2/fbn.htm

41
Observations

• So after looking at the examples we can infer
  that fuzzy belief networks takes polynomial, that
  is if you have n nodes we can inference in O(n).
• The higher the value of the rate, a for the
  sigmoid, i.e., the steeper the sigmoid, higher is
  the extent of influence of a variable state over
  its k-nbhds and vice versa.

42
1
f0
f0
f0
Sigmoid s
Sigmoid s
0
Sigmoid s
X0
0
1
43
Scope for Future Improvement

• Use of Sigmoids with different rates of
  influence, a, for each influence direction will
  yield better results.

44
Applications of FBNs

• Real World problems
• Medical Diagnostic Systems
• Modeling of Brain
• Data Mining
• Techniques

• Economics
• AI
• Speech Recognition
• Genetics
• Image Modeling
• Space exploration

45
Advantages of Fuzzy Belief Networks Fuzzy
Influence Propagation

• Computationally Simple
• Cheap
• Intuitive
• Fine Tuning of FSMF from Real World Data possible

• Bidirectional conditional Propagation
• Few constraints
• Linear Time Complexity

46
Conclusions

• Thus it has been conclusively shown that FBNs
  with some further improvement, can easily shadow
  the contemporarily used Bayesian Networks and can
  help advance computational science in all fields
  of applications.

47
Some Software

• Netica-Bayesian network development software
• Microsoft Bayesian Network toolkit
• Fuzzy belief ultima.cs.unr.edu/fzBN2/fbn.htm
• A lot more resources on the web!!!

48
References

• Dr. Looneys class notes for Fuzzy
  logic-ultima.cs.unr.edu/cs773/u8/unit8773c.htm
• NEURAL FUZZY SYSTEMS-www.ece.purdue.edu/csglee/nf
  s.html
• A casual mapping approach to constructing
  bayesian networks-Shenoy and Nadkarni-ftp//ftp.bs
  chool.ukans.edu/home/pshenoy/wp289.pdf
• Belief networks-http//www.aiai.ed.ac.uk/links/bn.
  html
• Neural logic Belief NWs
• http//citeseer.nj.nec.com/345016.html
• Introduction to belief Networks
• httpanc.ed.ac.uk/amos/belief.htm
• Bayes Rule
• http//murrayc.com/learning/AI/bbn.shtml

49
Thank You!
?
Questions