Dealing with Uncertainty

1
Dealing with Uncertainty

• (Reference - Course readings p. 111 "The
  Inference Engine (Uncertainty)"
• and p. 76 "Rule-based Expert Systems")
• Decision making in a climate of uncertainty
• Sources of uncertainty
• ambiguity (conflicting or ambiguous
  information)
• incomplete (insufficient information)
• naturally occurring imprecision
• for information arising from measurement, we may
  be uncertain because of
• measurement errors (can be quantified, a
  property of the measuring device)
• systematic errors (an error that occurs
  consistently)
• random errors (an error that occurs randomly
  or by chance)

2
Dealing with Uncertainty

• In terms of a knowledge based system
• uncertainty can exist in the knowledge base
  itself
• conflicting or overlapping rules
• differing rule priorities (implicit/explicit)
• you can set the priority of one rule over
  another (explicit) or the inference engine can
  determine the priority during inferencing
  (implicit)
• within the rules
• how certain are we of the truth of a rule?
• how certain are we of the facts used during
  inferencing?
• how confident is the rule base of the
  conclusion(s)?

3
Dealing with Uncertainty

• Techniques
• how do we estimate uncertainty?
• how do deal with it during reasoning?
• There are several possibilities
• Conditional Probability
• the uncertainty of a fact, hypothesis, rule etc
  is encapsulated into a single value --- the
  probability (or chance) that the fact holds.
• probability is a bounded concept 0 --- no
  chance that the fact holds, 1 --- we are totally
  certain that the fact holds
• probability that the fact ltetcgt does not hold
  1 - Pr(fact)
• the probability that a hypothesis holds, given
  some supporting evidence can be determined using
  conditional probabilities and Bayes' law

4
Dealing with Uncertainty

• For example, from a medical diagnosis expert
  system
• Pr(D given S) (Pr(S given D) Pr(D))/Pr(S)
• where
• D is the disease
• S is a set of symptoms exhibited by a patient.
• Pr(D given S) is the probability that the
  patient has the disease D given that they exhibit
  the set of symptoms, S.
• Pr(S given D) is the probability that the
  patient will exhibit the set of symptons S when
  the disease D is present.
• Pr(D) is the prior probability that the patient
  has the disease before any of the symptoms are
  known.
• Pr(S) is the prior probability that the patient
  has the symptoms S before the disease D is known.
• However these sorts of approaches have
  difficulty with the qualitative presentation of
  data and with unknown facts.
• The MYCIN experiments demonstrated that the
  estimation of probabilities by experts can be
  difficult. It was also found that in problem
  domains of this type experts were unwilling to
  assign a single value to indicate the probability
  of a patient having a disease and at the same
  time implying a probability that they didn't.
• In other words they wished to express some
  uncertainty about the chance of a diagnosis
• _

5
Certainty factors

• The use of certainty factors to quantify and
  deal with uncertainty arose from the MYCIN
  project.
• They are built around the belief (disbelief)
  that a hypothesis (fact) may be true given some
  supporting evidence.
• original definition ('certainty' of a
  hypothesis H given evidence E)
• CF(H,E) mB(H,E) - mD(H,E)
• CFs range between -1 and 1 It is
  essentially a technique to establish a CF in a
  rule
• consequent.
• problems with the original MYCIN definition,
  particularly when CFs were being used as
  threshold for firing rules.
• updated definition
• CF(H,E) (mB(H,E) - mD(H,E))/ (1 -
  min(mB(H,E),mD(H,E)))
• so IF E THEN H then the CF of H is that
  of the rule holding

6
Certainty factors

• However if we are uncertain of the
  antecedent, what effect does that have on the
  uncertainty of the rule itself?
• The uncertainty calculus
• IF E THEN H (CF 0.9)
• What if our certainty of the fact E is only
  0.9 ?
• The CF for the rule is modified to 0.9 0.9
  0.81
• final rule CF rule CF antecedent CF
• For a compound antecedent
• IF A and B and C THEN H (CF 0.8)
• two steps
• determine CF of antecedent
• adjust CF of the rule rule CF
  antecedent CF
• assume CF(A) 0.9, CF(B) 1, CF(C) 0.6
• For the AND connective CF(antecedent)
  min(CF(clauses))
• i.e. CF(A and B and C) 0.6 and rule CF
  0.6 0.8 0.48

7
Certainty factors

• another example.
• IF A or B or C THEN H (CF 0.8)
• CF(antecedent) max(CF(clauses))
• so using previous values
• CF(antecedent) 1 and CF(rule) is still 0.8
• For the not clause
• IF not A THEN H (CF 0.8), using the previous
  example where CF(A) 0.9
• Then CF (antecedent) -0.9 and CF for the rule
  0.8 -0.9 -0.72
• Summary
• Using certainty factors from the MYCIN
  experiments
• range -1 to 1
• AND min(CFs)
• OR max(CFs)
• NOT -CF

8
Certainty factors

• Combining two or more rules
• R1 IF A and B THEN H (CF 0.7)
• R2 IF C and D THEN H (CF 0.6)
• CF(R1 followed by R2) CF(R1) CF(R2) -
  CF(R1)CF(R2)
• e.g. (0.7 0.6) - 0.70.6 1.3 - 0.42 0.88
• This technique can be extended to the
  combination of three or more rules leading to the
  hypothesis i.e.
• R3 IF E and F THEN H (CF 0.9)
• Then CF(R1,R2,R3) CF(R1,R2) CF(R3) -
  CF(R1,R2) CF(R3)
• i.e. 0.88 0.9 - 0.880.90 1.78 - 0.792 0.99
  
• In general the CF for a rule may be modified by
  the CFs for the antecedent
• Then the CF for a final hypothesis may be
  modified by successive firing of rules
• i.e. an accumulation of evidence

9
Summary

• Rule-based Systems
• Knowledge base --- a set of production rules
  (perhaps with some support from a database).
• A separate reasoning' part which processes' the
  rules to reach some sort of conclusion.
• A user interface to facilitate use.
• A method to deal with uncertainty'.
• Uses boolean logic/set theory as a basis for
  reasoning.

10
Next time

• Fuzzy rule based systems
• A method to deal with uncertainty, particularly
  'naturally occurring imprecision'.

11
Expert System Shells

• Software tools that provide an inference
  engine, explanation capability, a knowledge base
  editor and a user interface.
• Development platform for a KBS.
• The KBS developer provides the domain specific
  knowledge for a particular application area.
• The available knowledge representation schemes
  and reasoning capabilities will depend on the
  tool.

12
Expert System Shells

• Exsys Software (educational version of EXSYS
  Professional)
• PC-BASED software.
• Allows you to build a rule-based' systems.
• i.e the only knowledge representation scheme
  used is the production rule.
• There are restrictions on the number of rules in
  a given knowledge base (50 rules).
• It is very simple to use.

13
Expert System Shells

• Exsys Software (educational version of EXSYS
  Professional)
• Software contained in ESsoftware.zip and
  ESTutorial.zip, accessed through subject homepage
• Zip files contain
• EDITDEMO.EXE is the rule editor
• EXDESIGN.EXE is the custom design program
• Plus 5 tutorials
• lesson 1 pro_tut1.pdf Getting started
• lesson 2 pro_tut2.pdf Adding rules and
  backward chaining
• lesson 3 pro_tut3.pdf Validation
• lesson 4 pro_tut4.pdf Variables and Choices
  in the IF part
• lesson 5 pro_tut5.pdf Custom Screens
• Plus assorted other files

14
Expert System Shells

• Exsys Software (educational version of EXSYS
  Professional)
• A note on terminology
• choices The possible outcomes of the decision
• qualifiers The factors which influence the
  outcomes. They can have values they are normally
  framed as a question
• For example
• What is the author's reputation? (where the
  qualifiers name is reputation and the values
  might be good, fair)
• Rules can be set up using the rule editor with a
  qualifier on the LHS and a choice on the RHS