ICT619 Intelligent Systems Topic 2: Expert Systems

1
ICT619 Intelligent SystemsTopic 2 Expert
Systems
2
Expert Systems

• PART A
• IntroductionApplications of expert systems
• Structure of an expert system
• An example rule base
• Reasoning in a rule-based expert system
• Reasoning using forward and backward chaining
• Dealing with uncertainty
• PART B (next week)
• Developing expert systems
• Frame-based expert systems
• Advantages and disadvantages of expert systems
• Case studies

3
Expert Systems (ES)- what they are

• Intelligent systems for emulating human experts
• Used as decision support tools, sometimes control
  systems
• Can be
• consistent, unbiased substitutes for human
  experts
• repository for domain-specific knowledge
• Work by
• capturing human expertise about a specific area,
  or domain
• applying deductive reasoning to infer conclusions
• Not self-adaptive
• can not learn by themselves, all knowledge
  encoded and maintained by humans

4
Why expert systems?

• Problem solving knowledge is often expressed as
  heuristics or "rules of thumb"
• Easier to represent heuristics using rules than
  using algorithms
• A knowledge engineer extracts knowledge and
  encodes them into rules
• ES store knowledge as discrete rules in a rule
  base
• Distinction from conventional programs is that
• Knowledge separated from its processing.
• Easier to build and maintain as change in
  processing code does not affect knowledge and
  vice versa

5
Some advantages of ES

• Developed in the 1970s, currently well
  established as intelligent systems
• Expertise is available 24 hours a day
• Unlike human experts, they do not retire, die or
  resign
• They may provide consistent, unbiased
  recommendations
• Multiple copies can be produced and distributed
• Can justify conclusions by detailing the chain of
  reasoning followed

6
Applications of expert systems

• Early successful applications in medicine
• MYCIN (mid 70s) for diagnosing and recommending
  treatment for blood disease
• In science and engineering
• DENDRAL (chemistry) and PROSPECTOR (geology)
• First major commercial application
• XCON (Digital Equipment Corporation 1979)
• Other commercial applications in
• banking and finance, eg TARA (foreign currency
  trading)
• manufacturing
• personnel management

7
Applications of expert systems (contd)

• US public sector agencies making use of expert
  systems
• Environmental Protection Agency
• Immigration and Naturalisation Service
• Postal Service
• Internal Revenue Service
• Department of Energy
• British National Health Service ES with 11,200
  rules used to evaluate the performance of medical
  care providers
• American Expresss Authorizers Assistant helps
  decide approval of credit card charge

8
Structure of an expert system
User Interface - Menu-driven - GUI - Natural
language
Knowledge-base editor
Knowledge-base
User
Inference engine
Working memory
Explanation sub-system

• Three main components
• The rule-base,
• The working memory, and
• The inference engine

9
Structure of an expert system (contd)

• Rule base stores knowledge encoded as rules
• Working memory stores initial facts specific to
  the problem at hand, intermediate conclusions and
  hypotheses for this run
• Inference engine uses rules in knowledge base to
  arrive at final conclusion
• User interface
• Allows user to enter relevant facts by answering
  questions asked by the system
• Enables use of the explanation sub-system by
  asking why and how questions
• Knowledge-base editor used to create, debug and
  maintain rules
• Explanation system keeps track of reasoning
  process so that the user can verfiy conclusions

10
Knowledge representation using rules

• Knowledge represented in rules having the form
• IF ltconditiongt THEN lt conclusiongt
• Left hand side (LHS) is called the antecedent
• Right hand side (RHS) is called the consequent
• Propositional logic - basis of reasoning used in
  rule-based expert systems
• Antecedents and consequents are examples of
  propositions or statements in propositional logic

11
Knowledge representation using rules (contd)

• A rule can have more than one proposition in its
  antecedent or consequent
• For example, in the rule
• IF rain is forecast AND outdoor
  activity is anticipated
• THEN advice is take rain coat
• the antecedent consists of two propositions
  combined using the logical AND connective

12
An example rule-base the mortgage loan case
(Zahedi '93)

• The domain expertise needed for approving a
  mortgage loan contains the following knowledge
  base
• To get a mortgage loan,
• the applicant must have a steady job,
• acceptable income,
• good credit ratings and
• the property should be acceptable.
• If applicant does not have a steady job, then
  they must have adequate assets.
• The amount of loan cannot be more than 80 percent
  of the property value, and the applicant must
  have 20 percent of the property value in cash.

13
An example rule-base the mortgage loan case
(contd)

• The definition of a steady job
• Applicant should have been at the present job for
  more than two years.
• The definition of adequate assets
• Applicants properties must be valued at ten
  times the amount of the loan, or the applicant
  must have liquid assets valued at five times the
  amount of the loan.
• An acceptable property
• Either located in the banks lending zone with no
  legal constraints, or is on the banks exception
  list.
• The definition of adequate income
• If applicant is single, then mortgage payment
  must be less than 70 percent of their net income.
  
• If applicant is married, then mortgage payment
  must be less than 60 percent of the family net
  income.

14
The mortgage loan case rule-base

• 1. IF the applicant has a steady job AND the
  applicant has adequate income AND the property
  is acceptable AND the applicant has good credit
  ratings AND the amount of loan is less than 80
  of the property value AND the applicant has 20
  of the property value in cash THEN approve
  the loan
• 2. IF the applicant has adequate assets AND the
  applicant has adequate income AND the property
  is acceptable AND the applicant has good credit
  ratings AND the amount of loan is less than 80
  of the property value AND the applicant has 20
  of the property value in cash THEN approve
  the loan

15
The mortgage loan case rule-base

• 3. IF the applicant has a job AND the applicant
  has been more than two years at the present job
  THEN the applicant has a steady job
• 4. IF the property is in the banks lending
  zone OR the property is on exception list
  THEN the property is acceptable
• 5. IF the family income is adequate OR the
  single income is adequate THEN the applicant
  has adequate income

16
The mortgage loan case rule-base

• 6. IF the applicant is married AND mortgage
  payment is less than 60 of the family net
  income THEN the family income is adequate
• 7. IF NOT the applicant is married AND mortgage
  payment is less than 70 of applicants net
  income
• THEN the single income is adequate
• 8. IF applicant has properties with a value
  greater than 10 times the loan OR the
  applicant has liquid assets greater than 5 times
  the loan
• THEN the applicant has adequate assets.

17
Reasoning in a rule-based expert system

• Inference is performed through deductive
  reasoning
• Deductive reasoning
• reasoning process starts with a set of premises
  already proven or accepted
• new facts or conclusions are derived based on the
  premises using rules of inference
• ES combines facts and units of knowledge (rules)
  to deductively infer new knowledge as conclusions
  and recommendations

18
Rules of inference used in expert system reasoning

• Reasoning based on the following rules of
  inference borrowed from propositional logic
• Modus ponens
• Hypothetical syllogism
• Modus tollens
• and Boolean logic
• True AND True True, True AND False False,
• False AND False False, True OR False
  True,
• True OR True True, False OR False False
  etc.
• Modus ponens Given a rule, if the antecedent is
  true, conclude that the consequent is also true.
• Given IF X THEN Y
• then if X is true
• conclude Y is true

19
Rules of inference used in expert system
reasoning (contd)

• Hypothetical syllogism
• When the consequent of one rule is the
  antecedent of a second rule, then we can
  establish a third rule whose antecedent is that
  of the first rule and whose consequent is that
  of the second.
• IF X THEN Y
• IF Y THEN Z
• conclude IF X THEN Z
• Modus tollens (indirect proof)
• When the negation of a fact is established,
  given the consequent of a rule is not true,
  conclude that the antecedent is not true.
• Given IF X THEN Y
• then if Y is false
• conclude X is false

20
ES Reasoning process using multiple inferencing

• Multiple inferencing involves use of more than
  one rule for drawing a conclusion
• The inference engine matches facts in the working
  memory with rules in the rule-base to determine
  which rules apply
• More than one rule may match a fact. So all
  matching rules are put in a conflict set by the
  inference engine
• The inference engine selects one rule from the
  conflict set and fires (applies) it
• As a result of applying a rule, a new fact may be
  inferred, which is added to the working memory

21
ES Reasoning conflict resolution

• The expert system may come to a halt (no more
  changes to working memory) or repeat the
  match-and-fire cycle depending on the latest
  inference
• Different order of selecting rules from the
  conflict set may result in different outcomes
• Order of rule firing is determined by meta rules
  . The order can be made to be
• independent of problem
• specific to a problem

22
ES Reasoning conflict resolution (contd)

• Resolution independent of problem
• Fire rules in the order they appear in the rule
  base
• Fire rule matching most recently added fact
  (recency)Eg, (Negnevitsky 2005)
• Rule 1
• IF forecast is rain 0816 PM 11/25/96
• THEN advice is take an umbrella
• Rule 2
• IF weather is wet 1018 AM 11/26/96
• THEN advice is stay home

23
ES Reasoning conflict resolution (contd)

• Fire rules with a large number of conditions on
  the LHS first (specificity)
• Eg. (Negnevitsky 2005),
• Rule 1
• IF the season is autumn
• AND the sky is cloudy
• AND the forecast is rain
• THEN advice is stay home
• Rule 2
• IF the season is autumn
• THEN advice is take an umbrella
• Choice of rule may be specific to a problemEg,
  Favour rules dealing with high credit risk

24
Reasoning using forward and backward chaining

• There are two well-known methods of multiple
  inferencing.
• In backward chaining
• multiple inference starts with a goal
• It finds the rule whose consequent matches the
  goal
• and goes backward to the antecedent part of
  the rule
• It then tries to establish the truth value of the
  antecedent part of the rule
• It does this by establishing the truth values of
  the propositions in the antecedent
• This is done by finding rules having consequents
  matching the propositions
• If no such rule is found in the rule base, the
  user is asked to provide information to establish
  the truth of the propositions

25
Backward chaining in the mortgage loan example

• Reasoning process starts with the goal Approve
  loan for an applicant
• This goal is the consequent of rules (1) and (2).
• Assume rules are tested sequentially from the
  beginning of the rule base
• Rule (1) is fired, and its propositions become
  the current goals.
• The first proposition test is whether the
  applicant has a steady job.
• This is in the consequent of rule (3). It asks
  the user if the applicant has a job. If the
  answer is no, the inference engine does not go
  any further in (3). Since applicant has a
  steady job is not true, it does not go any
  further with rule (1) either
• As the goal could not be reached from rule (1),
  the inference engine tries rule (2) next to
  establish the goal.

26
The mortgage loan case rule-base

• 1. IF the applicant has a steady job AND the
  applicant has adequate income AND the property
  is acceptable AND the applicant has good credit
  ratings AND the amount of loan is less than 80
  of the property value AND the applicant has 20
  of the property value in cash THEN approve
  the loan
• 2. IF the applicant has adequate assets AND the
  applicant has adequate income AND the property
  is acceptable AND the applicant has good credit
  ratings AND the amount of loan is less than 80
  of the property value AND the applicant has 20
  of the property value in cash THEN approve
  the loan

27
The mortgage loan case rule-base

• 3. IF the applicant has a job AND the applicant
  has been more than two years at the present job
  THEN the applicant has a steady job
• 4. IF the property is in the banks lending
  zone OR the property is on exception list
  THEN the property is acceptable
• 5. IF the family income is adequate OR the
  single income is adequate THEN the applicant
  has adequate income

28
The mortgage loan case rule-base

• 6. IF the applicant is married AND mortgage
  payment is less than 60 of the family net
  income THEN the family income is adequate
• 7. IF NOT the applicant is married AND mortgage
  payment is less than 70 of applicants net
  income
• THEN the single income is adequate
• 8. IF applicant has properties with a value
  greater than 10 times the loan OR the
  applicant has liquid assets greater than 5 times
  the loan
• THEN the applicant has adequate assets.

29
Backward chaining in the mortgage loan example
(contd)

• The first proposition in rule (2) is if the
  applicant has adequate assets. This becomes the
  current goal.
• Inference engine searches the rule base from the
  beginning to see which rule has this proposition
  as its consequent. It finds rule (8).
• First proposition of rule (8) becomes the current
  goal.
• Inference engine searches entire rule base to see
  if any rule has the applicants properties as the
  consequent. It does not find any.
• So it asks the user if the applicant has
  properties with a value greater than ten times
  the loan.

30
Backward chaining in the mortgage loan example
(contd)

• If the answer is yes, the inference engine
  concludes that the applicant has adequate assets
  and attempts to check the truth value of other
  conditions in rule (2).
• If the answer is no, then the system asks whether
  the applicant has liquid assets greater than five
  times the loan.
• If the user says no, the inference engine does
  not go any further because it has failed to
  establish the truth of the goal. This means that
  the goal of approving the loan cannot be
  supported.
• If the users answer is positive, then the
  inference engine attempts to check the truth
  value of the second proposition in rule (2), and
  so on (the rest of this multiple inference is
  left as an exercise).

31
Forward Chaining

• The second method of multiple inferencing is
• forward chaining
• The system requires the user to provide facts
  pertaining to the problem
• The inference engine tries to match each fact
  with the antecedent of a rule
• If the match succeeds, the rule fires and the
  truth of the consequent of that rule is
  established, and is added to known facts of the
  case currently in working memory
• This process continues until the inference engine
  has drawn all possible conclusions by matching
  facts to antecedents of rules in the knowledge
  base

32
Forward Chaining example

• Assume the knowledge base consists of the
  following rules
• IF A THEN C
• IF D THEN E
• IF B AND C THEN F
• IF E OR F THEN G
• If we start with known facts that A and B are
  true, then the inference engine uses A and rule
  (1) to conclude that C is true.
• C is added to working memory as a known fact for
  the case
• Then it uses B and C and rule (3) to conclude
  that F is true.
• F added to working memory as a known fact for the
  case
• It then uses the truth of F and the last rule to
  establish that G is true.

33
Forward vs. Backward Chaining

• Forward chaining is data driven because it starts
  with the data about the case and moves forward
  from the antecedents of rules to conclude their
  consequents.
• Backward chaining is goal driven since it starts
  with objective of satisfying a goal.
• Backward chaining is useful when the number of
  goals is small
• Forward chaining performs well when
• the number of goals is large
• the user has a given set of facts at the start of
  the inquiry, and wants to find the implications
  of these facts
• Some expert system products allow for combining
  the two methods of multiple inference.

34
Dealing with uncertainty

• Facts and inferences in logic are categorical -
  either true or false
• But in expert systems applied to real life
  problems uncertainty may arise
• within the knowledge domain
• due to expert and knowledge engineer
• due to the user
• Uncertainty in knowledge domain
• Knowledge may be incomplete and imperfect
• Knowledge may be vague
• Knowledge may become uncertain due to measurement
  error
• Uncertainty may be introduced by conflicting
  expert opinions.

35
Dealing with uncertainty (contd)

• Uncertainty related to the expert and knowledge
  engineer
• Expert may not be 100 certain of a rule
• Knowledge engineer may not have 100 confidence
  in a rule expressed by expert.
• Uncertainty related to the data input by user
• User may be unsure about accuracy of data to be
  input

36
Dealing with uncertainty (contd)

• Use of certainty parameters in inference
• Uncertainty propagates between rules as the
  conclusion reached in an uncertain rule gets used
  in another rule
• Degree of uncertainty in a rule or fact may be
  expressed numerically using the certainty or
  confidence factor cf, in the range 0, 1
  or -1, 1
• It is difficult to implement probability-based
  uncertainty handling schemes
• Ad hoc schemes, although mathematically unsound,
  seem to work

37
Calculation of rule confidence factor (cf) for
uncertain facts

• A scheme for dealing with uncertainty
• Let P1 and P2 be two propositions and cf(P1) and
  cf(P2), their certainty parameters
• Then
• cf(P1 AND P2) min (cf(P1), cf(P2))
• cf(P1 OR P2) max (cf(P1), cf(P2))
• Given the rule
• IF P1 THEN P2 (Rule cf C)
• Then certainty of the consequent P2 is given by
• cf(P2) cf(P1) C

38
Calculation of rule cf for uncertain facts
(contd)

• Example
• IF interest rates fall (cf0.6)
• AND taxes are reduced (cf0.8)
• THEN stock market rises (Rule cf0.9)
• The cf of the conclusion that the stock market is
  rising can be calculated to be
• (min(0.6,0.8) 0.9 0.6 0.9 0.54
• If more than one rules lead to the same
  conclusion, the final conclusion is given maximum
  cf value of all these rules
• CF system works, but only under fairly
  restrictive conditions (eg single connections
  between rules)

39
Uncertainty handling using probability theory

• There are schemes for handling uncertainty based
  on probability theory, but they suffer from
  practical limitations
• Difficulty with using probability theory
• It is difficult for human experts to express
  likelihood estimates in terms of probabilities
• Not all information will be available for correct
  probabilistic treatment of uncertainties, eg, to
  evaluate certainty of rule
• IF A OR B THEN C
• needs probabilities of both A and B, as well
  as correlation between occurrences of A and B
• In general, probabilistic reasoning is very
  different from the logical reasoning used by
  expert systems and combining the two properly is
  hard
• Uncertainty is handled much more effectively
  using fuzzy rather than traditional logic (Topic
  3)

40
REFERENCES

• AI Expert, October 1991 presents applications
  of expert systems
• Dhar, V., Stein, R., Seven Methods for
  Transforming Corporate Data into Business
  Intelligence., Prentice Hall 1997, Ch 7
• Giarratano, J., Riley, G. Expert Systems
  Principles and Programming, Thomson Course
  Technology, 2005.
• Negnevitsky, M. Artificial Intelligence A Guide
  to Intelligent Systems, Addison-Wesley 2005.
• Zahedi, F., Intelligent systems for Business,
  Wadsworth Publishing, Belmont, California, 1993.