HLTHINFO 730 Healthcare Decision Support Systems Lecture 5: Production Rule Systems

1
HLTHINFO 730Healthcare Decision Support
Systems Lecture 5 Production Rule Systems

• Lecturer Prof Jim Warren

2
Expert Systems

• Expert System a system that is (in some sense)
  a synthetic expert
• Generally for a very narrow domain of knowledge
• Reasons in a way that emulates a human expert
• Uses
• For safety and completeness as a check, even
  though the user is an expert too
• To extend the user because they have less than
  ideal expertise themselves
• To train or test the user possibly working on
  canned data and/or with the system providing
  explanation of its recommendations

3
Rules and heuristics

• Expert systems are most commonly built from rules
• E.g., IF light is red, THEN stop
• Rules may be more or less certain
• E.g., IF Temperaturegt37 THEN infection may be
  present
• Confidence in a rule may be quantified as a
  probability
• Heuristics are rules of thumb
• See http//www.phrases.org.uk/meanings/307000.html
  for origin of the phrase
• Adequate, but not logically fool-proof, ways of
  reasoning based on experience
• Heuristic knowledge may help the search for a
  solution
• E.g., allow extrapolation of therapeutic
  benefits from one drug to another if they are of
  the same physiologic class - http//www.annals.or
  g/cgi/content/full/124/1_Part_1/56

4
Knowledge Engineering (KE)

• KE developing (and also potentially
  maintaining) a knowledge-based system (KBS),
  especially an Expert System
• A key aspect is Knowledge Acquisition (KA)
• Getting the rules and heuristics (and their
  confidence levels)
• Best to have the participation of a live expert
  (or preferably a panel of them)
• Suboptimal to work from a textbook or printed
  guideline, esp. if the knowledge engineer has
  only a technical (not clinical) background
• This can lead to a bottleneck (if the expertise
  were easy to come by, we probably wouldnt build
  the system!)

5
KA techniques

• In addition to reviewing documents (e.g.,
  guidelines)
• Interviewing
• Just ask the expert how they solve problems in
  the domain may first develop a number of
  reference cases to guide the conversation
• Protocol analysis
• Have the expert talk aloud as they solve a
  problem (or have two experts talk to each other
  as they solve the problem)
• Make a video (be sure audio is good!)

6
KA synthesis and verification

• Once youve identified some initial concepts and
  rules, begin formalisation and review with
  experts
• Process diagrams make a flowchart of the
  process thats followed for key decisions
• Ontology formulation collect all the concepts
  in the domain into a database (Protégé is a tool
  to support this)
• Often a concept will have more than one term
  (word or phrase used to refer to the concept)
• Create influence diagrams (show the relationship
  of factors to one another, to actions, and to
  outcomes)

7
Influence diagrams

• Arrows indicate influence (generally positive
  correlation / increased probability)

from http//www.cs.ru.nl/peterl/aisb.pdf
8
Program flowcharts

• Simple notation to show order of steps in a
  process
• Use diamond to indicate a decision and include
  two outbound branches
• Use rectangle with double sidebars to indicate
  details are defined elsewhere
• E.g.,

9
Production rules

• A Production Rule System emulates human reasoning
  using a set of productions
• Productions have two parts
• Sensory precondition (IF part)
• Action (THEN part)
• When the state of the world matches the IF
  part, the production is fired, meaning the action
  is executed
• The world is the set of data values in the
  systems working memory
• For a clinical expert systems, this is usually
  data about a patient, which, ideally, has come
  from (and may go back to) an electronic medical
  record, or it may be entered interactively (or
  usually a little of each)
• So production rules link facts (IF parts, also
  called antecedents) to conclusions (THEN parts,
  also called consequents)

10
MYCIN innovations

• MYCIN
• Developed at Stanford from 1972 to 1980
• Helped physicians diagnose a range of infectious
  blood diseases
• Separated the methods of reasoning on productions
  (inference engine) from the rules themselves
  (the knowledge base)
• Became the first expert systems shell when
  distributed as empty MYCIN (EMYCIN)
• Also provided explicit certainty factors (crude
  analog to probability), used backward chaining
  and provided an explanation facility

11
Example MYCIN rule
IF the stain of the organism is gram negative
    AND the morphology of the organism is rod
        AND the aerobicity of the organism is
anaerobic             THEN there is strongly
suggestive evidence (0.8) that the class of the
organism is Enterobacter iaceae.

• This rule has three predicates (yes/no, or
  Boolean, values that determine if it should fire)
• In this case each predicate involves the equality
  of a data field about a patient to a specific
  qualitative value(e.g., stain of the organism
  gram negative)
• Note that human expertise is still needed e.g.,
  to decide that the morphology of the organism is
  rod (nonetheless to understand its vocabulary!)
• Notice it produces a new fact (regarding class
  of the organism)
• Note this is symbolic reasoning working with
  concepts as compared to numbers (its not like y
  x1 4.6 x2)

12
Forward v. Backward chaining

• Forward chaining is reasoning from facts to the
  conclusions resulting from those facts
• E.g., if you see that it is raining before
  leaving from home (the fact) then you should
  decide to take an umbrella (the conclusion)
• Backward chaining involves reasoning in reverse
  from a hypothesis - from a potential conclusion
  to be proved, to the facts that support the
  hypothesis
• E.g., if you have not looked outside and someone
  enters with wet shoes and an umbrella, your
  hypothesis is that it is raining
• In order to support this hypothesis you could ask
  the person if it was, in fact, raining
• If the response is yes, then the hypothesis is
  proved true and becomes a fact

13
Backward chaining example

• Suppose that the goal is to conclude the colour
  of my pet Fritz, given that he croaks and eats
  flies, and that the rule base contains the
  following four rules
• If X croaks and eats flies - Then X is a frog
• If X chirps and sings - Then X is a canary
• If X is a frog - Then X is green
• If X is a canary - Then X is yellow
• This rule base would be searched and the third
  and fourth rules would be selected, because their
  consequents (Then Fritz is green, Then Fritz is
  yellow) match the goal (to determine Fritz's
  colour). It is not yet known that Fritz is a
  frog, so both the antecedents (If Fritz is a
  frog, If Fritz is a canary) are added to the goal
  list. The rule base is again searched and this
  time the first two rules are selected, because
  their consequents (Then X is a frog or canary)
  match the new goals that were just added to the
  list. The antecedent (If Fritz croaks and eats
  flies) is known to be true and therefore it can
  be concluded that Fritz is a frog, and not a
  canary. The goal of determining Fritz's colour is
  now achieved (Fritz is green if he is a frog, and
  yellow if he is a canary, but since he croaks and
  eats flies, he is a frog, and, therefore, he is
  green).

14
Backward chaining in practice

• Unlike the previous example, often we dont have
  all the information at first to reach the goal
• The system may ask questions interactively and
  then make more inferences
• A long chain of reasoning may be abandoned if its
  certainty becomes too low
• A practical system may ask a specific battery of
  questions up front just to get things started
• Some rules, or parts of rules, may be there for
  heuristic purposes to smooth the interaction
  (e.g., to not look for smoking as a factor in
  patients under 10 years of age not 100 valid,
  but possibly useful)
• As production rule systems get more complex,
  their behavior can become very hard to predict
  and they become impractical to verify

15
Shells

• There are some free expert systems shells
  nowadays (these things were hot in the 1980s, but
  are giveaways now!)
• CLIPS (C Language Integrated Production System)
  developed by NASA
• http//www.ghg.net/clips/CLIPS.html
• Commonly used in university AI courses
• Runs under Windows XP or MacOS X
• e2gLite an expert system as a Java applet (runs
  in your browser!) by expertise2go
• http//www.expertise2go.com/webesie/e2gdoc/
• Just put e2glite.jar and a .kb file in a
  directory together, along with a .html file that
  invokes the applet, and you can run it locally
  via your browser

16
e2gLite example

• A small .kb file (i.e., knowledge base)
• Well do a hands-on tutorial with this system

REM Test Knowledge Base (weather.kb) RULE Is it
going to rain? If precipitation
"expected Then the recommendation "wear a
raincoat PROMPT precipitation MultChoice
CF "According to the weather forecast,
precipitation is "expected "not
expected GOAL the recommendation
17
Explanation Facility

• Ability to explain its recommendations has always
  been considered part of an expert system
• Because its something a human expert is also
  expected to do
• Useful for
• Tutoring the student can learn what rules
  applied to the case at hand
• Use in practice the doctor (or other expert
  user) can confirm that the concur with the
  reasoning (and possibly then translate it for the
  patient as appropriate)
• Accuracy/safety/debugging the user might see an
  error in the explanation
• Readability
• Unfortunately, an automatically generated
  explanation can be rather cryptic
• Aided by use of good predicate names in the
  production rules
• Provenance
• Ideally, the system should allow links to
  supporting literature to support the rules
  themselves
• And its important that we know who was involved
  in the knowledge engineering, and how the system
  has been tested

18
Testing

• White box testing
• Testing with an understanding of the way the
  system is put together
• Design test cases that try out all the rules
• Black box testing
• Test without knowledge of (or sympathy for!) the
  structure of the system
• Ideally done by people removed from the main
  KA/KE process
• Obviously, for clinical expert systems, clinical
  people are the appropriate black box testers
  (this can be expensive)

19
Maintenance

• An expert system can be dead, but it can never be
  complete
• A non-trivial rule base cannot be proved correct
• Must always have a feedback process for end-users
  readily to report issues for investigation
• And medical knowledge doesnt sit still
• MYCIN is now an expert thats 28 years out of
  date!
• So production expert systems need a routine
  process of review, revision and re-distribution
• Last step easier if the systems web based, but
  at least users need a way to find out about the
  latest changes
• And when you change something, you have to
  re-test everything

20
Some Medical Expert Systems
21
Some current clinical expert systems

• DxPlain
• Provides justification of why to consider any of
  various diseases for a set of findings
• Knowledge base includes over 2200 diseases and
  4900 findings (http//www.lcs.mgh.harvard.edu/proj
  ects/dxplain.html)
• Slightly clunky slideshow available from
  http//dxplain.org/dxp/dxp.sdemo.pl?logindemslcsh
  ome
• Clinical Knowledge Summaries
• From the UK NHS - http//cks.library.nhs.uk/
• Organises advice on scenarios (e.g.,
  http//cks.library.nhs.uk/hypertension/view_whole_
  guidanceNodeIdn180741n180870n180871)
• A compromise between an online guideline and a
  running expert system
• PREDICT CVD/Diabetes
• Developed by School of Population Health and
  Engima Publishing
• Advises on management of CVD event risk

22
Further reading

• History and applications of AI by George Luger -
  http//www.cs.unm.edu/luger/ai-final/chapter1.htm
  l
• Introduction to expert systems by expertise2go -
  http//www.expertise2go.com/webesie/tutorials/ESIn
  tro/
• Clem McDonald, Medical Heuristics The Silent
  Adjudicators of Clinical Practice, Annals of
  Internal Medicine 124(1) 56-62, 1996 (from
  http//www.annals.org/cgi/content/full/124/1_Part_
  1/56)