What is being taught Modelling the Domain

1
What is being taught? Modelling the Domain
2
Contents

• 1. Black Box models
• 2. Glass Box models
• 3. Cognitive Models

3
Encoding Domain Knowledge

• Black Box Models
• Reason about domain without actually coding
  knowledge e.g. mathematical models
• Generate the correct input-output behaviour over
  range of tasks can use as a judge of
  correctness.
• BUT means by which provides behaviour not
  available for inspection or use.
• 2. Glass Box Expert Systems
• Develop an expert system
• Knowledge Acquisition from human, code and apply
• May not correspond to way human applies it
• 3. Cognitive Models
• Make a simulation (at some level of abstraction)
  of how humans uses knowledge

4
1. Black Box Models
5
Black Box Models Sophie (Brown et al., 1982)

• Students troubleshooting electronic circuits
• try out ideas, have them critiqued, receive
  advice
• Fault inserted for student to debug.
• The system can
• answer hypothetical questions (what if?)
• evaluate hypotheses (consistent?)
• list all possible hypotheses (gen and test)
• evaluate a new measurement (redundant?)
• Representations
• simulation based mathematical model
• procedural specialists that use model
• semantic net of declarative facts

6
SOPHIE I

• Performs correct behaviour, but.
• Mathematical simulation model - so means by which
  behaviour is produced is not available for
  inspection
• Does not have student model, nor explicit
  knowledge of troubleshooting
• Makes no use of kind of causal reasoning
  performed by expert troubleshooters

7
Illustration of Sophie (Brown et al., 75)

• Student What could be wrong?
• SOPHIEHere are some faults which explain the
  measurements you observed
• the beta of Q3 is low
• the beta of Q4 is low
• or R22 has the wrong resistance
• Student What would the output voltage be if the
  beta of Q3 were 8?
• SOPHIE If the beta of Q3 were 8 under the
  present setting, the output voltage would be 10.5
  volts. In a working circuit it is 19.9 volts.
• Student Is the emitter current of Q4 correct?
• SOPHIEThe emitter current of Q4 is -61
  milliamps. In a working circuit the emitter
  current of Q4 is -49 milliamps. That value seems
  to be incorrect.
• Student Could R22 be low?
• SOPHIE Is it OK if I use 25 ohms for the
  resistance of R22?
• Student Yes
• SOPHIEThat fault agrees with all the
  measurements you have made.

8
SOPHIE II - an articulate expert - 1976

• Not qualitative simulation, but improvement
• Troubleshooting expert added to lab.
• Able to demonstrate strategies, though cannot
  reason about student strategies
• Troubleshooting expert reasons qualitatively
• makes causally meaningful measurements
• explains strategic decisions as proceeds
• uses decision tree, pre-stored explanations
  attached to nodes
• So we can start with a simulation, and then add
  further knowledge and reasoning on top of it

9
SOPHIE II excerpt (Brown et al, 1976)

• SOPHIE-II How does the OUTPUT CURRENT of the
  CONSTANT CURRENT SOURCE compare with a working
  circuit?
• Student OK
• SOPHIE-II The current source is working then.
  The modules now left are CL,VL,OPF and DARL. I
  think I will make my next measurement on the
  Darlington amplifier. I want to know whether the
  BASE CURRENT of Q4 is OK.
• Student HELP
• Referee Lets use SOPHIE to find the answer.
• Is the base current of Q4 OK?
• SOPHIE-I The base current of Q4 is 0.0 amps,
  which seems to be incorrect because in a
  working circuit it is .031 milliamps.
• Referee So, you see, the BASE CURRENT of Q4 is
  LOW in the faulted circuit. Lets go on.
• SOPHIE-II Low input current to the Darlington
  amplifier would certainly result in lower than
  normal voltage at its output under moderate
  current loads. I think then, the Darlington
  amplifier is OK, since it is following its input.
  Now, since we have already seen that the CSS
  output current is OK, perhaps either the current
  limiter or the voltage limiter must be drawing
  excessive current from the CSS.

10
2. Glass Box Models
11
Glass Box Models

• General Method
• knowledge engineer domain expert
• identify problem area and scope
• enumerate and formalise key concepts
• formulate system to implement K
• test and refine iteratively
• Knowledge Acquisition
• time-consuming
• provides articulate, human-like representation of
  knowledge underlying expertise in domain.
• may not match novice learners' domain
  knowledge....
• There are a number of examples of this, classic
  one being Guidon (Clancey, 1983) in the medical
  domain

12
GUIDON (Clancey, 1983)

• Built on top of MYCIN (Shortliffe 1976)
• expert system diagnosing bacterial infections
• Aim to engineer a system capable of mixed
  initiative dialogue about kind of problem MYCIN
  solves
• Presents successive specific cases to convey
  knowledge to students in realistic
  problem-solving context
• Guides students reasoning
• Separation of domain and pedagogical knowledge.
  Tutorial system as Expert System itself (c.200
  rules)

13
How students use Guidon

• Case selected, initial data presented to student
• Student acts as diagnostician, asks questions,
  gathers further data, proposes hypotheses and
  requests help
• Guidon guides student's reasoning in solving
  case, using MYCIN's rules as 'issues'
• Student's behaviour compared with MYCIN's
• Guidon intervenes if
• student asks for help
• if student behaviour suboptimal (e.g. asks
  irrelevant questions)
• Dialogue of intervention guided by tutoring
  rules
• Uses MYCIN's reasoning in explanations

14
Excerpt from dialogue

• Guidon You should now proceed to ask questions
  that will enable you to make a hypothesis about
  the organisms (other than those seen on cultures
  or smears) which might be causing the infection.
  
• Student has initial data to go on at this point
• HOW IS THE FACT THAT THE PATIENT IS BURNED
  RELEVANT?
• Guidon The fact that the patient is seriously
  burned will be used to conclude that
  Pseudomonas-Aeruginosa is one of the organisms
  which might be present (in addition to those seen
  on cultures or smears) - Rule 578

15
Dialogue, continued

• HELP
• Guidon Try to determine the type of the
  infection bacterial, fungal, viral or TB.
• WHAT IS THE PATIENT'S WBC?
• Guidon The white bloodcell count from the
  patient's peripheral CBC is 1.9 thousand.

16
Consider domain rule 578

• IF
• the infection needing therapy is meningitis
• organisms were not seen on the stain of the
  culture
• the type of the infection is bacterial
• the patient has been seriously burned
• THEN
• Pseudomonas Aeruginosa might be one of the
  organisms (other than those seen on cultures or
  smears) causing the infection (0.5)

17
What is happening in this extract?

• Rule 578 applies to the patient in question,
• Student asks for help,
• GUIDON
• chooses a rule to discuss (one just mentioned)
• chooses a way to present it - tells student to
  work on untackled subgoals in the rule (type of
  infection)
• This becomes the next topic
• Next question relevant so GUIDON does not
  intervene.

18
Notes on guidon

• GUIDON used a version of MYCIN's rule base which
  had some extra annotations, such as further
  canned text suitable for explaining parts of the
  rules -- for example, how some lab test is
  performed.
• It was also able to filter out much of the
  medically irrelevant parts of a rule (in order to
  pick medically relevant subgoals to focus upon)
  by making comparisons with similar rules to see
  how they differed.

19
Some Problems

• Suitable domain for expert system does not imply
  suitable domain for ITS focus on EXPERTISE not
  on LEARNER (student as subset of knowledge base)
• Rules may encode expert knowledge but control
  stucture/reasoning strategy not same
• forces MYCIN's top-down strategy on user
• can reject user's reasonable hypothesis
• Also, rules may be too complex for novices
• hard to understand/remember/make sense of
• no distinction between different types of
  conditions,
• e.g. which most critical/or easiest to test or
  eliminate
• Cost-effective Expert System may not be effective
  for ITS

20
e.g. rule 507

• IF
• the infection needing therapy is meningitis
• organisms were not seen on the stain of the
  culture
• the type of the infection is bacterial
• the patient does not have a head injury, and
• the age of the patient is between 15 and 55 years
• THEN the organisms that might be causing the
  infection are diplococcus-pneumoniae(.75) and
  neisseria-meningitidis(.74)
• Mixes test data, age, strategic knowledge, meta
  interpreter knowledge, initial data

21
Neomycin and Guidon 2

• Collected protocols of experts' diagnosis and
  teachers articulating reasoning teachers'
  explanations more general, not specific to
  medical domain
• Re-configured MYCIN to get explicit model of
  "diagnostic thinking
• separation of strategic K from domain facts and
  rules
• metarules representing hierarchical reasoning
  strategy notion of hypotheses
• Changed some ordering of conditions in rules
• Organised rules into types of information
  general principles common world realities
  definitional and taxonomic relations causal
  relations heuristic rules.
• Wider range of diseases covered decrease in
  number of questions justifications and
  explanations in terms of strategic goals and r.e.
  specific hypotheses

22
3. Qualitative Models
23
Qualitative Models

• Concern with reasoning in qualitative terms about
  the causal structure of world.
• Allow reasoning about dynamic processes.
• Not necessarily same as cognitive fidelity but
  does aim for it.

24
QUEST White and Frederiksen (1986)

• The domain is electrical circuits.
• Internal representation uses causal calculus
  basically component-oriented, but incorporates
  some higher-level concepts guiding evaluation of
  component states.
• Development of student modelled in progressions
  of mental models
• the model worked with changes as the students
  understanding of the domain progresses
• the more advanced the students understanding,
  the higher level the model

25
White and Frederiksen, 1987, p. 282

• "In developing this theoretical framework, our
  research has focussed on
• modelling possible evolutions in students
  reasoning about electrical circuits as they come
  to understand more and more about circuit
  behaviour,
• and on using these model progressions as the
  basis for an intelligent learning environment
  that helps students learn
• a. to predict and explain circuit behaviour, and
• b. to troubleshoot by locating opens and shorts
  to ground in series-parallel circuits"

26
QUEST a circuit amenable to zero-order
qualitative reasoning (White Frederiksen, 1986)

• In order for the bulb to light, there must be a
  voltage drop across it. There is a device in
  parallel with the bulb, the switch. Two devices
  in parallel have the same voltage across them.
  Voltage drop is directly proportional to
  resistance If there is no resistance, there can
  be no voltage. Since the switch has no
  resistance, there is no voltage drop across the
  switch. Thus, there is no voltage drop across the
  light, so the light will be off.

27
De Kleers ENVISION Theory, 1983

• Mechanistic Mental Models ( causal/qualitative)
• Reasoning about physical devices
• Causal model of buzzer envisionment
• links components with respect to behaviour
• describes device in terms of component states,
  changes in states and consequences for other
  components
• Model can be run on specific inputs to yield
  predictions.
• Envision theory attempts to provide modelling
  framework for reproducing causality from
  structure
• Models expert/scientist's knowledge

28
Construction of a causal model for a buzzer
(adapted by Wenger, 1987, from de Kleer and
Brown, 1983)
29
Further figures from Wenger, 1987
30
Qualitative Process Theory - Forbus, 1984

• Reasoning about processes
• Attempts to provide a language for encoding
  causality as perceived by people (more like
  naive physics
• Qualitative Process Theory description of a
  process of heat transfer, (from Wenger, 1987,
  after Forbus, 1984)

31
Process heat-flow

• Individuals
• source an object, Has-Quantity(source, heat)
• destination an object, Has-Quantity(destination,
  heat)
• path a Heat-Path,Has-Connection(path,source,dest
  ination)
• Preconditions
• Heat-Aligned(path)
• Quantity Conditions
• A temperature (source) A temperature
  (destination)
• Relations
• Let flow-rate be a quantity A flow-rate
  ZERO
• flow-rateaQ(temperature(source)-temperature(dest
  ination))
• temperature (source)aQ heat(source)
• temperature (destination)aQ
  heat(destination)
• Influences
• I- (heat(source), Aflow-rate)
• I (heat(destination), Aflow-rate)

32
Causal model for the Cerrado communities (Salles
et al)
33
Deriving Explanations from Qualitative Models
34
4. Cognitive Models
35
Cognitive Models

• Anderson says that these are essential to
  producing high-performance tutoring systems.
• Some supposedly Cognitive Models
• may lack Psychological validity
• may only really be Glass Box Models
• Sophie relied primarily on Quantitative
  simulation.
• Qualitative reasoning was needed to provide more
  causal reasoning

36
ITS's built on Cognitive models

• Take particular cognitive theory or framework as
  a starting point
• Based on theoretical assumptions that they imply
• Example 1
• tutors produced by Anderson's group at CMU
• based on ACT-R
• various aspects of Maths and Programming.
• used in many schools and adult training
  situations
• Example 2
• tutors produced by Johnsons group at ISI, USC
• use the SOAR cognitive framework
• form the basis of STEVE and others