KANAL: Knowledge ANALysis

1
KANALKnowledge ANALysis

• Jihie Kim
• Yolanda Gil
• USC/ISI
• www.isi.edu/expect/rkf/

2
Role of Knowledge Analysis in SRI Team

• To point out to the Interaction Manager what
  additional K needs to be acquired or what
  existing K needs to be modified
• To guard the knowledge server from invalid
  statements entered by the user

3
Approach Using Interdependency Models

• Relating different pieces of Knowledge among
  themselves and to the existing KB
• (e.g., how different pieces of knowledge are put
  together to generate an answer)
• Successfully used in checking problem-solving K
  in EXPECT (Gil Melz 96 Kim Gil 99)

4
Current Focus Checking Process Models

• Verification checks model is correct (e.g., no
  steps missing
• Validation checks model is as user intended
  (e.g., alert user of impossible paths)

Interaction Manager
KM
UI
5
Validating Complex Process Models
Lambda Virus Invasion 2


Transcribe
Assemble
Enter

Replicate
Arrive

Integrate
Divide
Disintegrate
Circularize
Synthesize
Copy
6
Describing Process Models (Composed Concepts)

• Each individual step has
• Preconditions, Add-list, Delete-list
• Links among the steps
• Decomposition links between steps and substeps
• Disjunctive alternatives
• Temporal links

7
Checks on Process Models

• All the steps are properly linked (substep,
  nextstep, disjunctive nextstep, conjunctive
  nextstep, )
• All the preconditions of each step are satisfied
  during the simulation
• All the expected effects can be achieved
• There are no unexpected effects
• There are no impossible paths
• . . .

8
Current Focus Dynamic Checks

• Simulation (or symbolic execution) results show
  how substeps of the process model are related
  each other (Interdependency Model)
• Perform various kinds of checks
• unachieved preconditions
• expected/unexpected effects
• disjunctive branches
• loops
• causal links
• redundancies
• unordered steps

Implemented
9
Checking Unachieved Preconditions

• During simulation, collect unachieved
  preconditions by tracing failed expressions
• Suggest fixes
• Add a step that can achieve the condition
• Add ordering constraints between the failed step
  and another step that undid the condition
• Delete the step
• . . .

VirusInvasion
Failed Precondition Virus near Cell Proposed
Fixes Add an Arrive step Add a Move step
. . .
Enter
Integrate
10
Checking Effects

• Compute the effects by simulation
• Suggest fixes for unachieved expected effects
• Add steps that can achieve the effect
• Add ordering constraints between effect adding
  steps and effect deleting steps
• Check unexpected effects

After VirusInvasion ProteinCoat of the virus
broken ? Achieved DNA of the virus has
replicates ? Unachieved ltProposed Fixesgt Add a
Replicate step Add a Divide step
11
CheckingDisjunctive Branches

• Inform all the combinations of alternatives so
  that the user can check if some are impossible
• KANAL can simulate and highlight disjunctive
  paths

12
Example Lambda Virus Invasion
(From Alberts ECB Chapter 9)
ltPaths Simulatedgt Path1 Arrive1 ? Enter2 ?
Circularize3 ? Integrate4 ? Divide5 ?
Disintegrate6 ? Synthesize7 ? Replicate8 Path2
Arrive1 ? Enter2 ? Circularize3 ? Synthesize7 ?
Replicate8
13
ExampleConjunctive Branches
Life cycle of a virus (from Alberts ECB Chapter 9)
Transcribe
Enter
Assemble
Conjunction
Arrive
Replicate
ltSimulation sequencegt Arrive1 ? Enter2 ?
Trascribe3 ? Replicate4 ? Assembly5 Arrive1 ?
Enter2 ? Replicate4 ? Trascribe3 ? Assembly5
14
Checking Loops
ltLoops Foundgt Loop1 Arrive1 ? Enter2 ?
Circularize3 ? Integrate4 ? Divide5 ?
Disintegrate6 ? Synthesize7 ? Replicate8?
Arrive1 Loop2 Arrive1 ?Enter2 ? Circularize3 ?
Synthesize7 ? Replicate8 ?
Enter1 Loop3 Divide5? Divide5
15
Checking Causal Links

• Describe which step enables (or disables)
• a given step

ltCausal Linksgt Arrive1 enables Enter2 by
achieving Virus near Cell Integrate4 enables
Disintegrate6 by achieving
Virus DNA integrated with chromosome
16
Fixing Problems UsingInteraction Plans

• Interaction Plan describes how to proceed with
  the user interaction
• direct what to do next based on the results from
  K Analysis
• KANALs dialogue for fixing errors is implemented
  with interaction plans
• Will be integrated with the Interaction Manager

17
Keeping Track of Interaction History

• ...
• Choose what to simulate
• choose model VirusInvadesCell
• choose substep to test VirusInvadesCell
• Simulate model VirusInvadesCell
• simulate-steps--find-failed-events
• ask-to-fix-failed-event (failed
  preconditions of Enter)
• propose-fixes-for-failed-event
• ask-what-to-fix-for-failed-event
• ((the location of (the patient of
  Enter))
• (the space-near of (the agent of
  Enter)))
• ask-how-to-fix-failed-event (add Arrive
  before Enter)

18
Future Extensions (I)Static Checks

• Let user pose questions about various features of
  the process model to test the model
• KANAL will maintain test suites
• Users pick from sample query templates
• example retrieving role values, part-of
  relations, type definitions,..
• Users may specify their expected results
• Users may vary the initial situations to start
  from
• Explanation or trace of the answer to a query
  show how different pieces of K are used to
  generate the answer (Interdependency Model)

19
Future Extensions (II)

• Exploiting history and evolution of
  Interdependency Models (for both simulations and
  queries)
• Example Check what tests were correctly answered
  before
• Using heuristics to focus K analysis
• Example when invalid results are obtained, KANAL
  will use a divide-and-conquer strategy and check
  intermediate results to find the sources of the
  problem
• Testing with different initial states and
  different arguments

20
Future Extensions (III)

• Interdependency Models for problem solving
  knowledge
• EKCP
• Build on past work on EXPECT

21
Using KANAL for Intelligent Tutoring Systems

• ITSs can acquire domain knowledge from human
  instructor and use simulations to refine the
  knowledge (Johnson et al 2000, Scholer et al
  2000, Angros et al 99)
• We are exploring the use of KANAL to check and
  analyze the domain models while it is being built

22
Knowledge Authoring Environment for Tutoring
Systems (current)
Instructor
Student
Demonstration
Final Model (Lessons)
Steve Agent
Initial Model
Refined Model
Experimenter
23
Knowledge Authoring Environment for Tutoring
Systems (future)
Instructor
Student
Editor
Demonstration
Final Model (Lessons)
Steve Agent
Initial Model
Refined Model
Experimenter
KANAL