Theories in Formation TIF

1
Theories in Formation (TIF)

• DAY ONEEnd-to End System and
• CP Scenarios

2
Theories in Formation (TIF)Overview

• Functionality
• Integrates knowledge acquired using separate
  interfaces and modalities (Cmaps, NL, sample
  cases, sketching, etc.)
• Holds knowledge that is not yet in appropriate
  form for the KS and transforms it into formal
  representations
• Relates the different knowledge inputs among
  themselves and to the existing KB and thus
  detects possible inconsistencies
• Brings to bear relevant background k to help
  formalize inputs
• Assists the user in resolving inconsistencies and
  other issues that may arise as these connections
  are being worked out
• Contribution to the overall architecture
• Enable combined use of alternative input
  modalities
• Proactive assistance to user entering new
  knowledge
• Guard and/or inform K Server (KS) about possibly
  invalid statements

3
What can be Found in a Theory in Formation 1)
Content Knowledge

• Formal and semi-formal knowledge structures
• E.g., Cmaps, examples, analogies, sketches
• Connections among k strucs (given or implied by
  user)
• E.g., the acid mentioned in the virus
  degradation process is the same acid that
  appears in an earlier definition of a lysosome
• Connections of new k to relevant background
  knowledge
• E.g., the fermentors used in the Anthrax prod
  model are defined by the KS concept
  fermentor-container
• Assertions derived or hypothesized by the system
• E.g., aflatoxin is likely to be a virus based
  on what is said about it
• New terms (not yet defined but known by
  reference)
• E.g., the user mentioned the use of a rotary
  evaporator to dry toxins, but has not defined
  what it is (hyp 1 it is equipment hyp 2 it is
  a stimulant substance)

4
What can be Found in a Theory in Formation 2)
Meta-knowledge

• Alternative hypotheses and interpretations
• E.g., user said Lybia produces Anthrax much like
  Irak does, yet user also stated that Lybia does
  not have chemfacs which implies it does not have
  centrifugal separators. Hyp 1 the dispersion
  process must be done differently than Iraks Hyp
  2 there must be alt equpt other than cent. seps.
  that can be used
• Source attributions
• E.g., P was stated by user, Q was deduced by the
  KS, R was hypothesized by the TIF, S is the case
  in an example
• Qualification statements and their rationale
• E.g., assumption P generated correct answers in
  lines of reasoning R1R7 S is inconsistent with
  T because...
• Links to dialogue/session history

5
Theories in Formation
Users input
Text fragments
Established connections
Hypotheses and assumptions
Qualifications
Lines of reasoning other deductions
FRINGE OF THE KS
Relevant background knowledge
((( )) ())))
(defconcept bridge ()))
6
Basic Principles to Focus Acquisition (I)

• Principle of practical validation
• Invalid/incomplete statements are more likely to
  appear in k fragments that have not been
  exercised by using them to solve problems or
  answer questions
• Principle of experiential context
• Invalid/incomplete statements are more likely to
  appear in k fragments where limited prior
  knowledge (theories, components, models, etc.)
  can be or has been brought to bear
• Principle of local consistency
• Inconsistencies are more likely to appear in k
  fragments that have not been defined and/or
  cannot be viewed in proximity (spatial, temporal,
  representational, or inferencial) by the user

7
Basic Principles to Focus Acquisition (and II)

• Principle of purposeful knowledge capture
• Additional knowledge should be sought until the
  system can perform the desired tasks, other
  additional knowledge is desirable but not
  strictly necessary
• Principle of organized interaction (POI)
• A system that is proactively assisting the user
  can be more effective if the interaction with the
  user is organized and set in context (by topic,
  by a typical acquisition strategy, by priority)

8
Basic Functions

• Knowledge formation
• Transform knowledge into a form adequate for the
  KSs reasoners
• Knowledge assimilation
• Validate knowledge, note inconsistencies, detect
  gaps
• Knowledge extraction
• Guide acquisition of additional knowledge by
  generating and organizing follow-up questions to
  user

9
The Role of the KS

• KS (including analogy) is invoked to
• Generate deductions from certain sets of
  statements
• E.g., given what dual-use equipment does this
  country own
• Solve problems or answer questions
• E.g., PQs for EKCP, views
• Suggest relevant models/theories/principles
• E.g., could fit the model of a release
  process and how?
• Seek related cases or examples of certain
  statements
• E.g., do you already know about any countries
  that seem to be capable of producing Anthrax but
  do not have fermentors?
• Generate explanations
• Perform KB diagnostics (KSL)

10
Structuring the User Interaction

• Generating follow-up questions
• Based on inconsistencies and knowledge gaps
  detected
• Based on potential suggestions of applicable
  models/theories
• Based on plausible hypotheses and assumptions
  generated
• Organizing and prioritizing follow-up questions
• Coherent dialogue Easier for user if system
  brings up together questions on a topic
• Adequate sequencing The answers to some
  questions may help resolve others
• KA strategies guide user through typical KA
  tasks such as placing a new object within a
  hierarchy, filling attribute/value pairs through
  tables, specializing a process description, etc.

11
Integration with Overall Architecture
K. server API
Semi-structured knowledge/ assertions
INTERACTION
K S E R V E R
MANAGER
Knowledge Formation
Newly formalized knowledge
Follow-up questions
FRINGE OF THE KB
Dialogue/ Session History
12
Development Plans

• Version 1 (Jan 2001?)
• Capabilities
• Integrate inputs from alternative UIs (conc.
  maps, WebMod, WebNolgoy)
• Push knowledge into KS, thus augmenting systems
  Q/A capabilities
• Generate follow-up acquisition questions to guide
  user to clarify/fix/resolve inconsistencies and
  other issues
• Limitations
• Inputs from UIs are potential KS assertions (no
  semi-formal strucs)
• All UIs acquire knowledge using the KS as
  guidance (e.g., all new processes need to be
  specified as extensions to existing process
  descriptions, no free-form input)
• Version 2 (Jan 2002?) No limitations -)
• Version 3 (??) Integrates inputs from multiple
  users
• Techniques will be essentially the same
• Will exploit Ocelots capabilities

13
Theories in Formation (TIF)

• DAY TWOComponent Evaluations

14
Claims and Experiment Overview

• Our claims are stated in terms of the basic
  principles used by the system
• System will be developed so that each principle
  is operationalized in certain steps or submodules
  
• Steps or submodules used to implement a principle
  can be disabled to build ablated versions of the
  tool
• Gold standard a consistent, complete body of
  domain knowledge that is removed from the KS for
  the experiment
• Users re-enter that body of knowledge using full
  or ablated version of the system
• Claims can be tested in a variety of contexts at
  different stages of KA, with different kinds of
  k, etc.

15
Claim 1 Principle of Practical Validation

• Principle of practical validation (PPV)
• Invalid/incomplete statements are more likely
  to appear in k fragments that have not been
  exercised by using them to solve problems or
  answer questions
• Claim
• Users will enter fewer incomplete/incorrect
  statements
• because system follows PPV and thus
• tests knowledge in the context of
  specific tasks

16
Claim 2 Principle of Experiential Context

• Principle of experiential context (PEC)
• Invalid/incomplete statements are more likely
  to appear in k fragments where limited prior
  knowledge (theories, components, models, etc.)
  can be or has been brought to bear
• Claim
• Users will enter knowledge more
  efficiently more correctly in
• topics where system has relevant
  background k to bring to bear
• because system follows PEC and thus
• can use background knowledge to check,
  augment,
• and structure new knowledge

17
Claim 3 Principle of Local Consistency

• Principle of local consistency (PLC)
• Inconsistencies are more likely to appear in
  k fragments that have not been defined and/or
  cannot be viewed in proximity (spatial, temporal,
  representational, or inferencial) by the user
• Claim
• Users will enter fewer incorrect/invalid
  statements
• because system follows PLC and thus
• makes explicit the connections across
  knowledge
• fragments and analyzes them

18
Claim 4 Principle of Purposeful Knowledge Capture

• Principle of purposeful knowledge capture
  (PPKC)
• Additional knowledge should be sought until
  the system can perform the desired tasks, other
  additional knowledge is desirable but not
  strictly necessary
• Claim
• Users will enter knowledge more efficiently
• because system follows PPKL and thus
• focuses on acquiring knowledge
  relevant/needed
• for intended tasks

19
Claim 5 Principle of Organized Interaction

• Principle of organized interaction (POI)
• A system that is proactively assisting the
  user can be more effective if the interaction
  with the user is organized and set in context (by
  topic, by a typical acquisition strategy, by
  priority)
• Claim
• Users will enter knowledge more efficiently
  and more correctly
• because system follows POI and thus
• organizes clarification and correction
  questions
• resulting in more effective
  communication of each
• follow-up question and its context