Knowledge acquisition (Ch. 17 Durkin)

1
Knowledge acquisition (Ch. 17 Durkin)

• knowledge engineering building expert systems
• knowledge acquisition process of extracting
  knowledge from an expert, organizing it, and
  encoding it into a knowledge base
• knowledge elicitation extracting knowledge from
  an expert
• knowledge acquisition is the principle bottleneck
  in expert system development
• many techniques and theories about how to best do
  this
• more tools are appearing to help in this
• early example inductive inference tables
• active research area
• psychologists are especially interested in
  elicitation issues, as it is a fundamental
  problem of human psychology

2
Knowledge acquisition
Expert
data, problems, questions
knowledge concepts solutions
Formalized structured knowledge
KNOWLEDGE BASE
Knowledge engineer
Needs, usability, feedback
Prototypes, needs queries
End user
Also other experts, literature
3
Some problematic phenomena
1. Paradox of expertise The more competent a
domain expert is, the less able she is to
describe the knowledge they use to solve
problems. - studies experience shows that
experts are experts because they compile
their vast knowledge into compact, efficiently
retrievable form - as a result, they ignore
lots of details about how they derive
conclusions --gt intuition is prevalent
structured principles are ignored - for
example, experts use lots of generalization and
pattern matching to solve standard and new
problems 2. Experts make bad knowledge
engineers - domain experts are the worst
people for formalizing their own knowledge -
non-objective, unfamiliar with AI technology,
... - need an objective view of knowledge,
which isnt possible from expert - eg. try
to formalize how you go about creating a
computer program to solve some problem
4
Some problematic phenomena
3. Don't believe everything experts say.
experts rely on intuition, compiled knowledge
unaware of the deep reasoning use shallow
reasoning ie. often short-term memory isnt
usedrather, long-term memory as obtained via
past experiences is relied upon ---gt huge
gaps in knowledge because experts don't know
the formal structure of their knowledge, their
descriptions will likely be wrong - they arent
used to verbalizing their expertise!
therefore, knowledge engineer must watch for
knowledge that is... - irrelevant,
incomplete, incorrect, inconsistent - knowledge
engineer will formalize an expert's knowledge,
and then test it to see whether it is
logically consistent
5
Steps in knowledge acquisition

• 1. Collect (elicitation)
• - getting the knowledge out of the expert
• - most difficult step
• - lots of strategies
• 2. Interpret
• - review collected knowledge, organize, filter
• 3. Analyze
• - determining types of knowledge, conceptual
  relationships
• - determining appropriate knowledge represention
  inference structure
• 4. Design
• - extracting more knowledge after using above
  principles
• Lets look at these in more detail...

6
Tasks of main players

• Durkin 17.4

7
Preliminary steps

• Durkin 17.7

8
Interviews and questions

• Interacting with the expert is the primary means
  of eliciting knowledge
• 17.9, 17.10

9
Interview strategies

• there are different interview techniques some
  are suited to different phases of the elicitation
  process
• Funnel sequencing technique interview progresses
  from general, exploratory questions, to detailed
  questions
• Prompts Indirect
  Beginning of topic General
• Probes
• Direct
  End of topic Concrete

SUMMARIZE INTERVIEW
10
1. Unstructured interview

• a spontaneous, natural means to let expert talk
  freely on anything in domain
• expert verbalizes responses to general questions
  asked by KE
• stream of consciousness sometimes used
• KE keeps a minimal level of focus on topics
  discussed
• goal not to let KE unduly influence early
  explorations of knowledge
• 17.14, 17.15

11
2. Structured interview

• much more focussed and disciplined than
  unstructured interview
• KEs task is to discover concrete information
  about specific questions
• topic to be explored has been established at
  earlier sessions
• not as exploratory as unstructured --gt better for
  advanced phases
• 17.18, 17.19

12
To interview or not to interview

• Interviewing is primary means of knowledge
  elicitation.
• However, there are weaknesses
• procedural knowledge difficult to verbalize
• easier to do than to describe
• plus some knowledge (physical, artistic) not
  easily verablized
• ineffective long-term memory
• expert just doesnt remember details of problems
• compiled knowledge is difficult to reconstruct
• Case studies another strategy useful in concert
  with interviews

13
3. Retrospective case study

• ask expert to review and explain a solved case
• expert goes over all the steps, explaining as she
  or he goes
• KE will record the protocol the sequence of
  problem-solving steps or strategies used by
  expert
• types of case studies
• a) familiar case a typical vanilla case
• general info is obtained
• best for early phases when foundations are sought
• b) unusual case a new problem hereforeto unseen
  by expert
• good way to get deeper, detailed, more
  introspective expert feedback
• best for intermediate, later stages
• 17.22, 17.23

14
4. Observational case study

• rather than giving expert the whole case, just
  supply the problem description
• then watch record the expert as he or she
  solves the problem
• stream of consciousness useful
• both familiar and unfamiliar problems can be used
• familiar more general knowledge obtained
• unfamiliar detailed, deeper insight into problem
  solving obtained
• 17.26, 17.27, 17.30, 17.31

15
Summary strategy effectiveness

• 17.32, 17.33, 17.34

16
Analyzing the knowledge

• 1. data from expert interview observation is
  then transcribed into text form
• important to document all data date, who,
  what,...
• 2. the text is interpreted
• identifying chunks labelling key parts of the
  knowledge
• what portions of knowledge? what are they?
• 3. Analyzing (sorting) the knowledge
• interrelating the knowledge with previous
  sessions
• determining its representation in
  domain-friendly notation
• converting it to KB language
• this is done iteratively and incrementally
• must pass it by expert for confirmation and
  corrections
• knowledge dictionary akin to data dictionary
  in DB systems
• a system document that indexes all terms, rules,
  etc

17
Example transcript (step 1)

• 17.11

18
Interpreted Transcript (step 2)

• 17.12

19
Interpreting transcript

• 17.36, 17.37

20
Knowledge analysis

• Graphical representation of knowledge is an
  effective means of organizing it
• both KE and expert can understand
• idea is that graphical notations close the
  semantic gap between expert knowledge and
  formalized form
• Some techniques
• cognitive maps hierarchical, frame-like graphs
• inference networks/trees AND-OR tree
• flowcharts great for procedural knowledge
• decision tree
• example table (from which decision tree, neural
  net derivable)
• contemporary knowlege engineering tools
  incorporate graphical denotations of KB

21
Graphical representations

• 17.7, 17.8, 17.9, 17.10

22
Conclusion
research in AI, psychology is forming models of
how people experts organize knowledge,
learn, and do problem solving - these models
will give means for determining the best way to
extract knowledge from experts, and encode it
into a KB in the meantime, knowledge engineers
(experts themselves) rely on experience for
acquiring knowledge and constructing expert
systems - what about an expert system for
creating expert systems? KE is quite an
interesting and challenging - lucrative
profession - active research area