Workshop on Explanation in CBR: Summary

1
Workshop on Explanation in CBR Summary
David W. Aha Intelligent Decision Aids
Group Naval Research Laboratory Washington, DC
USA david.aha_at_nrl.navy.mil
Explanation Weve a long way to our goal
2
Context
Motivation
Research on some recent topics (e.g., on product
recommendation, mixed-initiative systems,
intelligent tutoring systems) indicate that
traditional approaches to explanation in CBR may
no longer be adequate.
Objective
Encourage contributions that emphasize the use of
explanation to support CBR processes, and
subsequent discussion
3
Translation
Now is the right time to return to the roots of
explanation/CBR, which has its roots in part with
the work of Leake, Aamodt, and others.
Aamodt
Leake
4
Schedule
Welcome and Introduction (David Leake) Session 1
Explanation in Knowledge-Intensive CBR Knowing
What to Explain and When (Jörg Cassens) Explanatio
n-Boosted Question Selection in Conversational
CBR (Mingyang Gu Agnar Aamodt) Session 2
Advances in Explanation Engineering Explanation
Goals in CBR (Frode Sørmo Jörg Cassens) A
Case-Based Explanation System for Black-Box
Systems (Conor Nugent and Pádraig
Cunningham) Visualization of CBR for Explanation
(Stewart Massie, Susan Craw, Nirmalie
Wiratunga) Explaining Similarity in CBR (Eva
Armengol, Santi Ontañón, Enric Plaza) Session
3 Potential New roles of Explanation in CBR On
the Role of Explanation for Hierarchical CBP in
RTS Games (Héctor Muñoz-Avila David W.
Aha) Explanation in Recommender Systems (David
McSherry) Thinking Positively Explanatory
Feedback for Conversational Recommenders (Kevin
McCarthy, James Reilly, Lorraine McGinty, Barry
Smyth) Discussion Future Challenges Research
Opportunities (Derek Bridge)
5
Key Questions
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
6
Key Questions
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
7
Explanation Goals in CBR(Frode Sørmo Jörg
Cassens)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (i.e., context, receiver goals and
  capabilities) explicit to choose appropriate
  explanations

Jörg Cassens
Explanation types

• Justification Explain why the answer is good
• Transparency Explain how the system reached an
  answer
• Relevance (in CCBR) Explain why a question asked
  is relevant
• Learning (ITS) Teach the user about the domain

Novelty

• Short survey on theories of explanation
• Extended discussion on the limitations of cases
  as explanations
• In complex domains, may fail Transparency goal
• For novice users, may fail Justification goal

8
Explaining Similarity in CBR(Eva Armengol, Santi
Ontañón, Enric Plaza)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (local feature interactions)
  explicit so that we can justify a problems
  solution

Santi Ontañón
Novelty

• Explanation as similitude terms (conjuncts of
  local features)
• C-LID Search for (and cache/reuse) best
  similitude term, which encodes similarities among
  query and retrieved cases

Tasks

• Classification
• Multi-agent learning

9
A Case-Based Explanation Systemfor Black-Box
Systems (Conor Nugent and Pádraig Cunningham)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Goal Make knowledge (prediction) explicit using
  CBR
• Means Make knowledge (local feature relevance)
  explicit to support retrieval

Pádraig Cunningham
Observations Novelty

• People are reluctant to accept predictions of
  non-transparent ML approaches
• Yet approaches that reveal them trade off
  fidelity for interpretability
• Solution Use a case-based approach

Task
Numeric prediction
10
Explanation in Recommender Systems
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (relations among remaining cases)
  explicit to most quickly confirm a target case
  for recommendation

David McSherry (The Royal Me)
Novelty/Approach

• Domination maximization in Top Case Selects the
  attribute that maximizes the number of cases
  dominated by the target case

11
Thinking Positively Explanatory Feedback for
Conversational Recommender Systems(Kevin
McCarthy, James Reilly, Lorraine McGinty, Barry
Smyth)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (trends in the remaining cases)
  explicit so that the user can continue
  conversations efficiently

Lorraine McGinty
Novelty

• Dont look backwards after a failure has occurred
• Instead, look forwards so as to prevent them

Approach Dynamic critiquing

• Dynamically capture local feature relevance
• Reveal trends in interactions among them
• Display them to the user

12
Explanation-Boosted Question Selection in
Conversational CBR(Mingyang Gu Agnar Aamodt)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (meta-relations) explicit so as to
  support question ranking via dialogue inferencing

Mingyang Gu
Novelty

• Use of explanation paths for dialogue inferencing
• Conversational CBR approach w/ structured
  representations

Approach

• Knowledge model Meta- and Object-levels
• Explanation path construction via
• Subclass inheritance
• Plausible inheritance
• 3-phase reasoning process Activate, Explain,
  Focus

13
Knowing What to Explain and When
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (workflow) explicit to reason on
  user goals context to present explanations when
  problems occur

Jörg Cassens
Novelty
Activity Theory Used for analyzing work
processes Activities Top-level a set of actions
(drive a car) Actions Performed consciously a
set of operations (change lanes) Operations
Non-conscious primitives (change gears)
Approach

• When breakdown situations (i.e., when failures
  occur) occur in an action cycle, operations can
  become conscious
• Task Planning (pilot training)

14
On the Role of Explanation for Hierarchical CBP
in RTS Games(Héctor Muñoz-Avila David W. Aha)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make (planning) knowledge explicit to support
  explanations for a variety of tasks

Héctor Muñoz-Avila
t
t
Novelty

• Explanation as a subset of annotated HTN plan
  elements

case
case
1
2
1
t
t
t
t
t
t
t
t
t
t
11
12
13
14
15
11
12
13
14
15
tasks
tasks
Tasks

• Strategy selection
• Course-of-action outcome
• Model update
• Prediction

t
t
t
t
j
k
j
k
t
t
t
t
t
t
t
t
t
t
j1
j3
j2
k1
k2
j1
j3
j2
k1
k2
plan

plan

a
a
a
a
a
a
a
a
a
a
1
2
3
4
5
1
2
3
4
5
15
Visualization for CBR for Explanation(Stewart
Massie, Susan Craw, Nirmalie Wiratunga)
What is explanation in CBR? What tasks is it good
for? When/how should it be used? How does it
work? How can we measure its effects?
Theme

• Make knowledge (e.g., case base) explicit to
  inform the user who will perform adaptation

Susan Craw
Novelty
Visualization
Approach

• Parallel coordinate graphs
• System FormuCaseViz
• Task Design (of tablets)
• Initial user study

16
Summary of Trends
Observations

• Good things happen by explicating useful
  knowledge
• For the user and/or
• For the system to use
• Lots of interest in using local knowledge to
  support explanations
• Confidence in explanations is an ongoing research
  issue
• Explanations can play roles in any type of
  problem-solving task

Types of explanations in CBR

• Analysis tasks
• Classification/Prediction Local meta-features
• Diagnosis Local case meta relations, local
  meta-features meta-relations
• Synthesis tasks
• Planning User goals, annotated (HTN) plan
  elements
• Design Case representation and relations
  (visual) among cases

17
Fini
Explanation Weve moved a wee bit closer to our
goal