Knowledge Acquisition and Problem Solving

1
CS 785 Fall 2004
Knowledge Acquisition and Problem Solving
Agent Teaching and Multistrategy Learning
Gheorghe Tecuci tecuci_at_gmu.eduhttp//lac.gmu.edu
/
Learning Agents Center and Computer Science
Department George Mason University
2
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
3
What is Learning?

Learning is a very general term denoting the way
in which people and computers

• acquire and organize knowledge (by building,
  modifying and organizing internal representations
  of some external reality),
• discover new knowledge and theories (by creating
  hypotheses that explain some data or phenomena),
  or
• acquire skills (by gradually improving their
  motor or cognitive skills through repeated
  practice, sometimes involving little or no
  conscious thought).

Learning results in changes in the agent (or
mind) that improve its competence and/or
efficiency.
4
The Disciple agent is concerned with the first
type of learning acquiring and organizing
knowledge from a subject matter expert (by
building, modifying and organizing internal
representations of some external reality). The
external reality is a strategic scenario and how
a subject matter expert reasons to identify and
test strategic COG candidates for that scenario.
5
The architecture of an intelligent agent
Implements a general problem solving method that
uses the knowledge from the knowledge base to
interpret the input and provide an appropriate
output.
Intelligent Agent
Implements learning methods for extending and
refining the knowledge in the knowledge base.
Problem Solving Engine
Input/
Sensors
Learning Engine
User/ Environment
Output/
Ontology Rules/Cases/
Knowledge Base
Effectors
Data structures that represent the objects from
the application domain, general laws governing
them, actions that can be performed with them,
etc.
6
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
7
Generalization and specialization rules
Fundamental to learning are the processes of
generalization and specialization. We will
present several basic rules for generalizing or
specializing expressions representing concepts.
These rule are used to generalize concepts or to
specialize concepts.
A generalization rule is a rule that transforms
an expression into a more general expression.
A specialization rule is a rule that transforms
an expression into a less general expression.
The reverse of any generalization rule is a
specialization rule.
8
Turning constants into variables
Generalizes an expression by replacing a constant
with a variable.
The set of multi_group_forces with 5 subgroups.
?O1 is multi_group_force
number_of_subgroups 5
Japan_1944_Armed_Forces
generalization
specialization
Axis_forces_Sicily
5 ? ?N1
?N1? 5
Allied_forces_operation_Husky
?O1 is multi_group_force
number_of_subgroups ?N1
The set of multi_group_forces with any number of
subgroups.
9
The top expression represents the following
concept the set of multi group forces with 5
subgroups. This set contains, for instance,
Axis_forces_Sicily from the Sicily_1943 scenario.
By replacing 5 with a variable ?N1 that can take
any value, we generalize this concept to the
following one the set of multi group forces with
any number of subgroups. In particular ?N1 could
be 5. Therefore the second concept includes the
first one. Conversely, by replacing ?N1 with 5,
we specialize the bottom concept to the top
one. The important thing to notice here is that
by a simple syntactic operation (transforming a
number into a variable) we can generalize a
concept. This is how an agent generalizes
concepts.
10
Climbing/descending the generalization hierarchy
Generalizes an expression by replacing a concept
with a more general one.
democratic_government
representative_democracy
parliamentary_democracy
The set of single state forces governed by
representative democracies
?O1 is single_state_force has_as_governing_body
?O2 ?O2 is representative_democracy
generalization
specialization
representative_democracy ? democratic_government
democratic_government ? representative_democracy
The set of single state forces governed by
democracies
?O1 is single_state_force has_as_governing_body
?O2 ?O2 is democratic_government
11
One can also generalize an expression by
replacing a concept from its description with a
more general concept, according to some
generalization hierarchy. The reverse operation,
of replacing a concept with a less general one,
leads to the specialization of an
expression. The agent can also generalize a
concept by dropping a condition. That is, by
dropping a constraint that its instances must
satisfy. This rule is illustrated in the next
slide.
12
Dropping/adding condition
Generalizes an expression by removing a
constraint from its description.
The set of multi-member forces that have
international legitimacy.
?O1 is multi_member_force has_international_legit
imacy yes
generalization
specialization
?O1 is multi_member_force
The set of multi-member forces (that may or may
not have international legitimacy).
13
Generalizing/specializing numbers
Generalizes an expression by replacing a number
with an interval, or by replacing an interval
with a larger interval.
The set of multi_group_forces with exactly 5
subgroups.
?O1 is multi_group_force
number_of_subgroups 5
generalization
specialization
5 ? 3 .. 7
3 .. 7 ? 5
?O1 is multi_group_force
number_of_subgroups ?N1 ?N1 is-in 3 .. 7
The set of multi_group_forces with at least 3
subgroups and at most 7 subgroups.
generalization
specialization
3 .. 7 ? 2 .. 10
2 .. 10 ? 3 .. 7
?O1 is multi_group_force
number_of_subgroups ?N1 ?N1 is-in 2 ..
10
The set of multi_group_forces with at most 10
subgroups.
14
A concept may also be generalized by replacing a
number with an interval containing it, or by
replacing an interval with a larger interval. The
reverse operations specialize the concept. Yet
another generalization rule, which is illustrated
in the next slide, is to add alternatives. Accordi
ng to the expression from the top of this slide,
?O1 is any alliance. Therefore this expression
represents the following concept the set of all
alliances. This concept can be generalized by
adding another alternative for ?O1, namely the
alternative of being a coalition. Now ?O1 could
be either an alliance or coalition. Consequently,
the expression from the bottom of this slide
represents the following more general concept
the set of all alliances and coalitions.
15
Adding/removing alternatives
Generalizes an expression by replacing a concept
C1 with the union (C1 U C2), which is a more
general concept.
The set of alliances.
?O1 is alliance has_as_member ?O2
generalization
specialization
?O1 is alliance coalition
has_as_member ?O2
The set including both the alliances and the
coalitions.
16
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
17
Representative learning strategies

• Learning by analogy
• Instance-based learning
• Reinforcement learning
• Neural networks
• Genetic algorithms and evolutionary
  computation
• Bayesian Learning
• Multistrategy learning

• Rote learning
• Learning from instruction
• Learning from examples
• Explanation-based learning
• Conceptual clustering
• Quantitative discovery
• Abductive learning

18
Empirical inductive learning from examples

The learning problem
Given a set of positive examples (E1, ..., En)
of a concept a set of negative examples (C1,
... , Cm) of the same concept a learning bias
other background knowledge Determine a
concept description which is a generalization of
the positive examples that does not cover any
of the negative examples
Purpose of concept learning Predict if an
instance is an example of the learned concept.
19
Learning from examples
Compares the positive and the negative examples
of a concept, in terms of their similarities and
differences, and learns the concept as a
generalized description of the similarities of
the positive examples. This allows the agent to
recognize other entities as being instances of
the learned concept.
Illustration
Positive examples of cups P1
P2 ...
Negative examples of cups N1

Description of the cup concept
has-handle(x), ...
Requires many examples
Does not need much domain knowledge
Improves the competence of the agent
20
The goal of this learning strategy is to learn a
general description of a concept (for instance
the concept of cup) by analyzing positive
examples of cups (i.e. objects that are cups) and
negative examples of cups (i.e. objects that are
not cups). The learning agent will attempt to
find out what is common to the cups and what
distinguishes them from non-cups. For instance,
in this illustration, the agent may learn that a
cup should have a handle because all the positive
examples of cups have handles, and the negative
examples of cups do not have handles. However,
the color does not seem to be important for a cup
because the same color is encountered for both
cups and non-cups. To learn a good concept
description through this learning strategy
requires a very large set of positive and
negative examples. On the other hand, this is the
only information that the agent needs. That is,
the agent does not require prior knowledge to
perform this type of learning. The result of this
learning strategy is the increase of the problem
solving competence of the agent. Indeed, the
agent will learn to do things it was not able to
do before. In this illustration it will learn to
recognize cups, something that it was not able to
do before.
21
Explanation-based learning (EBL)
The EBL problem

Given A concept example cup(o1) Ü color(o1,
white), made-of(o1, plastic), light-mat(plastic),
has-handle(o1), has-flat-bottom(o1),
up-concave(o1),... Goal the learned concept
should have only operational features (e.g.
features present in the examples) BK cup(x)
Ü liftable(x), stable(x), open-vessel(x). liftabl
e(x) Ü light(x), graspable(x). stable(x) Ü
has-flat-bottom(x). ... Determine An
operational concept definition cup(x) Ü
made-of(x, y), light-mat(y), has-handle(x),
has-flat-bottom(x), up-concave(x).
22
Explanation-based learning
Learns to recognize more efficiently the examples
of a concept by proving that a specific instance
is an example of it, and thus identifying the
characteristic features of the concept.
A example of a cup cup(o1) color(o1, white),
made-of(o1, plastic), light-mat(plastic),
has-handle(o1), has-flat-bottom(o1),
up-concave(o1),...
Proof generalization generalizes them
The proof identifies the characteristic features
made-of(o1, plastic) is needed to prove cup(o1)
made-of(o1, plastic) is generalized to
made-of(x, y) the material needs not be
plastic.
has-handle(o1) is needed to prove cup(o1)
color(o1,white) is not needed to prove cup(o1)
23
The goal of this learning strategy is to improve
the efficiency in problem solving. The agent is
able to perform some task but in an inefficient
way. We would like to teach the agent to perform
the task faster. Consider, for instance, an agent
that is able to recognize cups. The agent
receives a description of a cup that includes
many features. The agent will recognize that this
object is a cup by performing a complex reasoning
process, based on its prior knowledge. This
process is illustrated by the proof tree from the
left hand side of this slide. The object o1 is
made of plastic which is a light material.
Therefore o1 is light. o1 has a handle and
therefore it is graspable. Being light and
graspable, it is liftable. And so on being
liftable, stable and an open vessel, it is a cup.
However, the agent can learn from this process to
recognize a cup faster. Notice that the agent
used the fact that o1 has a handle in order to
prove that o1 is a cup. This means that having a
handle is an important feature. On the other hand
the agent did not use the color of o1 to prove
that o1 is a cup. This means that color is not
important. Notice how the agent reaches the same
conclusions as in learning from examples, but
through a different line of reasoning, and based
on a different type of information. The next step
in the learning process is to generalize the tree
from the left hand side into the tree from the
right hand side. While the tree from the left
hand side proves that the specific object o1 is a
cup, the tree from the right hand side shows that
any object x that is made of some light material
y, has a handle and some other features is a cup.
Therefore, to recognize that an object o2 is a
cup, the agent only needs to look for the
presence of these features discovered as
important. It no longer needs to build a complex
proof tree. Therefore cup recognition is done
much faster. Finally, notice that the agent needs
only one example to learn from. However, it needs
a lot of prior knowledge to prove that this
example is a cup. Providing such prior knowledge
to the agent is a very complex task.
24
General features of Explanation-based learning
Needs only one example
Requires complete knowledge about the concept
(which makes this learning strategy
impractical).
Improves agent's efficiency in problem solving
25
Learning by analogy
The learning problem
Learns new knowledge about an input entity by
transferring it from a known similar entity.
The learning method

ACCESS find a known entity that is analogous
with the input entity. MATCHING match the two
entities and hypothesize knowledge.
EVALUATION test the hypotheses. LEARNING
store or generalize the new knowledge.
26
Learning by analogy illustration
Illustration The hydrogen atom is like our solar
system.
27
Learning by analogy is the process of learning
new knowledge about some entity by transferring
it from a known entity. For instance, I can
teach students about the structure of the
hydrogen atom by using the analogy with the solar
system. I am telling the students that the
hydrogen atom has a similar structure with the
solar system, where the electrons revolve around
the nucleus as the planets revolve around the
sun. The students may then infer that other
features of the solar system are also features of
the hydrogen atom. For instance, in the solar
system, the greater mass of the sun and its
attraction of the planets cause the planets to
revolve around it. Therefore, we may conclude
that this is also true in the case of the
hydrogen atom the greater mass of the nucleus
and its attraction of the electrons cause the
electrons to revolve around the sun. This is
indeed true and represents a very interesting
discovery. The main problem with analogical
reasoning is that not all the facts related to
the solar system are true for the hydrogen atom.
For instance, the sun is yellow, but the nucleus
is not. Therefore, facts derived by analogy have
to be verified. A general heuristic is that
similar causes have similar effects. That is, if
A is similar to A and A causes B. Then we would
expect A to cause B which should be similar to
B.
28
Learning by analogy illustration
Illustration The hydrogen atom is like our solar
system. The Sun has a greater mass than the
Earth and attracts it, causing the Earth to
revolve around the Sun. The nucleus also has a
greater mass then the electron and attracts
it. Therefore it is plausible that the electron
also revolves around the nucleus.
General idea of analogical transfer similar
causes have similar effects.
29
Multistrategy learning
Multistrategy learning is concerned with
developing learning agents that synergistically
integrate two or more learning strategies in
order to solve learning tasks that are beyond the
capabilities of the individual learning
strategies that are integrated.
30
Complementariness of learning strategies
Explanation- based learning
Multistrategy
Learningfrom examples
learning
Examples
several
many
one
needed
Knowledge
complete
incomplete
very little
knowledge
needed
knowledge
Type of
induction and/
induction
deduction
inference
or deduction
improves
Effect on
improves
improves
competence or/
agent's
competence
efficiency
and efficiency
behavior
31
The individual learning strategies have
complementary strengths and weaknesses. For
instance learning from example requires a lot of
example while explanation-based learning requires
only one example. On the other hand, learning
from examples does not require any prior
knowledge while explanation-based learning
requires a lot of prior knowledge.
Multistrategy learning attempts to
synergistically integrate such complementary
learning strategies, in order to take advantage
of their relative strengths to compensate for
their relative weaknesses. The Disciple agent
uses a multistrategy learning strategy, as will
be presented in the following.
32
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
33
The rule learning problem definition

GIVEN an example of a problem solving
episode a knowledge base that includes an
object ontology and a set of problem solving
rules an expert that understands why the
given example is correct and may answer agents
questions. DETERMINE a plausible version
space rule that is an analogy-based
generalization of the specific problem solving
episode.
34
Input example

I need to
Identify and test a strategic COG candidate
corresponding to a member of the
Allied_Forces_1943
Which is a member of Allied_Forces_1943?
US_1943
Therefore I need to
Identify and test a strategic COG candidate for
US_1943
This is an example of a problem solving step from
which the agent will learn a general problem
solving rule.
35
Learned PVS rule


IF Identify and test a strategic COG candidate
corresponding to a member of a force The force
is ?O1
IF Identify and test a strategic COG candidate
corresponding to a member of the ?O1
QuestionWhich is a member of ?O1 ? Answer?O2
explanation ?O1 has_as_member ?O2
Plausible Upper Bound Condition
?O1 is multi_member_force has_as_member ?O2
?O2 is force
THEN Identify and test a strategic COG candidate
for ?O2
Plausible Lower Bound Condition
?O1 is equal_partners_multi_state_alliance has_as
_member ?O2 ?O2 is single_state_force
INFORMAL STRUCTURE OF THE RULE
THEN Identify and test a strategic COG candidate
for a force The force is ?O2
FORMAL STRUCTURE OF THE RULE
36
This is the rule that is learned from the input
example. It has both a formal structure (used for
formal reasoning), and an informal structure
(used to communicate more naturally with the
user). Let us consider the formal structure of
the rule. This is an IF-THEN structure that
specifies the condition under which the task from
the IF part can be reduced to the task from the
THEN part. This rule, however, is only partially
learned. Indeed, instead of a single
applicability condition, it has two conditions
1) a plausible upper bound condition which is
more general than the exact (but not yet known)
condition, and 2) a plausible lower bound
condition which is less general than the exact
condition. Completely learning the rule means
learning an exact condition. However, for now we
will show how the agent learns this rule from the
input example shown on a previous slide. The
basic steps of the learning method are those from
the next side.
37
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
38
Rule learning method
Analogy and Hint Guided Explanation
Analogy-based Generalization
Plausible version space rule
plausible explanations
PUB
guidance, hints
Example of a task reduction step
PLB
Incomplete justification
analogy
Knowledge Base
39
Basic steps of the rule learning method
1. Formalize and learn the tasks
2. Find a formal explanation of why the example
is correct. This explanation is an approximation
of the question and the answer, in the object
ontology.
3. Generalize the example and the explanation
into a plausible version space rule.
40
1. Formalize the tasks

• Sample formalization rule
• obtain the task name by replacing each specific
  instance with a more general concept
• for each replaced instance define a task feature
  of the form The concept is instance

Task name
Task features


I need to
I need to
Identify and test a strategic COG candidate
corresponding to a member of the
Allied_Forces_1943
Identify and test a strategic COG candidate
corresponding to a member of a force The force
is Allied_Forces_1943
Therefore I need to
Therefore I need to
Identify and test a strategic COG candidate for a
force The force is US_1943
Identify and test a strategic COG candidate for
US_1943
41
Because the tasks from the modeling are in
unrestricted English Disciple cannot reason with
them. We need to formalize these tasks. For each
task we need to define an abstract phrase that
indicates what this task is about (the task
name), and a list of specific phrases that give
all the details about the task (the task
features). The task name should not contain any
instance (such as Allied_Forces_1943). All these
instances should appear in the task features. In
general, the task name may be obtained from the
English expression in the left hand side by
simply replacing each specific object with a more
abstract concept. Then we will add a
corresponding task feature that specifies the
value for this abstract concept.
42
Sample task formalizations

• Any other formalization is acceptable if
• the task name does not contain any instance or
  constant
• each instance from the informal task appears in a
  feature of the formalized task.

I need to
Identify the strategic COG candidatesfor the
Sicily_1943 scenario
Identify the strategic COG candidatesfor a
scenario
The scenario is Sicily_1943
Which is an opposing forcein the Sicily_1943
scenario?
Anglo_allies_1943
Therefore I need to
Identify the strategic COG candidatesfor
Anglo_allies_1943
Identify the strategic COG candidatesfor an
opposing force
The opposing force is Anglo_allies_1943
Is Anglo_allies_1943 a single member force or a
multi-member force?
Anglo_allies_1943 is a multi-member force
Therefore I need to
Identify the strategic COG candidates forthe
Anglo_allies_1943 which isa multi-member force
Identify the strategic COG candidates foran
opposing force which isa multi-member force
The opposing force is Anglo_allies_1943
43
Task learning
Identify and test a strategic COG candidate for
US_1943
Identify and test a strategic COG candidate for a
force The force is US_1943
Identify and test a strategic COG candidate for
?O1
object
subconcept_of
INFORMAL STRUCTURE OF THE TASK
force
subconcept_of
subconcept_of
multi_member_force
opposing_force
subconcept_of
subconcept_of

Identify and test a strategic COG candidate for a
force The force is ?O1
multi_state_force
single_member_force
subconcept_of
instance_of
instance_of
multi_state_alliance
Plausible upper bound condition ?O1 is
force Plausible lower bound condition ?O1 is
single_state_force
subconcept_of
Single_state_force
equal_partners_ multi_state_alliance
instance_of
instance_of
has_as_member
US_1943
FORMAL STRUCTURE OF THE TASK
Allied_Forces_1943
44
The top part of this slide shows the English
expression and the formalized expression of a
specific task. From the English expression of
the specific task the agent learns the informal
structure of the general task by replacing the
specific instance US_1943, with the variable
?O1. From the formalized expression of the
specific task, the agent learns the formal
structure of the general task. The formal
structure also specifies the conditions that ?O1
should satisfy. However, the agent cannot
formulate the exact condition, but only two
bounds for the exact condition that will have to
be learned. The plausible lower bound condition
is more restrictive, allowing ?O1 to only be a
single-state force. This condition is obtained by
replacing US_1943 with its most specific
generalization in the object ontology. The
plausible upper bound condition is less
restrictive. ?O1 could be any force. This
condition is obtained by replacing US_1943 with
the most general sub-concept of ltobjectgt which is
more general than US_1943. The plausible upper
bound condition allows the agent to generate more
tasks, because now ?O1 can be replaced with any
instance of force. However, there is no guarantee
that the generated task is a correct
expression. The agent will continue to improve
the learned task, generalizing the plausible
lower bound condition and specializing the
plausible upper bound condition until they become
identical and each object that satisfies the
obtained condition leads to a correct task
expression.
45
2. Find an explanation of why the example is
correct

I need to
Identify and test a strategic COG candidate
corresponding to a member of the
Allied_Forces_1943
Which is a member of Allied_Forces_1943?
US_1943
Therefore I need to
Identify and test a strategic COG candidate for
US_1943
The explanation is an approximation of the
question and the answer, in the object ontology.
has_as_member
US_1943
Allied_Forces_1943
46
The expert has defined the example during the
modeling process. During the task formalization
process, the expert and the agent have
collaborated to formalize the tasks. Now the
expert and the agent have to collaborate to also
formalize the question and the answer. This
formalization is the explanation from the bottom
of this slide. It consists of a relation between
two elements from the agent's ontology Allied_Fo
rces_1943 has_as_member US_1943 It states, in
Disciples language, that US_1943 is a member of
Allied_Forces_1943. An expert can understand such
formal expressions because they actually
correspond to his own explanations. However, he
cannot be expected to be able to define them
because he is not a knowledge engineer. For one
thing, he would need to use the formal language
of the agent. But this would not be enough. He
would also need to know the names of the
potentially many thousands of concepts and
features from the agents ontology (such as
has_as_member). While defining the formal
explanation of this task reduction step is beyond
the individual capabilities of the expert and the
agent, it is not beyond their joint capabilities.
Finding such explanation pieces is a
mixed-initiative process involving the expert and
the agent. In essence, the agent will use
analogical reasoning and help from the expert to
identify and propose a set of plausible
explanation pieces from which the expert will
have to select the correct ones. Once the expert
is satisfied with the identified explanation
pieces, the agent will generate a general rule.
47

We need to
3. Generate the PVS rule
Identify and test a strategic COG candidate
corresponding to a member of a force The force
is Allied_Forces_1943
has_as_member
US_1943
Allied_Forces_1943
Therefore we need to
Identify and test a strategic COG candidate for a
force The force is US_1943

IF Identify and test a strategic COG candidate
corresponding to a member of a force The force
is ?O1
Rewrite as
explanation ?O1 has_as_member ?O2
Most general generalization
Plausible Upper Bound Condition
?O1 is multi_member_force has_as_member ?O2
?O2 is force
Condition ?O1 is Allied_Forces_1943
has_as_member ?O2 ?O2 is US_1943
Plausible Lower Bound Condition
?O1 is equal_partners_multi_state_alliance has_as
_member ?O2 ?O2 is single_state_force
Most specific generalization
has_as_member domain multi_member_force
range force
THEN Identify and test a strategic COG candidate
for a force The force is ?O2
48
Notice that the explanation is first re-written
as a task condition, and then two generalizations
of this condition are created a most
conservative one (the plausible lower bound
condition) and a most aggressive one (the
plausible upper bound condition). The plausible
lower bound is the minimal generalization of the
condition from the left hand side of the
slide. Similarly, the most general generalization
is the plausible upper bound.
49
Analogical reasoning
50
The agent uses analogical reasoning to generalize
the example and its explanation into a plausible
version space rule. This slide provides a
justification for the generalization procedure
used by the agent. Let us consider that the
expert has provided to the agent the task
reduction example from the bottom left of this
slide. This reduction is correct because
Allied_Forces_1943 has_as_member US_1943. Now
let us consider the European_Axis_1943 which has
as member Germany_1943. Using the same logic as
above, one can create the task reduction example
from the bottom right of the slide. This is a
type of analogical reasoning that the agent
performs. The explanation from the left hand side
of this slide explains the task reduction from
the left hand side. This explanation is similar
with the explanation from the right hand side of
this slide (they have the same structure, being
both less general than the analogy criterion from
the top of this slide). Therefore one could
expect that this explanation from the right hand
side of the slide would explain an example that
would be similar with the initial example. This
example is the one from the right hand side of
the slide. To summarize The expert provided the
example from the left hand side of this slide and
helped the agent to find its explanation. Using
analogical reasoning the agent can perform by
itself the reasoning from the bottom right hand
side of the slide.
51
Generalization by analogy
52
Notice that in the previous illustration we could
have used any other forces ?O1 and ?O2 instead of
European_Axis_1943 and Germany_1943. As long as
?O1 has as member ?O2, the agent would
hypothesize that in order to identify and test a
strategic COG for ?O1 one could identify and test
a strategic COG for ?O2. The agent uses various
constraints from the knowledge base to restrict
the values that the variables ?O1 and ?O2 could
take. For instance, ?O1 should have the feature
has_as_member and the domain of this feature
(i.e. the set of objects that may have this
feature) is multi_member_force. Therefore ?O1
should be a multi_member_force. Also, ?O1 is the
value of the task feature The force is the
range of which is force. Therefore ?O1 should
also be a force. From these two restrictions, we
conclude that ?O1 should be a multi_member_force.
Using this kind of reasoning, the agent
generalizes the example from the left hand side
of this slide to the expression from the right
hand side of this slide.
53
Characterization of the learned rule
Plausible Upper Bound Condition
Universe of Instances
Eh
Plausible Lower Bound Condition
54
As discussed previously, the plausible upper
bound condition of the learned rule is an analogy
criterion that allows the agent to solve problems
by analogy with the example from which the rule
was learned. Because analogy is only a plausible
reasoning process, some of the examples covered
by the rule may be wrong. The plausible upper
bound of the rule is therefore only an
approximation of a hypothetical exact condition
that will cover only positive examples of the
rule. That is why it is called plausible upper
bound. The plausible lower bound condition of
the rule covers the input example that is known
to be correct. However, the bound is a minimal
generalization performed in the context of an
incomplete ontology (generalization hierarchy).
Therefore it is also a plausible bound. The
previous slide shows the most likely relation
between the plausible lower bound, the plausible
upper bound and the hypothetical exact condition
of the rule. Notice that there are instances of
the plausible upper bound that are not instances
of the hypothetical exact condition of the rule.
This means that the learned rule could also
generate wrong solutions to some problems, as
already mentioned. Also, there are instances of
the hypothetical exact condition that are not
instances of the plausible upper bound. This
means that the plausible upper bound does not
cover all the cases in which the solution
provided by the rule would be correct.
Similarly, there may be cases that are covered
by the plausible lower bound, without being
covered by the hypothetical exact condition. All
these situations are a consequence of the fact
that the explanation of the initial example might
be incomplete, and that the representation
language for learning (which is based on the
object ontology) might also be incomplete. These
results are consistent with what one would expect
from an agent performing analogical reasoning.
55
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
56
Input example

IF the task to accomplish is
Identify the strategic COG candidates with
respect to the industrial civilization of US_1943
Who or what is a strategicallycritical
industrial civilizationelement in US_1943?
THEN
Industrial_capacity_of_US_1943
industrial_capacity_of_US_1943 is a strategic COG
candidate for US_1943
57
Learned PVS rule

IF Identify the strategic COG candidates with
respect to the industrial civilization of a
force The force is ?O1

IF Identify the strategic COG candidates with
respect to the industrial civilization of ?O1
QuestionWho or what is a strategically critical
industrialcivilization element in ?O1
? Answer?O2
explanation ?O1 has_as_industrial_factor
?O2 ?O2 is_a_major_generator_of ?O3
Plausible Upper Bound Condition?O1 IS Force has_
as_industrial_factor ?O2 ?O2 IS Industrial_fa
ctor is_a_major_generator_of
?O3 ?O3 IS Product
THEN ?O2 is a strategic COG candidate for ?O1
INFORMAL STRUCTURE OF THE RULE
Plausible Lower Bound Condition ?O1 IS US_1943 ha
s_as_industrial_factor ?O2 ?O2 IS Industrial_c
apacity_of_US_1943 is_a_major_generator_of
?O3 ?O3 IS War_materiel_and_transports_of_US_1943
THEN A strategic COG relevant factor is strategic
COG candidate for a force The force is ?O1 The
strategic COG relevant factor is ?O2
FORMAL STRUCTURE OF THE RULE
58
Explanation of the example
Natural Language
Logic
Identify the strategic COG candidates with
respect to the industrial civilization of a
force The force is US_1943
Identify the strategic COG candidates with
respect to the industrial civilization of US_1943
explanation US_1943 has_as_industrial_factor

Industrial_capacity_of_US_1943 Industrial_capacit
y_of_US_1943 is_a_major_generator_of
War_materiel_and_transports_of_US_19
43
Who or what is a strategicallycritical
industrial civilizationelement in US_1943?
Industrial_capacity_of_US_1943
A strategic COG relevant factor is strategic COG
candidate for a force The force is US_1943 The
strategic COG relevant factor is
Industrial_capacity_of_US_1943
industrial_capacity_of_US_1943 is a strategic COG
candidate for US_1943
59
The expert has defined the example during the
modeling process. The left hand side of this
slide shows how the expert reasons using task
reduction. All the expressions are in natural
language. The agent, however, does not reason
with natural language. It needs formal logical
expressions, like the ones shown in the right
hand side of this slide. During the task
formalization process, the expert and the agent
have collaborated to formalize the tasks. Now the
expert and the agent have to collaborate to also
formalize the question and the answer. This
formalization is the explanation from the right
hand side of this slide. It consists of various
relations between certain elements from the
agent's ontology US_1943 has_as_industrial_fact
or Industrial_capacity_of_US_1943Industrial_cap
acity_of_US_1943 is_a_major_generator_of
War_materiel_and_transports_of_US_1943  They
state, in Disciples language, that US_1943 has
as industrial factor its industrial capacity,
which is a major generator of war materiel and
transports. An expert can understand these formal
expressions because they actually correspond to
his own explanations. However, he cannot be
expected to be able to define them because he is
not a knowledge engineer. For one thing, he would
need to use the formal language of the agent. But
this would not be enough. He would also need to
know the names of the potentially many thousands
of concepts and features from the agents
ontology (such as is_a_major_generator_of).
While defining the formal explanation of this
task reduction step is beyond the individual
capabilities of the expert and the agent, it is
not beyond their joint capabilities. Finding
these explanation pieces is a mixed-initiative
process involving the expert and the agent. In
essence, the agent will use analogical reasoning
and help from the expert to identify and propose
a set of plausible explanation pieces from which
the expert will have to select the correct
ones. Once the expert is satisfied with the
identified explanation pieces, the agent will
generate a general rule.
60
What is the form of the explanation?
The explanation is a sequence of object
relationships that correspond to fragments of the
object ontology.
explanation US_1943 has_as_industrial_factor
Industrial_capacity_of_US_1943 Industrial_capaci
ty_of_US_1943 is_a_major_generator_of

War_materiel_and_transports_of_US_1943
War_materiel_and_transports_of_US_1943
is_a_major_generator_of
Industrial_capacity_of_US_1943
has_as_industrial_factor
US_1943
61
General heuristics for explanation generation
Look for the relationships between the objects
from the question and the answer
Look for the relationships between an object from
the IF task and an object from the question or
the answer
62
User hint selecting an object from the example

IF
Identify the strategic COG candidates with
respect to the industrial civilization of US_1943
Who or what is a strategicallycritical
industrial civilizationelement in US_1943?
THEN
industrial_capacity_of_US_1943 is a strategic COG
candidate for US_1943
Industrial_capacity_of_US_1943
The expert selects an object from the
example. The agent generates a list of plausible
explanations containing that object. The expert
selects the correct explanation(s).
Hint
Industrial_capacity_of_US_1943
Industrial_capacity_of_US_1943 IS
Industrial_capacity_of_US_1943 ?
has_as_industrial_factor --- US_1943
Industrial_capacity_of_US_1943 --
is_a_major_generator_of ?
War_materiel_and_transports_of_US_1943
63
The agent may need help to propose additional
necessary explanations. For instance, the
previously found explanation US_1943 --
has_as_industrial_factor --gt Industrial_capacity_
of_US_1943 does not express the fact that
Industrial_capacity_of_US_1943 is a strategically
critical element of US_1943. To help the agent to
generate an formal explanation that expresses
this fact, the expert may provide two types of
hint. One hint is to select one object from the
example. In this case the object would be
Industrial_capacity_of_US_1943. The agent will
then generate plausible explanations containing
this object, as illustrated in this slide. The
expert will then need to select the correct
explanation, as shown in this slide. Another type
of hint is to select two objects from the
example. In this case the agent will show the
relationships between these two objects. For
instance, if the agent would have not generated
the explanation US_1943 -- has_as_industrial_fa
ctor --gt Industrial_capacity_of_US_1943, the
expert could have simply selected these objects.
Then the agent would have generated the above
explanation.
64
Hint refinement

IF
Is Anglo_allies_1943 a single member force or a
multi-member force?
Identify the strategic COG candidatesfor
Anglo_allies_1943
THEN
Identify the strategic COG candidatesfor the
Anglo_allies_1943 which is a multi-member force
Anglo_allies_1943 is a multi-member force
The expert provides a hint Anglo_allies_1943
The agent generates a list of plausible
explanations containing that object. The expert
selects one expression that ends in and
clicks on EXPAND
EXPAND
Anglo_allies_1943 IS
Anglo_allies_1943 --- component_state ? US_1943
The agent generates the list of possible
expansions of the expression and the experts
select the explanation
Anglo_allies_1943 IS Strategic_COG_relevant_fact
or
Anglo_allies_1943 IS Force
Anglo_allies_1943 IS Military_factor
Anglo_allies_1943 IS Multi_member_force
Anglo_allies_1943 IS Multi_state_force
Anglo_allies_1943 IS Multi_state_alliance
Anglo_allies_1943 IS Equal_partners_multi_state_
alliance
65
In this example, according to the question and
the answer from the upper left of the slide, the
explanation should express the fact that
Anglo_allies_1943 is a multi member force. The
expert first gives a hint to the agent by
selecting Anglo_allies_1943. Then the agent
generates several explanations that start with
Anglo_allies_1943. In order to reduce the number
of plausible explanations that the expert has to
analyze, the agent may generate only the
beginning of a plausible explanation, followed by
The expert may select such an expression and
my ask the agent to expand it, as illustrated in
this slide. Indeed, the expert selected
Anglo_allies_1943 IS , because these are the
explanations that express the fact that
Anglo_allies_1943 is of a certain type. Then the
expert requests the agent to expand this
explanation and the agent lists all the types of
Anglo_allies_1943. After that the expert selects
the correct explanation. The next slide also
illustrates the hint refinement process, this
time for a different type of explanation. It is
important to stress that the hint may also be an
object from a previously selected explanation,
not necessarily from the initial example.
66
Hint refinement (another example)
Hint
Britain_1943
Generate
...
Civilization_of_Britain_1943
has_as_civilization
...
component_state
Britain_1943
Anglo_allies_1943
has_as_governing_body
...
...
Governing_body_of_Britain_1943
...
Select and expand
PM_Churchill
has_as_political_leader
has_as_governing_body
has_as_dominant_psychosocial_factor
Governing_body_of_Britain_1943
...
Britain_1943
the will of the people
has_as_ruling_political_party
...
Select explanation
Conservative_party
Britain_1943 -- has_as_governing_body ?
governing_body_of_Britain_1943 --
-- has_as_dominant_psychosocial_factor ? will
of the people
67
Learned plausible version space rule

IF Identify the strategic COG candidates with
respect to the industrial civilization of a
force The force is ?O1

IF Identify the strategic COG candidates with
respect to the industrial civilization of ?O1
QuestionWho or what is a strategically critical
industrialcivilization element in ?O1
? Answer?O2
explanation ?O1 has_as_industrial_factor
?O2 ?O2 is_a_major_generator_of ?O3
Plausible Upper Bound Condition?O1 IS Force has_
as_industrial_factor ?O2 ?O2 IS Industrial_fa
ctor is_a_major_generator_of
?O3 ?O3 IS Product
THEN ?O2 is a strategic COG candidate for ?O1
INFORMAL STRUCTURE OF THE RULE
Plausible Lower Bound Condition ?O1 IS US_1943 ha
s_as_industrial_factor ?O2 ?O2 IS Industrial_c
apacity_of_US_1943 is_a_major_generator_of
?O3 ?O3 IS War_materiel_and_transports_of_US_1943
THEN A strategic COG relevant factor is strategic
COG candidate for a force The force is ?O1 The
strategic COG relevant factor is ?O2
FORMAL STRUCTURE OF THE RULE
68
Analogical reasoning heuristic
1. Look for a rule Rk that reduce the current
task T1. 2. Extract the explanations Eg from the
rule Rk. 3. Look for explanations of the current
task reduction that are similar with Eg.
Example to be explained
Previously learned rule Rk
IF the task to accomplish is T1g Explanation Eg
PUB condition PLB condition THEN accomplish
T11gT1ng
IF the task to accomplish is T1 THEN accomplish
T1a,T1d
Look for explanations that are similar with Eg
69
Justification of the heuristic
This heuristic is based on the observation that
the explanations of the alternative reductions of
a task tend to have similar structures. The same
factors are considered, but the relationships
between them are different.
T1
Question
Q
Ab
Aa
Ae
Answers
T1a
T1b
T1e
Eb
Ee
Explanations
Ea
70
Another analogical reasoning heuristic
1. Look for a rule Rk that reduce a similar task
to similar subtasks. 2. Extract the explanations
Eg from the rule Rk. 3. Look for explanations of
the current task reduction that are similar with
Eg.
71
Justification of the heuristic
This heuristic is based on the observation that
similar problem solving episodes tend to have
similar explanations

72
Yet another analogical reasoning heuristic
1. Look for a rule Rk that reduces a task that is
similar to the current task even if the subtasks
are not similar. 2. Extract the explanations Eg
from the rule Rk. 3. Look for explanations of the
current task reduction that are similar with Eg.
The plausible explanations found by the agent can
be ordered by their plausibility (based on the
heuristics used).
73
No explanation necessary

IF
What type of strategic COGcandidates should
Iconsider for Britain_1943?
Identify the strategic COG candidatesfor
Britain_1943, a member ofAnglo_allies_1943
THEN
I consider strategic COG candidates with
respect to the governing elementof Britain_1943
Identify the strategic COG candidateswith
respect to the governing element of
Britain_1943, a member ofAnglo_allies_1943
Sometimes no formal explanation is necessary. In
this example, for instance, each time I want to
identify the strategic COG candidate for a state,
such as Britain, I would like to also consider
the candidates with respect the governing element
of this state. We need to invoke Rule Learning,
but then quit it without selecting any
explanation. The agent will generalize this
example to a rule.
74
Overview
What is Machine Learning
Generalization and Specialization Operations
Basic Learning Strategies
The Rule Learning Problem in Disciple
The Multistrategy Rule Learning Method
Strategies for Explanation Generation
Demo Agent Teaching and Rule Learning
Recommended Reading
75
DISCIPLE-RKF
Disciple-RKF/COG Agent Teaching and Rule
Learning
76
We will now demonstrate how Disciple learns
general tasks and rules from the experts
reasoning.
77
First the expert and Disciple have to formalize
the English statements of the tasks.
This is done in the Formalization mode.
78
In the formalization mode the tool shows
The modelingin English
The formalizedtasks
79
When the expert clicks on Formalize Disciple
will propose a formalization of the task
80
Formalization of the task proposed by Disciple
The expert may accept it or he may edit it
81
After the tasks from the current task reduction
step are formalized the expert may explain this
example to Disciple, which will learn a rule from
it
82
The Rule Learning tool helps the expert to
explain the example to Disciple
Disciple uses analogical reasoning and other
heuristics to propose plausible explanations
pieces that justify the task reduction step.
The tool shows the English form of the example
The expert selects those explanation pieces that
correspond to the meaning of the question-answer
pair from the task reduction example and clicks
on Accept
83
The expert may direct Disciple to generate
explanation pieces related to certain objects
from the example
84
The expert may direct Disciple to generate
explanation pieces related to certain objects
from the example
85
(No Transcript)
86
When the expert is satisfied with the identified
explanation pieces he can click on End
learning. Disciple will then create a general
rule corresponding to this example and its
explanation.
87
This is the general task reduction rule learned
by Disciple.
Notice that it has a plausible upper bound
condition and a plausible lower bound condition.
During rule refinement the two conditions will
converge toward one another, ultimately leading
to a rule with a single condition.
88
This is the general task reduction rule learned
by Disciple.
In addition to the formal structure of the rule,
which is used in problem solving and learning,
Disciple maintains also an informal structure of
the rule.
The informal structure is used in the
communication with the user.
89
After learning a rule from the current task
reduction step Disciple returns to the
formalization mode.
90
Following the same procedure, Disciple will learn
another rule from the this task reduction step
The expert selects the next task reduction step
by clicking on the corresponding question-answer
pair
91
(No Transcript)
92
If Disciple determines that additional
explanations are needed it will asks the expert
to provide them
When the expert clicks on End learning Disciple
analyzes the accepted explanations to determine
whether additional explanations are needed.
93
(No Transcript)
94
After learning a new rule from the current task
reduction step Disciple returns to the
formalization mode.
95
Following the same procedure, Disciple learned
another rule from the last task reduction step
96
Recommended reading
Tecuci G., Building Intelligent Agents A Theory,
Methodology, Tool and Case Studies, Academic
Press, 1998. Tecuci G., Boicu M., Marcu D.,
Stanescu B., Boicu C. and Comello J., Training
and Using Disciple Agents A Case Study in the
Military Center of Gravity Analysis Domain, in AI
Magazine, AAAI Press, Menlo Park, California,
2002. http//lalab.gmu.edu/publications/default.ht
m Tecuci G., Boicu M., Bowman M., and Marcu M.,
with a commentary by Burke M. An Innovative
Application from the DARPA Knowledge Bases
Programs Rapid Development of a High Performance
Knowledge Base for Course of Action Critiquing,
in AI Magazine, 22, 2, 2001, pp. 43-61. AAAI
Press, Menlo Park, California, 2001.
http//lalab.gmu.edu/publications/default.htm
Describes the course of action domain.