ICKEP International Competition for Knowledge Engineering in Planning A PROPOSAL

1
ICKEPInternational Competition for Knowledge
Engineering in Planning -A PROPOSAL

• Lee McCluskey
• KE TCU

2
Contents

• Aims / Benefits
• Current IPC
• Problems with an ICKEP
• A small start

3
Current ICP - Benefits

• The ICP has brought benefits to the community -
• - focussed some researchers on technology
  innovation
• - led to a rapid development of techniques
• - delivered a de facto standard for communicating
  the dynamics of domain models
• - helped in the validation of planning algorithms
• and hence led to the sharing of benchmark domain
  models, tasks and planning tools.

4
Current ICP - Problems

• However, the ICP is controversial - it encourages
  rapid development - but in a narrow area
• The ICP assumes that
• the input to a planning engine is correct and
  complete
• the input is in PDDL which was designed with the
  criterion of dynamics and nothing else. It was
  designed to reflect current languages and their
  underlying assumptions. It was NOT designed with
  a model building method in mind OR with many
  pragmatic feature which make building easier -
  it is a machine code rather than a language for
  human use!
• Plus lots of others I wont mention

5
Narrow views of Planning?
Complete, correct, formal, Precondition-effect,
Literal strips-based Model of dynamics
Acquisition, Debugging, Compiling, Configuration,
Modelling
Plan Generator
Execution, Scheduling
Something Else?
6
Aim of ICKEP

• The aim of a KE Competition will be to promote
  the knowledge-based aspects of planning (to
  include knowledge acquisition, knowledge
  modelling and domain validation) by evaluating KE
  tools within a competitive forum.

7
Possible Benefits

• it might address the main problem with the
  current competition - that, although the
  competition encourages rapid development, it
  tends to focus work narrowly.
• it might encourage the development and sharing
  of stand alone tools to help in the whole process
  of AI planning including domain modelling,
  heuristic acquisition, planner-domain matching
  and so forth.
• it might lead to some form of communication
  medium for knowledged-based domain models

8
Form of Current IPC

• for the IPC
• Competitors prepare before the event a planner
  which can input PDDL and gives out solutions in a
  prescribed format.
• Competitors are given at the event domain
  models, tasks, in PDDL
• During the event the planners are executed with
  the supplied domain models and tasks.
• Evaluation after the event tools are used to
  rate the planners on speed, coverage, and
  solution quality.

9
Form of ICKEP??

• But tools and methods to support knowledge
  acquisition and modelling
• do not have standard forms of input. They may
  acquire knowledge from domain experts or help
  planning researchers debug domain models.
• Cannot be easily evaluated by their outputs -
  what is the advantage of one domain model over
  another?
• Are heterogneous - there are several types of
  tools performing differing functions

10
PROPOSAL Start simple

• PROPOSAL Start off with initial competition
  which has a very simple format, along the
  following lines
• Competitors prepare before the event two types
  of tool
• (a) one that debugs domain models
• (b) one that extracts heuristics from domain
  models.
• Both tools will input a certain version of PDDL
• (a) will output a set of flaws in the domain
  model, and
• (b) will output a set of heuristics in a standard
  format, that can be used with a standard planner,
  to help solve plan generation problems.

11
PROPOSAL Start simple

• Competitors are given at the event flawed domain
  models for (a), domain models, a planner and
  tasks for (b).
• During the event the tools are executed with the
  supplied domain models and tasks.
• Evaluation after the event tools are used to
  rate the competitors' tools for (a) percentage
  and type of flaws uncovered (b) quality of
  heuristics acquired as judged by performance
  improvement on a standard planner.

12
TO DO

• Is there the will in the community to follow this
  through? After emailing 12 (?) top US researchers
  I got ONE reply
• If we go ahead
• Agree on the scope of an initial competition.
  Make up some outline rules.
• Target a conference for an initial competition.
• Get together an organising committee who can also
  compete!

13
COMPETITION

• There is a will in the community to follow this
  through.
• We will -
• Agree on the scope of an initial competition.
  Make up some outline rules.
• Target a conference for an initial competition.
• Get together an organising committee who can also
  compete!

14
Initial Working Group

• Prof Ruth Aylet, University of Salford, UK
• Dr Ronan Bartak, Charles University, Prague,
  Czech Republic
• Prof Daniel Borrajo, University Carlos III de
  Madrid, Spain
• Prof Susanne Biundo, University of Ulm, Germany
• Dr Christophe Doniat, Université Technologique de
  Troyes, France
• Dr Peter Jarvis, SRI International, USA
• Prof Lee McCluskey, University of Huddersfield, UK