Validating RuleBased Systems A Complete Methodology

1
A Step out of the Ivory Tower Experiences with
Adapting a Test Case Generation Idea to Business
Rules
Rainer Knauf Technical University of
Ilmenau School of Computer Science and
Automation Ilmenau, Germany
Silvie Spreeuwenberg Rik Gerrits LibRT
B.V. Amsterdam, The Netherlands
Martin Jendreck reinisch AG Karlsruhe, Germany
2
Content

• Motivation
• A Closer Look at Business Rules
• The Representation Aspect
• The Processing Aspect
• The Application Scenario to Perform Evaluation
• The Entire Framework
• The Part Implemented for AionBRE
• The Architecture of the Complete Evaluation
  Module
• The Technology to Generate QuEST
• Particular Problems of this Technology Transfer
• Dependencies of Outputs
• Critical Input Values and Appropriate Ranges
• Computing Potential test case values
• Composing Test Data
• Minimizing Test data
• Summary and Outlook

3

• What are Business Rules?
• Knowledge representation in commercial
  environments
• What are they used for?
• Describing business functions in descriptive
  manner
• Supporting decision making in commercial
  environments
• What are the application fields?
• The upcoming European Business Rule Conference
  (June 2004, Amsterdam) covers contributions about
  applications in
• E-government that aims at improving the
  government to citizen (G2C) and government to
  business (G2B) interaction (NL)
• Risk analyses of financial institutions (D)
• Pharmacy (NL)
• Personnel regulations for central government
  personnel (B)
• Security management in banking (CH)
• Monitoring economic and technical performance of
  the Dutch agriculture (NL)
• Construction of the new Metro Railway in
  Amsterdam (NL)
• A Business Rules Based Portal for Healthy Living
  (NL)
• Marketing in grocery industry (A, I)
• Automated order management for the retail
  business at Swisscom (CH)
• Insurance and healthcare case studies (UK)

4

• What kinds of Business Rules are distinguished?
• Rules that describe constraints ( knowledge
  representation rules)
• Rules that describe derivations ( knowledge
  processing rules)
• What are they formally ?
• Basically, HORN clauses with attribute-value
  pairs as their atomic components
• Concrete BR languages provide much more
  expressiveness that is far beyond a logic or any
  other formal calculus, such as
• function calls
• equations and in-equations with arithmetic terms,
  
• calls of methods,
• Thats the challenge!

5
Why and how to evaluate such rule bases?

• The usefulness of this approach corresponds to
  the correctness of its incorporated rule base.
• The need of formal verification is impressively
  shown by the selling success of LibRTs product
  VALENS, a verification tool for BR that reveals
  several kinds of
• Contradictions
• Redundancies
• Completeness (of value ranges, e.g.)
• Unused conclusions
• Loops
• Self-contradicting rule premises
• Unreachable conclusions
• The need of validation is indicated by LibRTs
  request to complete their evaluation tool by a
  validation method like our one

6
2 A Closer look at BR

• According to their purpose (control of business
  courses), BR appear in different forms (that are
  not necessarily rules from our viewpoint)
• definition of business terminologies
• by glossaries, entities of an ERM, description
  logic of an ontology
• facts express relations between terminologies
• by a graph or natural language statements
• instructions and restrictions control the access
  to business data
• by real rules IF THEN
• derivations define the way to infer new knowledge
• by computation rules and inference rules
• This large variety of formal, semi-formal and
  informal knowledge representations is not an
  appropriate subject for AI technologies
• formal sub-languages have been developed, for
  example CleverPath Aion Business Rules Expert (
  AionBRE )

7
AionBRE

• Knowledge Representation
• proposition calculus HORN clauses with an
  extended expressiveness
• atomic formulas are not limited to logic
  statements, but
• as premises
• attribute-value pairs
• equations and in-equations with attributes,
  values, arithmetic terms on both sides
• function calls
• calls of methods
• as conclusions
• attribute-value assignments
• calls of methods
• Knowledge Processing
• Takes place at two levels
• by choosing built-in inference methods like
• forward or backward chaining
• inheritance within a class hierarchy
• truth maintenance assignments
• by user defined inference rules like
• pattern matching rules

8

• 3 Validation Scenario
• 3.1 The Entire Framework

TURING Test
refined (better) knowledge base
validators
criteria
knowledge base
rules
test data
test case generation
test case ex-perimentation
solutions
outputs
protocol
system refinement
validity assessment
evaluation
report
validity statements
system user knowledge engineer
9
3.2 The Part Implemented for AionBRE
test case generation
expert(s)
QuESTer
criteria
test case experi-men-tation
rule base
QuEST
initial test case generation
ReST
knowledge base
reduction
10
3.3 The Architecture of the Complete Evaluation
Module

• number of computed potential test cases
• number of computed test data that infer an output
• QuEST and its cardinality
• hints to rule base anomalies

Rule base XML
VALENS
rules as rule base XML schema

• rules
• all final conclusions

• test data
• their outputs

QuESTer
11
3.4 The Technology to generate QuEST
rule set R
Computation of dependency sets
Ri R ? oi Si S ? oi
Dependencies
Computation of critical values scanning
distances
Skcrit ? values( sk ) sk subrange( sk )
Critical values Scanning distances
Computation of potential test data values
Vij f( oi , sj ) ? values( sj )
Test data values of sj for oi
Composing the set of potential test data
n
n m P ? ( Vi1 ?
Vi2 ? ... ? Vim ) ? ? Vij i1

i1
j1
Composed test data set for all oi
Minimizing the set of potential test data
oi - related partition of P minimized Pi -s
Composed
QuEST
Pi f( oi ) ? P Pi ? Pi
n QuEST ? ti , oi ti ? Pi
i1
test case set QuEST
12

• Particular Problems in Implementing this Idea
  (Problem / Solution)
• 4.1 Dependencies of Outputs

• Misunderstanding 1
• What is an output?
• A single expression of the proposition calculus
  or
• a written attribute that can be assigned to a
  particular value?
• The BR community prefers the 2nd definition we
  assumed the 1th one.
• In a first setting, we adopted definition (2).
• Since the number of used attribute values is
  usually a tiny subset of the complete rage,
  definition (2) increases the resulting number of
  test cases dramatically
• An example with just two short rules revealed 15
  test cases underlying definition 2 versus 7
  cases by underlying definition 1.
• Thus, later (when implementing the minimizing
  procedure) we turned back to our understanding of
  an output and considered an attribute-value pair
  as an output, that follows our intention of being
  a single statement.

13

• Misunderstanding 2
• What is a rule?
• An IF-THEN construction with logic elements
  that can be true or false or
• An IF-THEN construction that also includes
  function and/or method calls in the IF-part as
  well as method calls in the THEN-part?
• The BR community uses the 2nd definition we
  assumed the 1st one.
• To adapt our technology to practical needs (as
  claimed by the BR community), we adopted
  definition (2) and normalized the rule bases to
  equivalent ones, which follows definition (1)
• A rule that contains function calls in the
  IF-part is recompiled to several rules
  according to the possible values of the function
  call.
• A rule that contains method calls, is not really
  a problem as long as the methods do not have any
  access to the rule set itself. Following the HORN
  logic, these calls could be considered as
  logically true in the IF-part respectively false
  in the THEN-part.
• This raised some restrictions of the technologys
  applicability
• Meta-logic rules, whichs use depend on rules
  or even change rules, have to be forbidden. Thus,
  we excluded method calls doing that.

14

• Misunderstanding 3
• What is the logical structure of a rule?
• A rule is
• a HORN clause like A ? B1 ? ? Bn
• or at least a clause like A1 ? ? Am ? B1 ?
  ? Bn
• A rule can also have conjunctive conclusions,
  i.e.
• a construction like A1 ? ? Am ? B1 ? ? Bn
• The BR community allows the 2nd definition as
  well.
• To adapt our technology to practical need of the
  2nd rule structure, we adopted definition (2) and
  normalized the rule bases to equivalent ones.
  This could be performed by substituting rules of
  this structure by several ones with alternative
  THEN-parts

15
4.2 Critical Values and Scanning Distances

• Here, we compute
• critical input values, which mark a difference
  with the effect on an output and
• scanning distances, which define, how close the
  test cases should be located to these critical
  values.
• Problems solved
• Additional data types needed to be introduced to
  AionBRE
• Enumerable data (of the type string, integer, and
  real) needed to be distinguished in those with /
  without an inherent ordering relation.
• Several type conflicts are managed.
• For example, if two inputs of different types
  (real and integer, e.g.) are compared the
  computed critical values and scanning distances
  need to be rounded up or down to some closest
  valid value.
• The approach was extended to consider input data
  with several disjunctive ranges by adapting the
  computation of scanning values.
• Smallest possible steps needed to be defined
  for integers ( 1) and reals (by considering the
  way AionBRE handles fractions).
• Problem left as a restriction of the
  applicability (yet)
• Outputs, which refer to input values.
• But It should be easy to solve it by some
  normalization.

16
4.3 Computing Test Data Values

• Here, the interesting values Vij for an input si
  for an output oj is computed.
• In case si does not influence oj, some arbitrary
  (normal) value is introduced in Vij.
• This did not raise a problem, but an idea to
  refine the technology
• This arbitrary value is determined in a way that
  supports the minimality of the test case set We
  choose a value si that is a test case value for
  another output ok.

4.4 Composing Complete Test Data could be
implemented to AionBRE without any problem.
17

• 4.5 Minimizing the Set of Test Data
• The solution for misunderstanding 1 in
  section 4.1 has been developed here.
• Furthermore, the implementation of this step led
  to a refinement of the approach
• The approach is based on
• forming partitionings of test data into subsets,
  in which the elements differ from each other in
  just one input value and
• looking for values of this input, which are
  subsumed by other values.
• Since the test data are minimized for each output
  separately, input values that dont influence
  this output dont have to be considered while
  forming partitionings.
• This increased the sets and thus, the chances for
  such subsumptions, significantly.
• If there are several chances to define test data
  that subsumes other ones, some optimization has
  been introduced
• The test data, which can serve as test data for
  as many as possible other outputs as well, is
  preferred.

18

• Short Summary
• Implementing theoretical Ideas into commercial
  environments reveals problems far away from the
  imagination of developers working in an
  scientific environment an Ivory Tower.
• Fist of all, a common terminology needs to be
  agreed.
• The main challenge was the trade-off between
  logic approaches and the desired involvement of
  software technologies that can not be handled
  logically.
• The performed technology transfer turned out to
  be a chance to
• identify problems that havent considered before
  and to
• identify refinements and improvements of the
  theoretic ideas developed in the Ivory Tower.