Test Case Design by John Howard

1
Test Case Designby John Howard

• Test Case Design Using Classification Trees
• by Matthias Grochtmann
• Software Testing Analysis Review,
• Proceedings of STAR 1994, Washington D.C.
• Test Case Design by Means of the CTE XL
• by E. Lehmann and J. Wegener
• Proceedings of EuroSTAR 2000, Copenhagen

2

• The authors all worked for the Research and
  Technology division of Daimler-Chrysler in
  Germany during the last decade.
• Mr. Grochtmann is co-creator of the
  Classification Tree Method (CTM) which is the
  central research subject of both papers.
• Both papers present practical support for the
  systematic design of black-box test cases as
  practiced in the industrial world 1.
• Both papers address the development of a
  third-party 2 design tool called the
  Classification Tree Editor (CTE) for applying the
  CTM.
• I could not find eXtreme Progamming 3, 4
  research on test-driven development (TDD).
  Applying the CTM XL with TDD is new.

2
3

• Simply stated, it's the most important
  prerequisite for the testing of software because
  it determines the quality of a test.
  Exhaustively testing a program with all its
  possible input domain is often too difficult or
  impossible due to the huge number of test cases
  involved.
• Thus non-redundant subsets of the input domain
  (that is, test suites) must be selected based
  upon some criteria to construct relevant test
  cases so that the tester can determine the
  quality of the verification process. Selection of
  test cases determines the kind and scope of a
  test.
• The set of test suites should be as
  comprehensive as possible to be effective for
  detecting possible faults in the software.
• The set of test suites should be cost-effective
  to develop and reuse.

3
4

• Black box (functional specification-based)
• Examples
• Category-Partition Method 1
• Classification Tree Method 2
• White box (functional implementation-based)
• Examples
• Debugger
• statement testing, branch testing, path testing,
  condition testing
• Shadow box (multi-level approach to ignore less
  relevant test cases)
• Example
• White on Black 3

4
5
The main area of application of the CTM is
testing the functional behavior of systems that
compute discrete input data.
5
6
The fundamental concept of the CTM is to divide
the input domain of a program into disjoint sets
of classes, and form test cases by selecting from
these classes according to certain criteria 2.
6
7

• Selecting test objects
• Designing a classification tree
• Combining classes to form test cases

7
8
Example The test object is a Computer Vision
System which should determine the size of
different objects. The possible inputs are
wooden building blocks. Appropriate aspects
would be the size, color, and shape of a
block. Classification based on color leads to a
partition of the input domain into red, green,
and blue blocks. Classification based on shape
leads to a partition into circular, triangular,
and square blocks. Another aspect is introduced
for the triangle class its shape can be
equilateral, isosceles, or scalene.
8
9
Classifications are defined as aspects of viewing
the input domain of the program to be
tested. Classes are defined as disjoint subsets
of values of classifications. Identify all the
classifications and their classes from the
specifications (i.e. use a black box approach).
9
10
Construct the classification tree (CT) with the
classifications and classes. Thus a CT is
organized as a hierarchy chart or tree, where the
top circle (i.e. general root node of a tree
structure) represents the input domain of the
program. Classifications branch downward from
the root until the classes are reached at the
leaves (i.e. terminal nodes) of the tree.
10
11
The CTM uses a combination table to logically
connect a desired test case to the classes it
uses from the classification tree. Thus a
combination table is organized as a spreadsheet
of boolean values where each row represents a
test case and each column a possible class. A
mark for true is placed in a column to connect
that class with the test case.
11
12
The CTE XL allows the tester to specify logical
dependency rules between the classes of the
CT. The dependency rules are specified using
propositional logic. According to Cantone 7, a
propositional sentence is CONSISTENT if it
evaluates to true in at least one interpretation
of its variables TAUTOLOGICAL if it evaluates to
true in every interpretation of its
variables ABSURD if not CONSISTENT.
12
13

• The CTE XL offers five different mechanisms to
  specify combination rules for the generation of
  test cases.
• Using the name of a class in a combination rule
  indicates the class should be selected in at
  least one test case. If the class is further
  divided into subclasses then at least all
  possible combinations of the subclasses should be
  generated.

13
14
2. Using the name of a classification in a
combination rule indicates that all classes of
that classification should be selected in at
least one test case. If the classes of the
classification are further divided into
subclasses then all possible combinations of the
subclasses should be generated at least once.
14
15

• A B all patterns to be generated for A should
  be combined with all patterns to be generated for
  B.
• A B all patterns to be generated for A and all
  patterns to be generated for B will be produced
  and combined in test cases in an unsystematic
  way. (This operator guarantees generation of all
  specified patterns and tries to minimize the
  number of test cases generated.)
• Twowise(A1, A2, ) threewise(A1, A2, )
  generates a pair-wise or three-wise combination
  of all patterns specified by Ai. (This operator
  tries to minimize the number of test cases
  generated.)

15
16

• Advantages

Applying CTM with eXtended Logics supports
specifying logical dependencies between classes
by using propositional logic. Defining
combination rules for classifications or classes
enables the tester to specify their degree of
relevancy for the test to be generated. Automatic
test case generation is possible in CTE XL.
16
17

• Disadvantages

According to Leung and Wong 2 CTs have to be
constructed with reference to the relations
between classes. The classifications have only a
small role in test case generation. (CTE XL
solves this objection via logical
dependencies.) CTs require manual decisions to
filter out invalid test cases. (This can be
automated by CTE XL.) CTs have no formal
semantics which may cause ambiguity when
different people perform analysis.
17
18
Restructuring classification trees 4, 5 for
optimization of relevant paths. Using class
vectors and class graphs instead of
classification trees 2, 6. Automating
evolutionary testing 1 by using various logics
for structural testing and temporal system
behavior testing (multi-level approaches). Unit
test frameworks4 such as Junit for Java
language.
18
19
Near-term, probably integrating unit test
frameworks with evolutionary testing to support
test-driven development for mobile applications
needed by agile computing.
19
20
1) T.J. Ostrand and M.J. Balcer, The
category-partition method for specifying and
generating functional tests, Communications of
the ACM, vol. 31, no. 6, pp. 676-686, June
1988. 2) K.R.P.H. Leung, and W. Wong, Deriving
test cases using class vectors, IEEE
1530-1362/00 , pp. 146-153, 2000. 3) T.Y. Chen,
P.L. Poon, S.F. Tang, and Y.T. Yu, White on
Black a white-box-oriented approach for
selecting black-box-generated test cases, IEEE
0-7695-0825-1/00 , pp. 275-284, 2000.
20
21

• 4) T.Y. Chen and P.L. Poon, Improving the
  quality of classification trees via
  restructuring, IEEE 0-8186-7638-8/96, pp. 83-92,
  1996.
• 5) T.Y. Chen, P.L. Poon, and T.H. Tse,
  Classification-tree restructuring methodologies
  a new perspective, IEEE Proceedings-Software,
  vol. 149, no. 2, pp. 65-74, April 2002.
• K.R.P.H. Leung, and W. Wong, Towards a more
  efficient way of generating test cases class
  graphs, IEEE 0-7695-0825-1/00, pp. 285-293,
  2000.
• D. Cantone, E. Omodeo, A. Policriti, Set theory
  for computing, Springer, New York, 2001.

21
22
Resources
1 www.systematic-testing.com 2 Razorcat
Development GmbH of CTE XL (eXtended Logics). 3
www.extremeprogramming.org 4 www.junit.org
22