Testing Techniques

1
Testing Techniques
EE491.13
Maj JW Paul
Modified from a series of lectures by Dr Terry
Shepard Diane Kelly
2
In battle, numerical superiority is of no
consequence. Victory will go to the superior
tactician.
GA Custer
3
Review
Exploratory testing Requirements based test
design Design based testing Testing in the
small Testing in the large
outlines in broad terms how to use testing to
assess the extent to which one or more goals for
a product have been met
Strategy
Guides choice of test cases
dimensions of testing impacts choice(s)
implement strategies, provide the data needed to
do the assessment
Techniques
captures the above choices more than test cases,
choosing test purpose how to meet it, start
early, capture all dimensions,
Test Planning and Design
4
Techniques

• Dimensions of Testing

5
Examples of Techniques
6
Classification Scheme
derive test cases by examining what the system is
supposed to do
black box
derive test cases by examining how the program
does what it is supposed to do
white box
awareness of structure, but not details
grey box
complimentary activities
7
Error Guessing
?
?

• develop tests based on possible errors or
  error-prone situations
• similar to category partition method 12
• uses allowable (disallowed) combinations of
  parameters/inputs

8
Syntax testing
?

• Recognizing syntax (patterns, assumed formats,
  etc.) in input data
• expressing it in a standard form
• Backus-Naur Form a_vowel A/E/I/O/U
• use rules to mechanically generate input-data
  validation tests
• look for inputs that dont follow the syntax
• less useful if software designers have already
  recognized and used syntax of inputs

9
Transaction-flow testing
?

• Transaction
• unit of work seen from users viewpoint
• equivalent to path testing, but at the level of
  interfaces among software components rather than
  interfaces among individual statements
• aka scenario based or use-case based testing
• can derive tests from requirements

10
Logic-based testing
?
?

• Addresses the issue of how to combine a variety
  of different conditions in an orderly fashion to
  create multiple test cases
• Decision trees
• Decision tables
• Function lists
• Input/output tables
• Matrices (e.g. Printer compatibility matrix,
  operations versus modes matrix)

11
Transition testing
?
?

• Uses states and state graphs
• Some aspects of a design may be represented as
  relatively small finite state machine diagrams
• Testing these parts of a design can be done by
  undertaking a transition tour of the state
  diagram (state graph)
• not useful for state represented in the form of
  integers, floats, arrays, records, etc.

12
Mutation Testing
?

• generate mutations of program under test
• preferably automatically
• run tests on all mutations computationally long
• look for cases where mutation original
• add tests so they give different results, or
  conclude that the original program is in error.
• Attempt to reduce the amount of data required to
  exercise the program
• concentrate on data that reveals likely faults

13
Flowgraphs and Path testing
?
graphical representation of programs control
structure
Flowgraph
follows the edges of a flowgraph
Path

• every path from entry to exit
• impractical for most programs
• selected paths
• e.g. each loop 0, once, maxcount times
• all paths up to some length

14
Basis path coverage
?

• Select test cases that follow the control flow
  paths (basis paths) defined for the McCabe
  complexity measure
• note McCabe cyclometric complexity is often seen
  as a single measure of program quality
• McCabes conjecture was that such test cases
  would provide adequate coverage
• Myers triangle example follows 15

15
McCabe Complexity 5

• M L N 2P

L the number of links in the flowgraph N
the number of nodes in the flowgraph P
number of disconnected parts of the graph (eg
calling program and subroutine)
16
Myers triangle example 15
L
4
equilateral traingle?
number of links
N
4
number of nodes
Calculate area
Print error
P
1
number of parts
M
2
L-N2P
Return
This implies we need two tests to cover all cases
17
Sample Code
include ltstdio.hgt include ltstdlib.hgt include
ltmath.hgt main(argc, argv) int argc char
argv int sideA int sideB int
sideC double s double Area sideA
atoi(argv1) sideB atoi(argv2) sideC
atoi(argv2) if ( (sideA sideB) (sideA
sideC ) ) s 0.5 (sideA sideB
sideC) Area sqrt (s / (s - sideA) (s -
sideB) (s - sideC) ) printf ( "area g\n",
Area) else puts ("not an equilateral
triangle") return 0
18
Need two test cases (M 2)

• To exercise the TRUE branch
• use inputs
• Side A 2
• Side B 2
• Side C 2
• expected output
• area 1.73205

• To exercise the FALSE branch
• use inputs
• Side A 3
• Side B 4
• Side C 5
• expected output
• not an equilateral triangle

19
Two Errors not Caught

• area of equilateral triangle

____________________________ A ? s (s
- sideA) (s - sideB) (s - sideC)

• code

sideA atoi(argv1) sideB atoi(argv2) side
C atoi(argv2) s 0.5 (sideA sideB
sideC) Area sqrt(s/(s-sideA)(s-sideB)(s-sideC
))
should be argv3
should be multiplication two is the only value
this works for!
20
Moral of the Story...
21
Review

• Transaction-flow testing (bb)
• Syntax testing (bb)
• Logic-based testing (wbbb)
• States, state graphs and Transition testing
  (wbbb)
• Mutation Testing (wb)
• Flowgraphs and Path testing (wb)
• Statement, branch, path coverage (wb)
• Basis path coverage (wb)

22
Many others...

• domain testing, Data flow, Test Scenarios
• Partitions boundaries, intervals,
• Guerilla testing
• Smoke tests
• many more (why)

(lightly,but harshly test an area for a short
time)
(a narrow set of tests which determines if more
extensive testing is warranted)
Techniques implement strategies, provide the
data needed to do the assessment
To give focus and coverage
23
Questions?
24
Next Class

• Testing Papers due tomorrow
• Presenters Girard/Pollard
• Start Regression Testing
• Friday - Midterm debrief
• - Lab review