CS 425/625 Software Engineering Software Testing

1
CS 425/625 Software Engineering Software
Testing

• Based on Chapter 20 of the textbook Somm00 Ian
  Sommerville,
• Software Engineering, 6th Ed., Addison-Wesley,
  2000 and on the
• Ch20 PowerPoint presentation available at the
  books web-site
• www.comp.lancs.ac.uk/computing/resources/IanS/SE6/
  Slides/index.html
• November 12, 2002

2
Outline

• Introduction
• Defect Testing
• Black Box Testing
• Equivalence Partitioning
• Structural (White Box) Testing
• Path Testing
• Integration Testing
• Top-Down
• Bottom-Up
• Interface Testing
• Stress Testing
• Object Oriented Testing

3
Introduction..

• A very high level view of testing phases Fig.
  20.1, Somm00

4
.Introduction.

• Component testing is concerned with checking the
  functionality of methods, classes, and modules
• Integration testing focuses on sub-systems and
  their interaction as well as on the operation of
  the system as a whole
• Component testing is individual programmers task
  while integration testing is assigned to an
  integration team
• For critical systems independent testing teams
  are used during all testing
• Integration testing is based on written
  requirements specifications

5
..Introduction

• In object-oriented testing the boundaries between
  component and integration testing are blurred
  because
• Object-oriented systems do not have the clear
  distinction between program units and program
  modules characteristic to structured
  (function-oriented) systems
• The hierarchy of objects is not necessarily
  nested

6
Defect Testing

• Defect testing is aimed at discovering errors in
  code
• A successful defect test causes the system to
  perform incorrectly
• During the testing process the following are
  created
• Test cases
• Test data
• Test reports

7
Defect Testing

• Comprehensive testing is not possible. Subsets of
  tests cases are designed based on criteria such
  as
• All code statements should be executed at least
  once
• All functions accessed through menus should be
  tested
• All combinations of functions accessed through
  the same menu should be checked
• All user input situations should be checked with
  both correct and incorrect data

8
Defect Testing The Process

• The defect testing process Fig. 20.2, Somm00

9
Defect Testing Black Box Testing.

• In black box testing (or functional testing) the
  system is viewed from outside, only in terms of
  its functionality
• The software is tested against its specification
• Internal (implementation) details are ignored
• The behavior of the system is evaluated in terms
  of inputs received, outputs expected, and outputs
  actually produced
• A defect is discovered if the actual system
  output differs from the expected output
• Applicable to both function-oriented and
  object-oriented systems

10
Defect Testing .Black Box Testing

• The black-box testing model Fig. 20.3, Somm00

11
Defect Testing Equivalence Partitioning.

• Equivalence partitioning makes use of classes of
  input test data (e.g., positive numbers, negative
  numbers, strings, etc.)
• Generally, systems tend to behave in a comparable
  way for all members of a class
• Classes of input test data are called equivalence
  partitions or domains
• Equivalence partitions are determined based on
  program specification and user documentation
• Test cases are derived for each equivalence
  partition
• Partition mid-point and boundary values provide
  useful test data

12
.Defect Testing .Equivalence Partitioning

• Equivalence partitioning Fig. 20.4, Somm00

13
..Defect Testing Equivalence Partitioning..

• Equivalence partitions Fig. 20.5, Somm00

14
Defect Testing Equivalence Partitioning.

• Example A search routine Fig. 20.6, Somm00

• procedure Search (Key ELEM T ELEM_ARRAY
• Found in out BOOLEAN L in out
  ELEM_INDEX)
• Pre-condition -- the array has at least one
  element
• TFIRST lt TLAST
• Post-condition -- the element is found and is
  referenced by L
• ( Found and T (L) Key) or
• -- the element is not in the array
• ( not Found and not (exists i, TFIRST lt i lt
  TLAST, T (i) Key ))

15
.Defect Testing Equivalence Partitioning

• Equivalence partitions for search routine Fig.
  20.7, Somm00

16
Defect Testing Structural Testing.

• In contrast to black box testing structural
  testing (or white box, clear box, glass box
  testing) is based on the knowledge the testers
  have about the structure and the implementation
  of the software
• Applied generally to smaller program units
• The tester analyzes the code to determine test
  cases
• Types of structural testing
• Branch coverage every branch of the unit is
  tested at least once
• Statement coverage every statement of the unit
  is executed at least once
• Path coverage every path through the unit is
  executed at least once

17
.Defect Testing Structural Testing

• The structural testing process Fig. 20.8, Somm00

18
Structural Testing Example

• Java
• implementation
• Fig. 20.9, Somm00

19
Defect Testing Structural Testing.

• Equivalence classes for binary search Fig.
  20.10, Somm00

20
.Defect Testing Structural Testing

• Test cases for binary search Fig. 20.11, Somm00

21
Defect Testing Path Testing

• Path testing is a form of structural testing
  aimed at exercising each individual execution
  path of a program unit. It ensures that
• Each statement of the unit is executed at least
  once
• Each branch condition is tested with both true
  and false conditions
• The program flow graph is used for path testing.
  In a flow graph decisions are shown as nodes and
  the flow of control is represented by edges

22
Defect Testing .Path Testing..

• The cyclomatic complexity CC of a connected graph
  G is given by the formula
• CC(G) Number of edges - Number of nodes 2
• If there are no go to statements and no compound
  decisions (involving more than one test) then
• CC(G) Number of predicate nodes 1
• (predicate nodes contain conditions that
  determine branching of the execution flow)
• The cyclomatic complexity of a graph gives the
  minimum number of test cases needed to cover all
  paths

23
Defect Testing ..Path Testing.

• Flow graph for binary search Fig. 20.12, Somm00

24
Defect Testing Path Testing

• Independent paths in previous case

• 1, 2, 3, 8, 9
• 1, 2, 3, 4, 6, 7, 2
• 1, 2, 3, 4, 5, 7, 2
• 1, 2, 3, 4, 6, 7, 2, 8, 9
• In programs with complex branching structure the
  number of paths is high and it is often difficult
  to predict the programs behaviour
• A dynamic program analyzer can be used to
  identify all paths

25
Integration Testing...

• Integration involves building sub-systems from
  program units and the system from subsystems
• Integration testing uses software specifications
  as basis for checking the system
• Errors are more difficult to locate than in unit
  testing
• The incremental approach is recommended for
  system integration and testing
• Initially, a minimal system configuration can be
  used
• Tests need be repeated after each addition to the
  system

26
.Integration Testing..

• Incremental integration testing Fig. 20.13,
  Somm00

27
..Integration Testing.

• Strategies for system integration and testing
• Top-Down
• Starts with high level components and continues
  at lower levels
• Integral part of the top-down development process
• Stubs are needed to test the system

28
Integration Testing

• Strategies for system integration and testing
  contd
• Bottom-Up
• Starts with lower level components and builds up
  the system by adding testing higher level
  components
• Can be used when components from previously
  developed systems are reused or when the systems
  functionality relies on critical lower level
  modules
• Driver modules are used to test the system
• Sandwich
• Combines the above two strategies by using the
  Top-Down approach for logic modules (higher
  level) and the Bottom-Up approach for operational
  (lower level) modules

29
.Integration Testing..

• Top-down integration testing Fig. 20.14, Somm00

30
..Integration Testing.

• Bottom-up integration testing Fig. 20.14, Somm00

31
...Integration Testing

• Top-Down versus Bottom-Up
• Architectural validation
• Top-Down allows earlier discovery of high-level
  design errors
• System demonstration
• Top-Down supports better an early demo Bottom-Up
  can support such demo if many reusable components
  are utilized
• Test implementation
• Strict Top-Down testing is more difficult to
  implement given that stubs are needed. It is
  easier to use drivers in a strict Bottom-Up
  testing
• Test observation
• Equally challenging, since both stubs and drivers
  may not allow complete observations

32
Interface Testing

• Interface testing has the goal of detecting
  errors related to interfacing modules and
  sub-systems
• Particularly important for object-oriented
  testing given objects collaboration relies on
  their interfaces
• Types of interfaces
• Parameter interfaces
• Shared memory interfaces
• Procedural interfaces
• Message passing interfaces
• Classes of interface errors
• Interface misuse
• Interface misunderstanding
• Timing errors

33
.Interface Testing..

• Interface testing Fig. 20.15, Somm00 tests
  directed at the sub-system as a whole, not at
  particular components

34
..Interface Testing.

• Guidelines for interface testing
• Test all external calls with parameters at
  extreme ranges
• When pointers are used, test interfaces with null
  pointers
• For procedural interfaces try to design tests
  that would cause the components to fail
• Use stress testing for message passing interfaces
• Change activation order of components that share
  memory

35
Interface Testing

• Stress testing exercise the systems with
  overloading (e.g., number of transactions per
  minute, number of users in a distributed
  applications)
• The main purposes of stress testing
• Make sure the system does not lose or corrupt
  data when overloaded
• Reveal defects that otherwise would pass
  unnoticed

36
Object-Oriented Testing..

• Differences from function-oriented systems
• Objects are usually larger units, which encompass
  several methods
• Objects are typically loosely coupled and there
  may not be an obvious high top of the class
  hierarchy
• Testers may not have access to the code of reused
  objects

37
.Object-Oriented Testing.

• Levels of testing in object-oriented testing
• Testing operations of object classes (sequences
  of operations may be needed)
• Testing object classes (operations, attributes,
  and states)
• Testing clusters of objects (use case or
  scenarios based testing thread testing, etc.)
  each operation in each class should be checked at
  least once
• Testing the object-oriented system

38
..Object-Oriented Testing

• Compared with testing function-oriented systems,
  testing of object-oriented system is facilitated
  by inheritance reuse (previously tested
  objects) but specific challenges exist because
• Information hiding is a complicating factor
• Writing additional methods (invoked for testing
  purposes only) are often necessary
• New and overwritten methods must be tested
• Inherited methods must still be re-tested if they
  interact with newly written methods