Lecture 8: Testing, Verification and Validation

1
Lecture 8Testing, Verification and Validation

• Dr Valentina Plekhanova
• University of Sunderland, UK

2
Software Testing Fundamentals Some Testing
Terminology

• Faults - a mistake in the code that causes the
  software to not behave as expected (causes).
• Failures - the act of a product not behaving as
  expected - the manifestation of a fault
  (symptoms).

3
Some Testing Terminology

• Validation - establishing the fitness of a
  software product for its use - "are we building
  the right product?"
• Verification - establishing the correspondence
  between the software and its specification - "are
  we building the product right?"

4
Some Testing Terminology

• Test case - the collection of inputs, predicted
  results and execution conditions for a single
  test.
• Pass/fail criteria - decision rules used to
  determine whether a product passes or fails a
  given test.
• Test suite - a collection of test cases necessary
  to "adequately" test a product.

5
Some Testing Terminology

• Test plan - a document describing the scope,
  approach, resources and schedule of intended
  testing activity - identifies features to be
  tested, the testing tasks, who will do each task,
  and any risks requiring contingency planning.

6
Some Testing Terminology

• Bug report - a method of transmitting the
  occurrence of a discrepancy between actual and
  expected output to someone who cares for
  "follow-up" - also known as discrepancy report,
  defect report, problem report, etc.

7
What is Software Testing?

• Process of executing a computer program and
  comparing the actual behaviour with the expected
  behaviour.
• The comparison is intended to detect deviations
  (if any) between the actual behaviour and the
  expected behaviour.
• Testing serves as a barrier to allowing low
  quality products from reaching the customer.

8
Testing Objectives

• Testing is a process of executing a program with
  the intent of finding an error.
• A benefit of testing is that it demonstrates that
  the software appears to be working as stated in
  the specifications.

9
Testing Objectives

• The data collected through testing can also
  provide an indication of the software's
  reliability and quality.
• But, testing cannot show the absence of defect --
  it can only show that software defects are
  present.
• Functional Requirements and Non-functional
  Requirements must be tested.

10
Software Testing is Not Debugging

• Testing establishes the existence of defects
• Debugging is concerned with locating and
  correcting these defects.

11

• software testing discovers unexpected behaviour
• defect classification identifies the situation as
  a software error, a design error, a specification
  error, a testing error, etc. - if available, a
  "work-around" is identified
• if a software error, then debugging is performed
  which tracks down the cause of the error, and
  attempts to fix it.

12
The Testing Process
Unit Testing
Sub-System Testing
System Testing
Acceptance Testing or Alpha Testing
Beta Testing
13
The Testing Process Major Stages

• Unit Testing Individual components are tested
  to ensure that they operate correctly.
•  Module Testing A module is a collection of
  dependent components such as an objects,
  procedures, and functions. A module encapsulates
  related components so can be tested without other
  system modules.

14
The Testing Process

• Sub-System Testing This phase involves testing
  collections of modules, which have been
  integrated into sub-system. The sub-system test
  process should concentrate on the detection of
  interface errors by rigorously exercising these
  interfaces.

15
The Testing Process System Testing

• System Testing The testing process is concerned
  with finding errors, which normally result from
  unanticipated interactions between sub-systems
  and components. It is also concerned with
  validating that the system meets its functional
  and non-functional requirements.

16
The Testing Process Acceptance Testing

• Acceptance Testing Acceptance testing often
  reveals errors and omissions in the system
  requirements definition Sommerville.

17
The Testing Process

• Acceptance testing is sometimes called alpha
  testing.
• When a system is to be marked as a software
  product, a testing process called beta testing is
  often used.

18
The Testing Process Beta Testing

• Beta testing involves delivering a system to a
  number of potential customers who agree to use
  that system and to report problems to the system
  developers.

19
The Testing Process Regression Testing

• Errors in program components may be identified at
  the later stage of the testing process. Repairing
  program defects may introduce new defects so
  testing should be repeated after the system is
  modified. This is sometimes called regression
  testing Sommerville.

20
White Box Testing Sommerville

• White box testing is an approach to testing where
  the tests are derived from knowledge of the
  software structure.
• Test cases can be derived that
• guarantee that all independent paths within a
  module have been exercised at least once
• exercise all logical decisions on their true and
  false sides
• execute all loops at their boundaries and within
  their operational bounds
• exercise internal data structures to ensure their
  validity.

21
Black Box Testing

• Black box testing attempts to derive sets of
  inputs that will fully exercise all the
  functional requirements of a system.
• It is not an alternative to white box testing.

22
Black Box Testing

• incorrect or missing functions
• interface errors
• errors in data structures or external database
  access
• performance errors
• initialisation and termination errors.

23
White Box Black Box Testing

• White box testing should be performed early in
  the testing process, while black box testing
  tends to be applied during later stages.

24
Tests are designed to answer the following
questions

• How is the function's validity tested?
• What classes of input will make good test cases?
• Is the system particularly sensitive to certain
  input values?
• How are the boundaries of a data class isolated?
• What data rates and data volume can the system
  tolerate?
• What effect will specific combinations of data
  have on system operation?