Software Testing and Quality Assurance

1
Software Testing and Quality Assurance

• Lecture 29 - Introduction to Software Testing

2
Lecture Outline

• Introduction to testing types.
• Testing activities.
• Introduction to test cases, test oracles and
  their execution.

3
Testing Policy

• Only exhaustive testing can show a program is
  free from defects. However, exhaustive testing is
  impossible
• Testing policies define the approach to be used
  in selecting system tests For example,
• All functions accessed through menus should be
  tested
• Combinations of functions accessed through the
  same menu should be tested
• Where user input is required, all functions must
  be tested with correct and incorrect input.

4
Software Testing Development Life Cycle
Requirements
System Testing
Integration Testing
Architectural Design
Unit Testing
Detail Design
Implementation
5
Unit Testing
module to be tested
results
software engineer
test cases
6
Unit Testing
module to be tested


interface
local data structures

boundary conditions
independent paths

error handling paths
test cases
7
Unit Testing
driver

interface
local data structures

Module
boundary conditions
independent paths

error handling paths
stub
stub
test cases
RESULTS
8
Integration Testing

• Involves building a system from its components
  and testing it for problems that arise from
  component interactions.
• Top-down integration
• Develop the skeleton of the system and populate
  it with components.
• Bottom-up integration
• Integrate infrastructure components then add
  functional components.

To simplify error localisation, systems should
be incrementally integrated.
9
Integration Testing

• The Big-bang Approach.
• Incremental Approach.

10
Top-Down Integration
A
top module is tested with
stubs
B
F
G
stubs are replaced one at
a time, "depth first"
C
as new modules are integrated,
some subset of tests is re-run
D
E
11
Bottom-Up Integration
A
B
F
G
drivers are replaced one at a
time, "depth first"
C
worker modules are grouped into
builds and integrated
D
E
cluster
12
Thread-Based Integration
A
Top modules are tested with stubs
B
F
G
C
Worker modules are grouped into
builds and integrated
D
E
cluster
13
System Testing

• System Functional Test
• Test entire system against the functional
  requirements.
• System Performance Test
• Test the non-functional requirements of the
  system. For example,
• Response times, load testing etc.
• System Acceptance Test
• Set of tests that the software must pass before
  it is accepted by the client.

14
Trivial Example

• You have been asked to write a term paper on
  Integration Testing in Component Based System.
  To do this, you need to find references from a
  range of library databases.
• You logs on to the KFUPM library system and uses
  the search facility to find relevant papers from
  IEEE, ACM and Elsevier databases.
• One paper of special interest requires
  authentication and you have to fill an online
  form to receive the paper.
• If you are allowed, the paper will be downloaded
  and ready for collection.
• An email will be send to you once the paper is
  ready.

Scenario-based Testing
15
Student Activity

• Identify the possible interactions for the system
  testing of library system.

16
Trivial Example - System Testing

• Test the login mechanism using correct and
  incorrect login.
• Test the search facility using queries against
  known source to check that the search mechanism
  is actually documents.
• Test system presentation facility to check that
  information about documents is displayed
  properly.
• Test the mechanism to request permission for
  downloading.
• Test the e-mail response indicating that the
  download document is available.

17
Regression Testing

• Change do not always effect the entire program.
• Change in one part of system can effect other
  part.
• After each change
• Entire test suite of a system must be run again.

Need for an automatic test suite execution.
18
Test Activities

• Boils down to selecting and executing test cases.
  Test case consists of
• Set of test inputs, of if the program is
  non-terminating, a sequence of test inputs.
• Expected results when the inputs are executed
  and
• Execution conditions or execution environment in
  which the inputs are to be executed.

These steps generally remain same from unit
testing to system testing.
19
Test Case Selection

• Coverage criterion
• Equivalence Partitioning
• Boundary-Value Analysis
• Coverage-Based Testing
• Control-flow
• Data-flow
• Expected behavior of every test input to be
  generated. (Test Oracles)
• Testing environment.

20
Test Oracles

• Determines whether or not the program has passed
  or failed the test case.
• A test oracle is
• A program
• A process
• A body of data
• In many cases - directly form the requirements.
• For example, a test case assessing performance -
  performance threshold.

Difficult to automate or to assess their quality
21
Test Execution

• Test inputs on the program-under-test
• Record the actual behavior.

Generally can be automated to an extend !!!!
22
Test Evaluation

• Compare the actual behavior with the expected
  behavior.

Generally can be automated to an extend !!!!
23
Test Reporting

• Report the outcome of the testing.
• Developers
• Project Mangers etc.

Generally can be automated to an extend !!!!
24
Key Points

• A system typically undergoes a range of testing
  types.
• Each type of testing is aimed at detecting
  different kinds of failures.
• Testing boils down to the selection and execution
  of test cases.