Software Testing

1
Software Testing

• Necessary to measure and certify quality in
  software

2
Software Testing

• Often a neglected activity
• May use 20 of total development effort
• Can not do exhaustive testing
• Hard for programmer to test own program
• subconsciously we want testing to succeed

3
Historical Overview

• 1960s - very little testing
• 1968 - Hoares paper on proving
• 1970s - only good test is one that fails
• early 70s - reliability
• 1975 - Goodenough Gerharts paper on finite
  test sets
• mid 70s - (industrial) separate test teams
• late 70s - test coverage measures

4
TTYP 1

• Assume that you are given a program to test. The
  program accepts three numbers, interprets those
  numbers as lengths of the sides of a triangle,
  and returns the type of the triangle (scalene,
  isosceles, or equilateral)
• Write a good set of test cases for this program

5
Exhaustive Testing is impractical
6
Exhaustive Testing (2)
7
Typical Solutions

• We cant do exhaustive testing
• We cant test even a moderate percentage of the
  possibilities
• We can try to choose good tests
• Solution 1 Independent Test Teams
• Solution 2 Coverage Measures

8
Two common types of test coverage

• functional testing (cs541)
• black box
• based on requirements
• structural testing (cs540)
• white box
• based on code

9
Triangle Example
cin gtgt a gtgt b gtgt c type scalene if (a b
a c b c) type isoscoles if (a
b a c) type equilateral if (a gt bc
b gt ac c gt ab) typenot a
triangle if (a lt 0 b lt 0 c lt 0)
typebad input coutltlt type
TTYP 2 what is a set of tests that execute all
statements?
10
Control Flow Graph
TTYP3 what are the paths through this code?
11
A Model of Testing - Mappings

• a program maps elements of the domain to
  elements of the range
• the specification also specifies a mapping of the
  domain to the range
• correctness is that the program is the same
  mapping as the specification

12
Domains in testing
13
Subdomain typical definition

• A subdomain is a set of input points (data
  points, tests, etc) that are treated identically
  by the program (or almost identically). That is,
  they usually take the same path through the
  program and the same operations are applied to
  the values.
• Subdomains are normally mutually exclusive

14
TTYP 4

• What is a set of subdomains for the triangle code?

15
The specification
16
Program vs Specification

• Correctness is only in respect to a specification
• a program without a specification is correct
• Testing can not ignore the specification
• Testing can not ignore the program