INTROSE Introduction to Software Engineering

1
INTROSE Introduction to Software Engineering
Introduction to Software Testing

• Raymund Sison, PhD
• College of Computer Studies
• De La Salle University
• sisonr_at_dlsu.edu.ph

2

• What is software testing?

3
Short Definition Software Testing

• The assessment of software quality.

4
What is software quality?

• The quality of a piece of software is the degree
  to which it conforms to standards or requirements
  set by
• The softwares users
• The software organization
• The software industry ( other industries)

5
What is software quality?

• These standards could be external quality
  attributes
• Correctness
• Usability
• Efficiency
• Reliability

6
What is software quality?

• or internal quality attributes
• Maintainability
• Reusability
• Readability
• Testability

7
Short Definition Software Quality

• Conformance to standards/requirements.

8

• Testing Activities

9
2 Major Classes of Testing Activities

• 1. Dynamic testing
• Performed by running code on test cases
• Usually categorized into phases
• Unit Testing testing (by the developer) of a
  small element or unit of the system
• Integration Testing - a testing phase focused on
  the relationships and interfaces between pairs
  and groups of components or subsystems in the
  system
• System Testing - a testing phase focused on the
  behaviors, functions, and responses of the system
  as a whole
• Acceptance Testing - a testing phase designed to
  demonstrate that the system under test meets
  requirements.

10
Models of the Dynamic Testing Process

• Some models of the dynamic testing process in
  relation to the development process
• Waterfall
• Hinged Waterfall
• Parallel Waterfall

11
Waterfall Model of Development
Testing
Requirements
System Design
Program Design
Code Unit Test
Integration Testing
System Testing
Acceptance Testing
Operations Maintenance
12
V Diagram or Hinged Waterfall Model of
Development Testing
Validate requirements
Acceptance Testing
Requirements.
Verify design
System Testing
System Design
Verify design
Integration Testing
Program Design
Code Unit Test
13
Parallel Waterfall Model of
Development and Testing
Requirements
Verify
Validate
Test Thread
Development Thread
Verify
Verify
Verify
Verify
Verify
System Testing
Culbertson, R., Brown, C., Cobb, G. (2002). Rapid
Testing. Upper SaddleRiver, NJ Prentice-Hall.
Acceptance Testing
Operations Maintenance
14
Inputs and Outputs for the Activities on the Test
Thread
Culbertson, R., Brown, C., Cobb, G.(2002).Rapid
Testing. UpperSaddleRiver,NJPrentice-Hall.
15
Inputs and Outputs for the Activities on the Test
Thread
16
2 Major Classes of Testing Activities

• 2. Static testing
• Performed by examining documents
• Usually categorized according to the degree of
  formality
• Personal desk-checking
• Peer reviews
• Walkthroughs
• Inspections

17
Inspections vs. Walkthroughs vs. Peer Reviews
(Culbertson et al., 2002)
Culbertson, R., Brown, C., Cobb, G.(2002).Rapid
Testing. UpperSaddleRiver,NJPrentice-Hall.
18
Defect-Detection Rates of Various Testing
Activities
19
However, it is interesting to note that

• some studies show that pair programming/design
  might be more effective than peer reviews
  (Mühler, 2004) or inspections (Tomayko, 2002).

20

• Testing Techniques

21
Exhaustive Testing
loop lt 20x
There are 1014 possible paths! If we execute
one test per millisecond, it would take 3,170
years to test this program!
Pressman, R. (2004). Software Engineering, 6th
Ed. New York McGraw-Hill.
22
Selective Testing

• Since it is infeasible to test any non-trivial
  program exhaustively, we must turn to selective
  testing.
• The question is, how do we select which inputs or
  paths to test?

23
Testing Techniques

• Testing techniques attempt to answer the question
  of how to select inputs or paths to test.

24
2 Major Classes of Testing Techniques

• 1. Black box
• Also called behavioral or functional testing
• Focus is on whether a system/module produces
  expected outputs given specific inputs,
  regardless of how the inputs are processed to
  produce the outputs
• Problem Coincidental correctness

25
2 Major Classes of Testing Techniques

• 2. White box
• Also called structural testing
• Although the system/module must produce the right
  outputs given specific inputs, focus is on
  whether the inputs are processed correctly
• This addresses the problem of coincidental
  correctness
• But what is the problem with white-box
  techniques?

26
Some Black Box Techniques

• Domain testing
• Finite state testing
• Control-flow testing
• Loop testing
• Data flow testing
• Transaction flow testing
• Orthogonal arrays (Taguchi method)
• Use case testing

27
Some White Box Techniques

• Path testing
• Loop testing

28
Some Black Box Techniques

• Domain testing
• Finite state testing
• Control-flow testing
• Loop testing
• Data flow testing
• Transaction flow testing
• Orthogonal arrays (Taguchi method)
• Use case testing

29
Use Case Testing

• Compared to traditional testing techniques, use
  case testing offers a new perspective.
• All previously discussed techniques can be used
  in use case testing.

30
Example Purchase Order System
Collard, R. (1999). Use Case Testing. Proc.
Software Quality Engineering.
31
Example Use Case Description (1-Column Format)
32
Example Use Case Description (1-Column Format)
33
Example Use Case Description (1-Column Format)
34
Example Use Case Description (2-Column Format)
35
Example Use Case Description (2-Column Format)
36
Example Use Case Description (2-Column Format)
37
Use Case Testing Steps

• Verify the specifications always.
• Construct a model (control-flow, data-flow, or
  transaction-flow model) from the normal,
  alternate, and exception flows.
• Verify the model.
• Select the test paths/slices.
• Sensitize the test paths/slices.
• Predict and record the expected outcomes.

38
Use Case Testing Steps

• Define the verification criteria for each test.
• Run the tests.
• Confirm the outcomes.

39
Control-Flow Concepts

• Node
• A sequence of processing steps such that if any
  part of the sequence is executed, all will be
  executed (if there are no bugs)
• Link
• A directed arc from node A to B means that A is
  directly followed by B.

40
Control-Flow Concepts

• Predicate node
• A node with two or more outlinks each of which is
  weighted by a predicate value

T
Checked box 33c?
F
41
Control-Flow Concepts

• Selector node
• A node with more than 2 outlinks with each link
  weighted by a selector value

single
head
joint
separate
42
Control-Flow Concepts

• Junction node
• A node with 2 or more inlinks

3,800
0
5,600
43
Exercise 1

• Develop a control-flow graph for the Select
  Product use case.

44
Exercise 1
45
Exercise 1

• Develop a test case for the normal flow.

46
Exercise 1
47
Exercise 1
48
Exercise 1
49
Exercise 2

• Develop a set of test cases for the Search by
  Movie use case below.

50
Exercise 2
51
Exercise 2
52
Use Case TestingApplication Indicator

• Use case testing is best applied when the
  functional requirements of a system are specified
  using use cases.

53
Use Case TestingBug Assumptions

• The bug assumptions will depend on which
  lower-level testing technique(s) used.

54
Use Case TestingLimitations and Caveats

• Use case testing is a black-box technique, and
  therefore suffers from the limitation of all
  black-box techniques coincidental correctness.
• Use case testing, like all the techniques
  discussed so far, focuses on functional testing.
• The testing of each nonfunctional requirement
  (e.g., usability, performance) entails a
  different approach.

55
White Box Testing

• Black box techniques are typically used to check
  if the product conforms to its specifications.
• But what if the software performs some task that
  the black-box inputs have not detected?

56
Some White Box Techniques

• Path testing
• Loop testing

57
A Toy Program

• 1 program example(input,output)
• 2
• 6 begin
• 7 writeln(Input an integer between 1 and
  20)
• 8 readln(x)
• 9 while(xlt1) or (xgt20) do
• 10 begin
• 11 writeln (Input an integer between 1
  and 20)
• 12 readln(x)
• 13 end
• 14 writeln(Input,x,characters)
• 15 for i1 to x do
• 16 read(ai)
• 17 readln

58
A Toy Program

• 18 repeat
• 19 writeln(Input character to be searched
  for )
• 20 readln(c)
• 21 foundFALSE
• 22 i1
• 23 while(not(found)) and (iltx) do
• 24 begin
• 25 if ai c then
• 26 foundTRUE
• 27 else
• 28 ii1
• 29 end

59
A Toy Program

• 30 if found then
• 31 writeln(Character,c, appears at
  position,i)
• 32 else
• 33 writeln(Character,c, does not
  occur in the string)
• 34 writeln
• 35 writeln(Search for another character?
  y/n)
• 36 readln(response)
• 37 until (responsen) or (responseN)
• 38 end.

60
Exercise 3

• Construct the control flow graph for the given
  piece of code.
• Identify the minimal set of test data.

61
Exercise 3
while
9
8
repeat
7
10
if
6
5
for
14
if
4
3
17
until
2
while
62
Basis Path Testing

• Basis path testing is an easy way to determine
  the upper bound on the number of paths necessary
  to achieve branch coverage.
• McCabes Cyclomatic Complexity number is the
  number of linearly independent circuits (basis
  set) in a graph
• A linearly independent circuit is one that
  contains at least one new (i.e., previously
  unvisited) node and starts and ends on the same
  node

63
Cyclomatic Complexity

• The cyclomatic complexity of a graph, G, with e
  edges and n nodes is given by
• V(G) e n 2
• What is the cyclomatic complexity of the graph in
  the example/exercise?

64
Exercise 3

• List the basis set for the graph in the previous
  example/exercise.

65
Exercise 3

• Basis set
• 1-2-4-5-7-8-9-14-15-17-18
• 1-2-4-5-7-8-9-14-16-17-18
• 2-3-2
• 5-6-5
• 9-10-11-13-9
• 9-10-12-13-9
• 8-9-14-15-17-8

66
Exercise 3

• Basis path
• 1-2-4-5-7-8-9-10-11-13-9-14-15-17-18 (Character
  found in string)

67
Exercise 3

• Basis path
• 1-2-4-5-7-8-9- 9-10-12-13-9-14-16-17-18
  (Character NOT found in string)

68
Exercise 3

• Basis path
• 2-3-2 (Wrong input Size of string)

69
Exercise 3

• Basis path
• 5-6-5 (Two characters in string)

70
Exercise 3

• Basis path
• 8-9-14-15-17-8 (Another search)