Chapter 23: Software Testing

1
Chapter 23 Software Testing

• Yonsei University
• 2nd Semester, 2009
• Sanghyun Park

2
Topics Covered

• System Testing
• Component Testing
• Test Case Design
• Test Automation

3
The Testing Process

• Component testing
• Testing of _________ program components
• Usually the responsibility of the component
  ________(except sometimes for critical systems)
• Tests are derived from the developers
  experience
• System testing
• Testing of groups of components integrated to
  create a system or sub-system
• The responsibility of an independent testing
  _____
• Tests are based on a system ___________

4
Defect Testing

• The goal of defect testing is to _________
  defects in programs
• A successful defect test is a test which causes a
  program to behave in an _________ way
• Tests show the presence not the _________ of
  defects

5
The Software Testing Process
6
Testing Policies

• Only _________ testing can show a program is free
  from defects. However, exhaustive testing is
  impossible
• Testing policies define the approach to be used
  in selecting system tests. For example,
• All functions accessed through menus should be
  tested
• Combinations of functions accessed through the
  same menu should be tested
• Where user input is required, all functions must
  be tested with correct and ________ input

7
System Testing

• Involves __________ components to create a system
  or sub-system
• In an iterative development process, it is
  concerned with testing an __________ to be
  delivered to the customer
• In a waterfall process, it is concerned with
  testing the ______ system
• For most complex systems, there are two distinct
  phases
• Integration testing
• Release testing

8
Integration Testing

• Involves building a system from its components
  and testing the resultant system for problems
  that arise from component __________
• Top-down integration
• Develop the overall _______ of the system first
  and then add components to it
• Bottom-up integration
• Integrate __________ components that provide
  common services, such as network and database
  access, and then add the __________ components
• To simplify error localization, components should
  be integrated ____________

9
Incremental Integration Testing
10
Testing Approaches

• Architectural validation
• ________ integration testing is better at
  discovering errors in the system architecture and
  high-level design
• System demonstration
• _________ integration testing allows a limited
  demonstration at an early stage in the
  development
• Test implementation
• Often easier with __________ integration testing
• Test observation
• Problems with both approaches.Extra code may be
  required to observe tests

11
Release Testing

• The process of testing a release of a system that
  will bedistributed to customers
• Primary goal is to increase the suppliers
  confidence thatthe system meets its ___________
• Release testing is usually _________ or
  functional testing
• Based on the system _________ only
• Testers do not have knowledge of the system
  ____________

12
Black-Box Testing
13
Testing Guidelines

• Testing guidelines are hints for the testing team
  to help them choose tests that will reveal
  defects in the system
• Choose inputs that force the system to generate
  all _____ messages
• Design inputs that cause buffers to ________
• Repeat the same input or series of inputs
  numerous times
• Force _______ outputs to be generated
• Force computation results to be too _____ or too
  small

14
Performance Testing

• Part of release testing may involve testing the
  _________properties of a system, such as
  performance and reliability
• Performance tests usually involve planning a
  series of tests where the _____ is steadily
  increased until the system performance becomes
  unacceptable

15
Stress Testing

• Exercises the system ______ its maximum design
  load. Stressing the system often causes defects
  to come to light
• Stressing the system tests failure
  behaviour.Systems should not fail _____________.
  Stress testing checks for unacceptable _____ of
  service or data
• Stress testing is particularly relevant to
  ____________ systems that can exhibit severe
  degradation as a network becomes overloaded

16
Component Testing

• Component or unit testing is the process of
  testing _________ components in isolation
• It is a defect testing process so its goal is to
  expose faults in these components
• There are different types of component that may
  be tested as this stage
• Individual __________ or methods within an object
• Object ______ with several attributes and methods
• _________ components made up of several different
  objects or functions. They have a defined
  interface that is used toaccess their
  functionality

17
Object Class Testing

• Object class testing should include
• The testing in isolation of all _________
  associated with the object
• The setting and interrogation of all ________
  associated with the object
• The exercise of the object in all possible
  ______.This means that all events that cause a
  state change in the object should be simulated
• ___________ makes it more difficult to design
  object class tests as the information to be
  tested is not localized

18
Interface Testing

• Objectives are to detect faults due to ________
  errors or invalid assumptions about interfaces
• Particularly important for object-oriented
  development as objects are defined by their
  _________

19
Interface Types

• ___________ interfaces
• Data passed from one procedure to another
• ___________ memory interfaces
• Block of memory is shared between procedures or
  functions
• ___________ interfaces
• One component encapsulates a set of procedures
  that can be called by other components
• ___________ passing interfaces
• One component requests a service from another
  component by passing a message to it

20
Interface Errors

• Interface misuse
• A calling component calls some other component
  and makes an error in its use of its interface
  (e.g. parameters in the wrong order)
• Interface misunderstanding
• A calling component misunderstands the
  specification of the interface of the called
  component and makes assumptions about the
  behaviour of the called component
• Timing errors
• The called and the calling component operate at
  different speeds and out-of-date information is
  accessed

21
Interface Testing Guidelines

• Design tests so that parameters to a called
  procedure are at the ________ ends of their
  ranges
• Always test pointer parameters with ____
  pointers
• Design tests which cause the component to fail
• Use stress testing in message passing systems
• In shared memory systems, vary the _____ in which
  components are activated

22
Test Case Design

• Involves designing the test cases (inputs and
  outputs) used to test the system
• The goal of test case design is to create a set
  of tests that are effective in validation and
  defect testing
• Design approaches
• Requirement-based testing
• Partition testing
• Structural testing

23
Requirement-Based Testing

• A general principle of requirement engineering is
  that requirements should be _______
• Requirement-based testing is a validation testing
  technique where we consider each _________
  individually and derive a set of tests for that
  requirement

24
LIBSYS Requirements

• The user shall be able to search either __ of the
  initial set of databases or select a _____ from
  it
• The system shall provide appropriate ______ for
  the user to read documents in the document store
• Every ____ shall be allocated a unique ________
  (ORDER_ID) that the user shall be able to copy to
  the accounts permanent storage area

25
Tests For LIBSYS Requirement 1

• Initiate user searches for items that are known
  to be present and known not to be present, where
  the set of databases includes ___ database
• Initiate user searches for items that are know to
  be present and known not to be present, where the
  set of databases includes ___ databases
• Initiate user searches for items that are know to
  be present and known not to be present, where the
  set of databases includes _________ two databases
• Select ___ database from the set of databases and
  initiate user searches for items that are known
  to be present and known not to be present
• Select _________ one database from the set of
  databases and initiate user searches for items
  that are known to be present and known not to be
  present

26
Partition Testing

• Input data and output results of a program
  usually fallinto a number of different ______
  that have _______ characteristics such as
  positive numbers, negative numbers, and menu
  selections
• Programs normally behave in a _________ way for
  all members of a class
• Because of this equivalent behaviour, these
  classes are sometimes called equivalent ________
  or domains
• Test cases should be chosen from each partition

27
Equivalence Partitioning (1/2)
28
Equivalence Partitioning (2/2)
29
Search Routine Specification
procedure Search (Key ELEM T SEQ of ELEM
Found in out BOOLEAN L in out
ELEM_INDEX) Pre-condition -- the sequence has
at least one element TFIRST lt TLAST
Post-condition -- the element is found and is
referenced by L (Found and T(L) Key) or --
the element is not in the sequence (not Found
and not (exists i, TFIRST lt i lt
TLAST, T (i) Key))
30
Search Routine Input Partitions

• Inputs which _________ to the pre-condition
• Inputs which ________ conform the pre-condition
• Inputs where the key element is a _________ of
  the sequence
• Inputs where the key element is ____ a member of
  the sequence

31
Testing Guidelines (Sequences)

• Test with sequences of ____ length
• Test with sequences which have only a ______
  value
• Use different sequences of different sizes in
  different tests
• Derive tests so that the first, ______ and last
  elements of the sequence are accessed

32
Search Routine Input Partitions
33
Structural Testing (1/2)

• The tests are derived from knowledge of the
  softwares ________ and implementation
• This testing is sometimes called _________,
  glass-box testing or clear-box testing to
  distinguish it from _________ testing
• Understanding the algorithm used in a component
  can help us identify further partitions and test
  cases

34
Structural Testing (2/2)
35
Path Testing (1/2)

• Path testing is a _________ testing strategy
  whose objective is to exercise every independent
  execution ____ through a component or a program
• If every independent path is executed, then all
  statements in the component must have been
  executed at least once
• Furthermore, all ________ statements are tested
  for both true and false cases

36
Path Testing (2/2)

• The starting point for path testing is a program
  ___ graph that is a skeletal model of all paths
  through the program
• A flow graph consists of nodes representing
  ________ and edges showing flow of ______
• The flow graph is constructed by replacing
  program control statements with equivalent
  diagrams

37
Flow Graph For Binary Search
38
Independent Paths

• 1, 2, 8, 9
• 1, 2, 3, 8, 9
• 1, 2, 3, 4, 5, 7, 2,
• 1, 2, 3, 4, 6, 7, 2,
• Test cases should be derived so that all of these
  paths are executed
• A dynamic program analyzer may be used to check
  that paths have been executed

39
Test Automation

• Testing is an expensive and laborious phase of
  the software process
• As a result, testing tools were among the first
  software tools to be developed
• These tools now offer a range of facilities and
  their use can significantly reduce the costs of
  testing
• _____ is a set of Java classes that the user
  extends to create an automated testing
  environment

40
Software Testing Workbench

• A software testing workbench is an integrated set
  of tools to support the testing process
• A workbench includes
• Test manager
• Test data generator
• Oracle
• File comparator
• Report generator
• Dynamic analyzer
• Simulator

41
A Testing Workbench