Testing: General Requirements, DFT, Multilevel Testing

1
Testing General Requirements, DFT, Multilevel
Testing
Testing General Requirements DFT Multilevel
Testing-- System, Black Box, White Box Tests
2
Testing--General Requirements

• Testing--general requirements
• thorough
• ongoing
• DEVELOPED WITH DESIGN (DFT--design for test)
• note this implies that several LEVELS of
  testing will be carried out
• efficient

3
Good, Bad, and Successful Tests

• good test has a high probability of finding an
  error
• ("bad test" not likely to find anything new)
• successful test finds a new error

4
Most Effective Testing Is Independent
most effective testing by an "independent
third party Question what does this imply
about your team testing strategy for the quarter
project?
5
How Thoroughly Can We Test?
how thoroughly can we test? example VLSI
chip 200 inputs 2000 flipflops (one-bit
memory cells) exhaustive tests? What is
the overall time to test if we can do 1 test /
msec? 1 test / msec? 1 test /nsec?
6
Matching Testing with Design and Language
How will testing match with design/language? e.g.
procedural example--sort an array oo
example--develop an "array class what kinds
of test need to be done in each case?
7
Design for Testability (DFT)

• Design for Testability (DFT)--what makes software
  "testable"?
• operability few bugs, incremental test
• observability you can see the results of the
  test
• controllability you can control state input
  to test
• decomposability you can decompose into smaller
  problems and test each separately
• simplicity you choose the simplest solution
  that will work
• stability same test will give same results each
  time
• understandability you understand code, inputs,
  and outputs

8
Testing strategies
testing strategies verification--functions
correctly implemented validation--we are
implementing the correct functions (according
to requirements)
9
Spiral design/testing strategy
A general design/testing strategy can be
described as a "spiral requirements ? design
? code system test ?module,integ. tests ?unit
test (system) (black
(white box) box) when
is testing complete? One model "logarithmic
Poisson model f(t)(1/p)ln(I0pt1) f(t)
cumulative expected failures at time t I0
failures per time unit at beginning of testing p
reduction rate in failure intensity
Design/Module Tests
Code/Unit Tests
Design/Integration Tests
START
END
Requirements, Specs/System Tests
10
Development and Testing Stages
levels of test___ development stage white
box code black box class integrati
on ER-diag. system use case
11
Types of testing

• Types of testing
• white box--"internals (also called "glass box")
• black boxmodules and their "interfaces (also
  called "behavioral")
• system--functionality (can be based on specs,
  use cases)
• application-specific--
• GUIs
• Client/Server
• Real-time
• Documentation/help

12
Good testing strategy

• steps in good test strategy
• quantified requirements
• test objectives explicit
• user requirements clear
• use "rapid cycle testing"
• build self-testing software
• filter errors by technical reviews
• review test cases and strategy formally also
• continually improve testing process

13
OO testing strategy

• OO testing
• emphasis is on interfaces
• use UML tools to support testing strategies and
  development of test cases
• --system tests use cases quality measurements
• --black box tests ER diagrams, object message
  diagrams,
• dataflow and state diagrams
• --white box tests class and state diagrams, CRC
  cards

14
Black box tests system tests

• Black Box Tests
• System Tests

15
Black box testing--what to test

• black box testing test functional requirements
  of components.
• Must check
• incorrect or missing components
• interface errors
• data structures or external data access
• behavior /performance errors
• initialization and termination errors

16
Black box testing--testing graph

• start point is usually a graph
• objects to be modeled
• relationships connecting them ("links")
• link properties
• example can use ER diagrams, object message
  diagrams, state diagrams, with additional
  information on links as necessary
• Example p. 404, Pressman

17
Black box testing--examples

• Examples

Denotes link that leads to one or more test cases
18
Black box testing techniques

• Some useful techniques for black box testing
• transaction flow
• data flow
• state modeling
• timing modeling
• (all "dynamic")
• much testing focuses on "boundary values" between
  components

19
Black box testing--input partitioning
example equivalence partitioning for input for
black box test condition cases of
input range 1 valid, 2 invalid value 1
valid, 2 invalid set member 1 valid, 1
invalid boolean 1 valid, 1 invalid
20
Black box testing guidelines
General guidelines test BOUNDARIES test
output also choose "orthogonal cases if
possible
21
Using specifications for system tests
System tests should verify that specifications
have been met For UML-based strategy each
use case ---gt one or more system tests each
quality / performance requirement ? one or
more system tests Additional qualitative or
quantitative tests (not from use
cases) examples is system user-friendly?
are timing requirements met? are available
resources sufficient?
22
Using specifications for system tests
Example
Associated sequence diagrams 1. 2. 3.

Cellular

network


2. Receive call






Associated test cases 1. 2. 3.
3. Use

scheduler


User





3 use cases

Tests verify use case supported