Black Box Unit Testing

1
Black Box Unit Testing
1209
2
What is black-box testing?

• Unit (code, module) seen as a black box
• No access to the internal or logical structure
• Determine if given input produces expected
  output.

Output
Input
3
Black Box Testing

• Test set is derived from requirements
• Goal is to cover the input space
• Lots of approaches to describing input space
• Equivalence Classes
• Boundary Value Analysis
• Decision Tables
• State Transitions
• Use Cases
• . . .

4
Advantage and Disadvantage

• Advantage
• Disadvantage
• Need to define an efficient strategy to limit
  number of test cases

5
General Black Box Process

• Analyze requirements
• Select valid and invalid inputs
• Determine expected outputs
• Construct tests
• Run tests
• Compare actual outputs to expected outputs

6
Equivalence Relations

• Reflexive
• Symmetric
• Transitive

7
Equivalence classes Basic Strategy

• Partition the input into equivalence classes
• Test a subset from each class

8
Basic strategy

• Example fact(n)
• if nlt0, or ngt200 error
• if 0ltnlt20, exact value
• if 20ltnlt200, approximate value within .1

9
Simple Example

• Suppose you are building an airline reservation
  system. A traveler can be a child, an adult, or a
  senior.
• The price depends on the type of traveler.
• The seat reservation does not depend on the type
  of traveler.
• How many test cases are required for the
  reservation component and the billing component?

10
Finding equivalence classes

• Identify restrictions for inputs and outputs in
  the specification
• If there is a continuous numerical domain, create
  one valid and two or three invalid classes
  (above, below, and NaN)
• If a number of values is required, create one
  valid and two invalid classes

11
Boundary Values

• Programs that fail at interior elements of a
  class usually fail at the boundaries too.
• Test the boundaries.

12
Hints

• If a domain is a restricted set, check the
  boundaries. e.g., D1,10, test 0, 1, 10, 11
• It may be possible to test the boundaries of
  outputs, also.
• For ordered sets, check the first and last
  elements
• For complex data structures, the empty list, full
  lists, the zero array, and the null pointer
  should be tested

13
Boundary Value in class

• Determine the boundary values for US Postal
  Service ZIP codes
• Determine the boundary values for a 15- character
  last name entry.

14
Decision Tables

• Construct a table (to help organize the testing)
• Identify each rule or condition in the system
  that depends on some input
• For each input to one of these rules, list the
  combinations of inputs and the expected results

15
Decision Table Example
Theater ticket prices are discounted for senior
citizens and students.
Test Case C1 Student C2 Senior Result Discount?
1
2
3
4
16
Pairwise Testing
17
Problem 1From Lee Copeland, A Practitioners
Guide to Software Test Design, Artech House
Publishers, 2004.

• A web site must operate correctly with different
  browsers IE 5, IE 6, and IE 7 Mozilla 1.1
  Opera 7 and FireFox 2, 3, and 4.
• It must work using RealPlayer, MediaPlayer, or no
  plugin.
• It needs to run under Windows ME, NT, 2000, XP,
  and Vista, and 7.0
• It needs to accept pages from IIS, Apache and
  WebLogic running on Windows NT, 2000, and Linux
  servers

18
Problem 2
From Lee Copeland, A Practitioners Guide to
Software Test Design, Artech House Publishers,
2004.

• A bank is ready to test a data processing system
• Customer types
• Gold (i.e., normal)
• Platinum (i.e., important)
• Business
• Non profits
• Account types
• Checking
• Savings
• Mortgages
• Consumer loans
• Commercial loans
• States (with different rules) CA, NV, UT, ID,
  AZ, NM

19
When given a large number of combinations
(options improve )

• Give up and dont test
• Test all combinations miss targets, delay
  product launch, and go out of business
• Choose one or two cases

20
Orthogonal Arrays

• A 2-D array with the property
• All pairwise combinations occur in every pair of
  columns
• Example consider 3 variables (columns) with
  a,b, 1,2, and (a,ß)

1 2 3
1 1 A a
2 1 B ß
3 2 A ß
4 2 B a
21
Orthogonal Array Example
1 2 3
1 1 A a
2 1 B ß
3 2 A ß
4 2 B a
Look at each pair of columns (1 and 2),( 1 and
3), and (2 and 3) Does each of the 4 pairings
appear in each? (Yes, of course!)
22
Pairwise Testing Algorithm

1. Identify variables
2. Determine choices for each variable
3. Locate an orthogonal array
4. Map test cases to the orthogonal array
5. Construct tests

23
Example using Bank

• Identify variables
• Determine choices for each variable
• Locate an orthogonal array
• Map test cases to the orthogonal array
• Construct tests

24
Locate an orthogonal array (look up on web)
1 1 1
1 2 2
1 3 3
1 4 4
2 1 2
2 2 1
2 3 4
2 4 3
3 1 3
3 2 4
3 3 1
3 4 2
4 1 4
4 2 3
4 3 2
4 4 1
5 5 1
5 1 2
5 2 3
5 3 4
6 5 2
6 1 1
6 2 4
6 3 3
1 5 3
2 5 4
3 5 1
4 5 2
5 4 1
6 4 2
25
Map Test Cases (30)
CA Check Gold
CA Save Plat
CA Mort Busi
CA Cons NonP
NV Check Plat
NV Save Gold
NV Mort NonP
NV Cons Busi
UT Check Busi
UT Save NonP
UT Mort Gold
UT Cons Plat
ID Check NonP
ID Save Busi
ID Mort Plat
ID Cons Gold
AZ Comm Gold
AZ Check Plat
AZ Save Busi
AZ Mort NonP
NM Comm Plat
NM Check Gold
NM Save NonP
NM Mort Busi
CA Comm Busi
NV Comm NonP
UT Comm Gold
ID Comm Plat
AZ Cons Gold
NM Cons Plat
26
Pairwise Summary

• When the combination is large, test all pairs,
  not all combinations
• Studies indicate that most defects are single or
  double-mode effects
• Can be found in pairing
• Few defects require more than two modes
• Orthogonal arrays have the pairwise property and
  can be found or generated

27
State Transition Testing
28
State Transition Testing

• Build STD of system or component
• Cover the STD

29
Example
Payment made
Ticket Printed
Paid
Res Made
Ticketed
Cancel
Cancel/refund
Cust Order/start timer
Time Expires
Cancelled Cust
Cancelled Non Pay
Ticket Delivered
Cancel/refund
Used
30
Use Case Testing