Testing Basics

1
Testing Basics

• Detection probability

2
Triangle Example
cin gtgt a gtgt b gtgt c type scalene if (a b
a c b c) type isosceles if (a
b a c) type equilateral if (a gt bc
b gt ac c gt ab) typenot a
triangle if (a lt 0 b lt 0 c lt 0)
typebad input coutltlt type
3
Control Flow Graph
Operational profile 3,3,3 abcdegi equi 3,3,4 abce
gi isos 3,3,5 abcegi isos 3,3,6 abcefgi not 3,4
,3 abcegi isos 3,4,4 abcegi isos 3,4,5 acegi sc
al 3,4,6 acegi scal All inputs are equally
likely
4
What are the failure probability for each color
(separately)?
cin gtgt a gtgt b gtgt c type scalene if (a b
a c b c) type isosceles if (a
b a c) type equilateral if (a gt bc
b gt ac c gt ab) typenot a triangle if
(a lt 0 b lt 0 c lt 0) typebad
input coutltlt type
Blue Green Red
5
TTYP per path

• What is the probability of detection with one
  randomly chosen test case per path?
• What is the probability of detection with an
  equal number of randomly chosen test cases?

6
TTYP smaller subdomains

• What might be better smaller subdomains?
• Would MCC (multiple condition coverage) be better
  subdomains

7
TTYP2 C0 and C1 coverage

• How do we deal with C0 and C1 coverage since they
  are not subdomain testing methodologies?

8
TTYP3

• How could you estimate the det prob of C0 or C1
  testing?

9
Evaluating Testing Methods by Delivered
Reliability

• Frankl, Hamlet, Littlewood, Strigini
• IEEE TOSE Aug98

10
Testing

• Debug
• Operational

11
Fault Detection Probability

• Probability of a testing methodology finding a
  fault (if it existed)

12
Partition vs Random
13
Tests, Specifications, meets

• Test or test case
• single value of program input
• functional program - one input produces an output
• Specification - S
• set of input-output pairs
• Program meets specification
• iff for all x in spec, actual output matches spec
  output

14
Q probability distribution

• Q - probability distribution over input domain
• QD -gt 0,1 and S Q(t) 1

15
Q Failure Probability

• Q - failure probability for a randomly drawn
  point is S Qd
• Where d(t) 1 if f and 0 if s
• and f-phi(failure) and s-sigma(success)
• How does this relate to our notation?

16
Reliability

• R(N) (1- Q)N

17
Assumptions of initial model
18
Terms

• q
• d

19
3.2 SFR, w/o subdomains

• d StinF V(t)
• P(Q0) 1-(1-d)T
• P(Qq) (1-d)T
• E(Q) 0 P(Q0) q P(Qq)
• q(1-d)T