An Eternal Triangle

1
An Eternal Triangle

• Fault Size

• Testability

• D-to-R Ratio

Martin Woodward, U. Liverpool, UK Zuhoor
Al-Khanjari, SQU, Oman
2
Outline of Talk

• Functional view of software
• Domain-to-range ratio (DRR)
• Domain testability and DRR
• PIE testability and fault size
• PIE testability via mutation
• Fault size and DRR
• Related work
• Conclusions

3
A Functional View of Software

• Correct program P D R
• Suppose single fault f exists.

4
Domain-to-Range Ratio (DRR)

• DRR D / R, where
• D cardinality of inputs
• R cardinality of outputs
• High DRR, i.e. DgtR Þ
• greater potential to hide faults
• internal state collapse
• DRR is a rough approximation of the softwares
  testability

DgtR
DR
5
Domain Testability(Freedman, 1991)

• Observability distinct outputs are generated
  from distinct inputs
• " a, b ÎD P(a)¹P(b) Þ a¹b
• Controllability given any desired output value,
  an extra input exists which forces the
  component to that value
• " r Î R d Î D P(d) r

6
Domain Testability Extensions

• Observable extensions
• introduce n new input variables with types Ti
• Ob log2( T1 . . . Tn )
• Controllable extensions
• reduce the range by modifying m output variable
  types to Tk
• Ct log2( T1 . . . Tm )

7
Domain Testability and DRR

• Extend D by ?D giving D
• Reduce R by ?R giving R
• For testable component

8
PIE Testability and Fault Size

• PrFailurePrE?PrI E?PrP I
• EExecute, IInfect, PPropagate

9
PIE Testability via Mutation

• Let ML be representative non-equivalent mutant of
  location L
• TestabilityL min (semantic size of ML)
• Find proportion of large number of random test
  cases that kills ML
• Traditional mutation stops testing a mutant once
  dead
• c.f. size estimates for mutation operators
  (Offutt Hayes, 96)

10
Fault Size and DRR

• Define
• input fault size Df / D
• output fault size Rf / R
• DRR for correct program P under full input domain
  D
• Can be related to DRR for faulty version under Df
  

11
Related Work

• PIE (Voas,92), DRR(Voas,95)
• Fault size (Offutt Hayes, 96)
• Fault-based testing theory (Morell, 90)
• RELAY model (Richardson et al., 93).
• Dynamic impact analysis (Goradia, 93)
• Prioritizing fault-exposing tests (Rothermel et
  al., 99)

12
Conclusions

• Testability
• is an important attribute
• has various guises
• and links with other concepts
• Domain testability and DRR
• PIE testability and fault size
• Validation possible by
• empirical studies
• analytic approach (How Tai Wah,
  2000)