Statistical Software Quality Assurance

1
Statistical Software Quality Assurance

• Implies
• Information about defects is collected and
  categorized
• An attempt is made to trace each defect to
  underlying cause
• Use of Pareto Principle to identify vital causes
• (80 of defects can be traced to 20 of
  causes/mistakes)
• Move to correct the problems that have caused the
  defects

2

• Software Engineering II
• Lecture 30
• Fakhar Lodhi

3

• Recap

4
Example

• Information about defects is collected for one
  year and categorized as follows
• Incomplete or erroneous specifications (IES)
• Misinterpretation of customer communication (MCC)
• Intentional deviation from specifications (IDS)
• Violation of programming standards (VPS)
• Error in data representation (EDR)
• Inconsistent component interface (ICI)
• Error in digital logic (EDL)
• Incomplete or erroneous testing (IET)
• Inaccurate or incomplete documentation (IID)
• Error in programming language translation or
  design (PLT)
• Ambiguous or inconsistent human computer
  interface (HCI)
• Miscellaneous (MIS)

5
Error Category Serious Moderate Minor Sub Total
IES 34 68 103 205
MCC 12 68 76 156
IDS 1 24 23 48
VPS 0 15 10 25
EDR 26 68 36 130
ICI 9 18 31 58
EDL 14 12 19 45
IET 12 35 48 95
IID 2 20 14 36
PLT 15 19 26 60
HCI 3 17 8 28
MIS 0 15 41 56
   
Total 128 379 435 942
6
Error Category Sub Total of Total errors
IES 205 22
MCC 156 17
IDS 48 5
VPS 25 3
EDR 130 14
ICI 58 6
EDL 45 5
IET 95 10
IID 36 4
PLT 60 6
HCI 28 3
MIS 56 6
   
Total 942  
IES, MCC and EDR are vital errors - cause 53 of
all errors
7
Error Category Serious of Serious errors
IES 34 27
MCC 12 9
IDS 1 1
VPS 0 0
EDR 26 20
ICI 9 7
EDL 14 11
IET 12 9
IID 2 2
PLT 15 12
HCI 3 2
MIS 0 0
 
Total 128  
IES, EDR, PLT and EDL constitute about 80 of
serious errors
8
Excel example

• Now start corrective action focused on vital few
• For example for EDR
• Review the data representation techniques to
  identify the possible improvement areas
• Adopt a use case tool for data modeling and
  perform stringent data design reviews

9
Error Index (EI)

• Used to develop an overall indication of
  improvement in software quality
• Ei the total number of errors uncovered during
  the ith step in the SE process
• Si number of serious errors
• Mi number of moderate errors
• Ti number of minor errors
• PSi product size at the ith step
• ws, wm, wt weighting factors for serious,
  moderate, and minor errors
• Recommended values 10, 3, 1 respectively

10
Error Index (EI)

• At each step of the software process a Phase
  Index is computed
• PIi ws(Si/Ei) wm(Mi/Ei) wt(Ti/Ei)
• EI cumulative effect on each PIi ?(i x
  PIi)/PSi
• Weighting errors encountered in the SE processes
  more heavily than those encountered earlier
• Used to develop an overall indication of
  improvement in software quality

11
Software Reliability

• Defined as
• Probability of failure free operation of a
  computer program in a specified environment for a
  specified time
• E.g. program X is estimated to have a reliability
  of 0.96 over 8 elapsed hours.
• What is meant by the term failure?
• Failure is non-conformance to software
  requirements
• Grading
• From annoying to catastrophic
• Time to fix from minutes to months
• Ripples from fixing

12
Software reliability

• Hardware versus software reliability
• Hardware reliability is predicted on failure due
  to wear, rather than failure due to design
• Software no wear and tear
• Mean time between failure MTBF
• MTBF MTTF MTTR
• where MTTF is mean time to failure and
• MTTR is mean time to repair

13
Software reliability

• Arguably MTBF is far better than defects/kloc
• Each error does not have the same failure rate
• User is concerned with failure and not with total
  error count