The Organizational Impacts on Software Quality and Defect Estimation

1
The Organizational Impacts on Software Quality
and Defect Estimation

• Stephen Lopez-Couto

2
Discussion Topics

• Introduction
• Defect Concepts
• Definition of a defect
• Static vs. Dynamic Discovery
• Defect Estimation Methods
• Linear Regression
• Capture/Recapture
• AI
• Defect Estimation Variables
• Further Research
• Conclusion

3
Introduction

• Defect Estimation
• The act of guessing the number of defects that
  exist in a current software baseline
• Purpose of this paper is to determine the
  usefulness of metrics that are not explicitly
  tied to the code to make the estimates

4
Introduction

• Organizational Elements
• Development Team Size
• Developer Experience
• Institutional Processes
• CMMI level for example
• Development Tools
• Development Schedule
• Programming Language
• Software Architecture

5
Defect Concepts What is a Defect?

• There is no common definition of a defect
• Consider the following
• Program is supposed to add the variables y and
  i together and put the value into x

int i 5 int y 6 int x y1 Print(x)
6
Defect Concepts What is a Defect?
The programmer introduced a defect Correct code
int x y i

• There is no common definition of a defect
• Consider the following
• Program is supposed to add the variables y and
  i together and put the value into x

int i 5 int y 6 int x y1 Print(x)
7
Defect Concepts What is a Defect?

• Two things occur before a defect can be detected
• Fault Incorrect value in the internal state of
  the program
• Failure When a fault is realized as output

int i 5 int y 6 int x y1 // Causes a
Fault Print(x) // Causes a Failure
A Failure may not be detected until well after
the fault has occurred!
8
Defect Concepts Static Vs. Dynamic Discovery

• Static Does not utilize executing code
• Software Inspections
• Walkthroughs
• Complexity Mapping
• Dynamic Utilizes executing, compiled code
• Runtime tests
• Automated defect discovery tools

9
Defect Estimation Methods

• Estimating the number of defects is by no means
  an exact science
• There are lots of different ways to determine an
  estimate
• Most of the methods are highly tied to a specific
  organization or software baseline
• Not good enough for general use
• Three general methods will be discussed
• Linear Regression
• Capture/Recapture
• Artificial Intelligence
• Bayesian Belief Networks
• Neural Networks

10
Defect Estimation MethodsLinear Regression

• Method
• Determines a mathematical expression that relates
  some number of independent variables (the input
  metrics) and the dependent variable (defect
  estimation)
• Variable Type Numeric Only

11
Defect Estimation MethodsCapture/Recapture

• Method
• Some number of defects are seeded into the
  software at development time. When testing occurs
  ratio of seeded to unseeded defects found is used
  to estimate the total number.
• Variable Type N/A. Only track defect counts.

12
Defect Estimation MethodsAI Bayesian Belief
Networks

• Method
• A network that weighs the relationship among
  different variables is created (using expert
  knowledge) and processed using Bayesian
  probabilities to determine the quality of the
  software.
• Variable Type any

13
Defect Estimation MethodsAI Neural Networks

• Method
• A multilayer perceptron using the back
  propagation algorithm is trained on legacy defect
  data and then provides estimates based on input
  data.
• Variable Type Numeric Only

14
Defect Estimation Variables
15
Further Research

• Three main areas
• Transformation of non numerical data into a
  numerical form
• Additional estimation methods that do not utilize
  numeric only data
• Determination of common relationships among the
  input metrics, independent of which method is used

16
Conclusion

• Organizational Element Data is useful when paired
  with the proper estimation method
• Limitations of the most common method (linear
  regression) has led to a general shunning of
  these metrics
• Defect Estimation approaches are not ready for
  real world use
• They are too tied to specific cases

17
Questions

• Questions?