1
How to Measure the Impact of Specific
Development Practices on Fielded Defect Density

• by Ann Marie Neufelder
• Presented by Feride Padgett

2
Outline

• Introduction
• Results
• Conclusion

3
Introduction

• The author evaluated 17 organizations based on a
  questionnaire of 177 parameters
• The 17 organizations were
• Defense systems(5)
• Process control manufacturing equipment(11)
• Mathematical software(1)

4
The Questionnaire

• It had 5 sections
• Organization commitment
• Life cycle practices
• Product characteristics
• Change control practices
• Informational questions

5
Scoring

• Mathematical correlation of observed practices
  vs. observed defect density in terms of KSLOC of
  assembler
• A correlation of -1 means the practice perfectly
  correlates to LOWER defect density

6
High Scorers

• The common features of organizations with the
  lowest defect densities
• Software engineering is a part of the process,
  not an art form
• A well-rounded sound set of development practices
  vs. a single bullet
• Formal and informal reviews of requirements prior
  to design and code
• Testers are involved in requirements phase

7
Low Scorers

• The common features of organizations with the
  highest defect densities
• Lack of software management
• Misconception of programmer knows what customer
  needs better than the customer
• Complete void of requirements definition phase
• Insufficient testing methods

8
Ten Parameters with the Highest Correlation

• Consistent and documented formal and informal
  reviews of the software and system requirements
  prior to design and code -.87
• The language and operating system is well
  supported by industry -.86
• Existence and use of test beds -.85

9
Ten Parameters with the Highest Correlation

• Incremental testing (as opposed to big bang
  testing.) -.84
• Scheduled regression testing consistently
  performed by independent testers -.83
• Defect and failure tracking systems that are used
  by testers for test plan development -.81

10
Ten Parameters with the Highest Correlation

• There is a defined life cycle model that best
  suits the application, market, ad
  organization -.81
• Testers involved during requirements and design.
  Test plans are started during the requirements
  phase -.76

11
Ten Parameters with the Highest Correlation

• Automated unit testing tools -.76
• Explicit test cases for user documentation
• -.76

12
Reviews

• Requirements reviews -.87
• Design reviews -.65
• Coding reviews -.12

13
Results

• Software managers and engineers often assume
  engineering practices that decrease defects will
  also increase development time
• Her study shows that this assumption is not true
• Organizations with the best practices had the
  highest probability of making their schedules

14
Results

• Some companies were striving for more releases
  while their customer surveys showed that their
  customer wanted fewer releases with higher
  quality
• More corrective action releases mean more
  downtime for customers

15
Results

• This correlation can and is used to
• Predict defect density
• Improve software development practices for the
  best return on investment

16
Conclusion

• This is an ongoing work. New data is added on a
  yearly basis
• The study shows that making software engineering
  principles a part of the process decreases the
  defect density and increases the probability of
  on time delivery