SQA for Individuals

1
SQA for Individuals

• based on
• "A Personal Commitment to Software Quality"
• by W.S. Humphrey
• SEI at Carnegie Mellon University

2
Reasons for Looking at the Individual

• The root cause of all software defects is humans.
• Software productivity declines as product size
  increases.
• Increased complexity makes debugging difficult,
  so more time is spent in testing.
• If the all individual parts are of very high
  quality, your process can scale up with little
  reduction in productivity.

3
The Nature of Quality

• Most Important
• must provide necessary functions to the user
• includes usability and convenience
• Second Most Important
• the product must work
• Distance Third
• compatibility,
• and all the other "ilities"

4
Why Companies Don't do QA

• Schedules built on unreliable data force
  developers to spend much time reacting to crises.
• No time to do much if any QA
• hence, biggest budget item is usually Testing
• Stuck in rut of "test, debug, fix, repeat"
• Experimenting is a risk
• Developers have no data on the number of defects
  they inject or the costs of fixing those defects.

5
Costs of Defects

1. Determining that there is a problem
2. Isolating the source of the problem
3. Determining exactly what is wrong
4. Fixing the Requirements
5. Fixing the Design
6. Fixing the Implementation
7. Inspecting and Testing the fix
8. Testing the fix to ensure it doesn't cause other
   problems
9. Changing the Documentation to reflect the fix

6
Real Cost Data

• IBM (relative costs)
• during design 1.5
• prior to coding 1
• during coding 1.5
• during test 60
• in field use 100
• Reviews are 2 or 3 times as efficient as testing
  at finding defects.
• Combined design and code reviews have a yield of
  60 to 80.

7
Example

• Without any QA
• 50 KLOC with 2500 defects to be found
• average 4 hours of work per defect
• 10,000 hours to remove defects
• With Reviews
• 50 KLOC with 2500 defects
• 70 of defects removed via Reviews
• average 0.5 hours per defect
• 875 hours of work
• remaining 750 defects
• average 8 hours each
• 6000 hours
• Total 6,875 hours

8
Measures for Individuals

• Failure Costs
• total time in compile and test
• few developers count defects found during compile
  and individual unit testing
• Appraisal Costs
• time spent in design and code reviews
• any inspection time
• Prevention Costs
• prototype development
• casual analysis meetings
• process improvement actions or meetings

9
Measures for Individuals

• Yield (step n) 100 (defects removed in step
  n)
• (defects removed in n
  defects escaping n)
• Goals, perform small experiments to
• improve the yield of filters
• reduce the of defects injected
• improve productivity
• measured in LOC produced per month
• productivity increase comes from less time spent
  in testing