Estimating Tester to Developer Ratios

1
Estimating Tester to Developer Ratios

• Adjusting Standard Ratios Based on Technology,
  Process Maturity, and Skill Levels

Dana Spears (Liberty RAM, Sr. Software Engineer,
QE) Susan Morgan (Liberty RAM, Software Engineer,
QC)
2
Acknowledgement
Thank you to Kathy Iberle (Hewlett Packard) and
Sue Bartlett (IIS/STEP Technology) for use and
ideas from Estimating Tester to Developer Ratios
(Or Not) presented at the Pacific Northwest
Software Quality Conference. http//www.kiberle.co
m/ Johanna Rothman (Rothman Consulting Group,
INC.) for use and ideas from It Depends
Deciding on the Correct Ration of Developers to
Tester. Cem Kaner, J.D., Ph.D.(Flordia Institute
of Technology), Elisabeth Hendrickson (Quality
Tree Software, INC.) and Jennifer Smith-Brock
(Ajilon Software Quality Partners) for use and
ideas from Managing the Proportion of Tester to
(Other) Developers presented at Pacific
Northwest Software Quality Conference. Michael A
Eason (Peopleclick) for use and ideas from
Sizing a Software Quality Assurance Group,
Theory and Application
3
Tester/Developer Ratios - Why we Test

• Manage Risk
• Developed product wont meet the customers needs
• To asses the product state.
• We cant keep up with development, we need more
  testers.
• How do you justify your claim?

4
Tester/Developer Ratios Theories

• Ratio theory generally recommends
• 31 at most when
• Very mature processes
• Accurate, clear documentation
• Non-configurable application
• 11 or less when
• Little or no process
• Documentation unclear / out of date
• Highly configurable application
• These are generalizations at best but are useful
  for comparison purposes

5
Tester/Developer Ratios - Can They Be Used
Accurately?

• Can ratios be used to some level of accuracy?
• No one size fits all
• Adjustments should be made based on several
  factors

6
Tester/Developer Ratios Ideal Size Adjustment
Factors

• Application
• Legal / Regulatory
• Process
• Scope
• Experience / Expertise
• Environment

7
Tester/Developer Ratios Cause-Effect Diagram
8
Tester/Developer Ratios - Can They Be Used
Accurately and Why is sizing relevant?

• Right-sizing a companys Testing Team
• Under-sizing
• Oversizing
• Measurements
• The size of the Testing Team

9
How we approach Tester/Developer Ratio for new
projects
10
What Liberty Mutual does to adjust a standard
ratio for New projects?

• Started with a standard 13 ratio.
• Adjusted using a rating system and weighting in
  the following categories
• Project Information
• Num of Developers by Skill Level
• Review Skill Level
• Type of Testing
• Developed a scoring system for the total weighted
  values that correspond to an adjustment to the
  standard ratio.
• All developed in Excel for easy changing and use
  in everyday testing environments.

11
Project Information Subcategories

• Project Platform Data, Mainframe, Web?
• Development Effort New Development,
  Maintenance, Rework, Maintenance Enhancement.
• Has project planning started?
• Phase in SDLC Inception, Elaboration,
  Construction, Transition, Maintenance

12
Skill Level of Developers Subcategories

• Advanced
• Intermediate to Advanced
• Novice to Intermediate
• New to Technology

13
Review Subcategories

• Requirements Review
• Requirements and Design Reviews
• Code walkthrough and Reviews
• Unit Testing

14
Testing Type Subcategories

• Unit Testing
• Automated
• Manual
• System Testing
• Automated
• Manual
• Both

15
Weighting Scheme for Development of Ratio

• Each subcategory was given a weight based on its
  value to the overall development process.
• Each value for a subcategory was assigned a
  number. The higher the number, the more
  detrimental to the process the subcategory value
  could be.

16
Value/Weighting Example
Developer Skill
17
Value/Weighting Example (contd)
Where n number v value wt weight
In our example, let us assume that there are 7
developers on a project with the following mix of
experience 1 Advanced, 2 Intermediate to
Advanced, 1 Novice to Intermediate, and 3 New to
Technology. Total
18
Value/Weighting Example (contd)

• Developer Skill Weighted Value
• ((1/7)1)((2/7)2)((1/7)3)
• ((3/7)4).20
• (1/7)(4/7)(3/7)(12/7).20
• .57

19
Calculation of Weighted Value

• Continue working a similar formula for all
  categories/subcategories.
• Add all weighted values to determine a total
  weighted value for the project.
• Score the value to determine the number of
  additional testers needed.

20
Scoring
21
How we approach Tester/Developer Ratio for
existing projects
22
Tester/Developer Ratios - Summary

• Sometimes adding resources are not the answer
• Use a second method
• Your organization is unique
• Ratios are like metrics

23
Tester/Developer Ratios

• Questions and Answers