Predictive Models to Achieve Business Results

1
Predictive Models to Achieve Business Results
Place your image on top of this gray box. If no
graphic i applicable, delete gray box and
notch-out behind gray box, fromthe Title Master

• 19th International Forum on COCOMO and Software
  Cost Modeling
• Cvetan Redzic, Michael Crowley, Nancy Eickelmann,
  Jongmoon Baik
• Motorola, Inc.
• October 26, 2004

2
Outline

• Overview
• Business Goals
• Models Used
• COQUALMO
• CoQ-DES
• MotoROI
• Primary Model Inputs
• CMM
• Life Cycle Scope
• PCE / PSE
• Results
• Cost
• Quality

3
Business Goal Improved Customer Satisfaction
SW Quality

• Type of needs
• Basic Expectations (Must Be)
• Satisfier - Features
• Delighters (Attractive)

Kano Analysis
4
Cause Effect Diagram
5
Integrating Predictive Models

• Models Used
• COQUALMO
• CoQ-DES
• MotoROI

6
COQUALMO
7
Combined COQUALMO Injection Factors
8
CoQ-DES
9
CoQ-DES Simulation
10
MotoROI
11
MotoROI - DOORS ROI Analysis
12
Model Integration - Primary Model Inputs

• CMM
• Life Cycle
• PCE / PSE

13
CMM Process Maturity

• COQUALMO
• PMAT (process maturity has the greatest
  /-impact) on injection rates
• CoQ-DES
• Not Used directly but is inherent in
  organizational calibration
• MotoROI
• Process maturity as represented by the cost of
  quality/cost of poor quality financial structure
  is a primary factor.

Knox Theoretical Model of TCOQ (About 50 at CMM
Level 3)
14
Life Cycle

• COQUALMO
• Req., Des., Imp., and Code
• CoQ-DES
• Full Life Cycle
• MotoROI
• Full Life Cycle or Individual Phases

15
PCE and PSE
Phase Containment Effectiveness Phase
Screening Effectiveness

• COQUALMO
• PCE and PSE as evidenced by injection and removal
  rates
• CoQ-DES
• PCE and PSE as evidenced by injection and removal
  rates
• MotoROI
• PCE for DP or PSE for technology effectiveness

16
Measuring and Monitoring Results

• Quality
• Cost

17
Quality - Sources of Variation
Actual vs. COQUALMO Estimate

• For Release with about 100 Delta
• KLOC, no significant difference estimates
  actuals in DI DR
• For large size Release over 100 Delta KLOC, there
  is significant difference b/w estimates actuals
  in DI DR for Code

  REQ DES CODE
Calculated Chi-Square Value 0.19 0.57 11.97
Chi-Square (20.05) 5.99    
Significance No No Yes
18
Quality - Sigma Level
From PCE, SRE CRUD data

• Sigma Level
• Defects per Million Opportunities
• DPMO 1M D/(NO)
• D 2464 HS Faults
• (from PCE)
• N 139,595 Delta LOC
• DPMO 1M 2464/139,595
• DPMO 17651 - 3.61 s
• Stable processes
• Need Leap improvement SEI CMM Level 5 TCM

What is Sigma Level from release perspective
? Relatively stable across the releases
19
Quality - SRE Goal Setting
20
Quality As-Is Process
21
Quality - Rayleigh Model Analysis
22
Quality - Impact of Tactical Changes
Monte-Carlo simulation, to include uncertainty
risks In the expert based opinion
23
Quality - New Process Baseline
24
Cost - Vital X Monthly Review Charts
SLIM
25
Quality - Vital X Monthly Review Charts
Fault Injection Removal vs. Baselines
26
Quality - Vital X Monthly Review Charts
SRE
27
CRUD Goal Tracking
28
Summary

• Integrating predictive models provides multiple
  views of project quality, cost and schedule
  issues.
• More accurate predictions of defect injection are
  possible
• More accurate predictions of defect removal are
  possible
• More accurate predictions of overall staffing and
  project cost are possible