COCOMO II Calibration Status

1
COCOMO II Calibration Status

• USC-CSE Annual Research Review
• March 2004

2
A Little History

• Calibration effort started in January 2002
• Confusion
• Repository in an inconsistent state
• Uncharacterized data from many sources
• Process for duplicating the 2000 calibration
  results
• Schedule compression rating was inconsistent
• Expectation
• New data had a lot of variation but
• Affiliates (and the user population in general)
  want an Accurate and up-to-date model not
  just one that explained variation
• PRED(.25) versus R2

3
Change in Approach

• Removed pre-1990 data from dataset used in
  calibration
• This removed a lot of converted data
• Removed bad data
• Incomplete No duration data, estimated effort,
  no valid SLOC size
• Still use the Bayesian calibration approach
  developed by Chulani
• Changed to a holistic analysis approach
  considered effort and duration together
• Identified data that needed review
• Schedule compression was automatically set

4
Effort- Duration Error
5
Effort- Duration Error Interpretation
6
Effort Estimate Error Compared to Size
7
Duration Estimate Error Compared to Size
8
Preliminary Results

• 89 project data
• Effort Estimation Accuracy
• PRED(.30) 92.1 (was 75 for 2000 calibration)
• PRED(.25) 88.8 (was 68)
• PRED(.20) 84.3 (was 63)
• Duration Estimation Accuracy
• PRED(.30) 82.0 (was 64 for 2000 calibration)
• PRED(.25) 70.8 (was 55)
• PRED(.20) 64.0 (was 50)
• 65 more project data to review

9
Next Steps in Calibration

• Review and incorporate the remaining outstanding
  data
• Beta test the new driver values with some of the
  Affiliates
• Assess the need to elaborate (and possibly
  expand) the definition for Schedule Compression
• Attempt to develop pre-sets for different types
  of applications

10
How Should the Model Handle Level-loaded staffing?
11
Are Compression / Stretch-out Handled Adequately?

• Should there be an Extra Low rating?
• Should there be ratings for stretch-out other
  than 1.0

12
COCOMO II Driver Elaboration
13
Motivation

• Recent experience has shown that engineers have
  difficulty in rating some of the COCOMO II
  Drivers
• Researchers have documented impact of COCOMO
  Drivers subjectivity
• Calibration data appears to show some differences
  in the interpretation of COCOMO II Driver ratings
• Our goal is to address these problems

14
Examples of Questions
Programmer Capability What is the 75th
percentile?
Database Size Are you mixing two different
units of measure?
(DB bytes /
lt
100
D/P

DATA
lt
Pgm SLOC)

10
D/P lt 100

D/P gt 1000

lt 1000

lt 10

Required Software Reliability What if my
application is not financial?
Moderate,
low, easily
slight
easily
high

risk to

RELY

recoverable
inconvenience

recoverable
financial loss

human life

losses

losses


15
Elaborating Cost Driver Workshop Goals

• No new math (elaboration not re-definition)
• No scare-factor (not too many inputs)
• Something behind the curtain
• Gradual unfolding
• More comprehensible vocabulary
• Consider wider range of application users
• Applicable to Business and Military
• Make it easier to use
• Eye-ball average, optional weighted scoring
• Make it less subjective
• Crisper definitions

16
Driver Subjectivity Ranking

• Each driver was rated (1 to 10) for its level of
  subjectivity by participants.
• Ratings 1 Very Objective, 10 Very Subjective

17
Status

• Held a workshop in October 2003 on identifying
  drivers that need further elaboration
• Lack of progress
• Need an effective collaboration technique that is
  not moderator intensive
• Calibration work has taken precedence
• Possible collaboration solution exists by using
  same technology USC Software Engineering classes
  use for submission of their work.
• Calibration work is winding down (I think)
• Expect email soon on the startup of the Driver
  Elaboration working group

18
For more information, requests or questions
Brad Clark Software Metrics, Inc. brad_at_software-me
trics.com Ye Yang USC-CSE yey_at_usc.edu