Software Cost Estimation at GSFC - PowerPoint PPT Presentation

1 / 22
About This Presentation
Title:

Software Cost Estimation at GSFC

Description:

... from Mission Critical Software 'Look Back' study. Based on similar study ... a long life system (to support the goal of a 15 year consistent climate data set) ... – PowerPoint PPT presentation

Number of Views:42
Avg rating:3.0/5.0
Slides: 23
Provided by: lindaro4
Category:

less

Transcript and Presenter's Notes

Title: Software Cost Estimation at GSFC


1
Software Cost Estimation at GSFC

2
GSFC Cost EstimationAgenda
  • Overview
  • Examples of project cost data
  • Mission Critical Software cost growth study
  • Flight Software Approach
  • Ground Science Software Approach
  • Wide band Delphi method a team approach to cost
    estimation
  • Cost estimates and the software management plan
    (product plan).

3
Cost EstimationExamples from missions
  • Provided by experienced software managers
  • Some examples
  • Flight MAP was 75 staff years, EO-1 20
  • Ground
  • Total including science operations Great
    Observatory 60-80 staff years, Small Explorer
    25-35
  • Mission data system range of 10 (SWIFT)-20 (MAP)
    staff years through launch
  • Where does the money go?
  • (MAP flight SW example)

4
Study Of Why SW Cost Grows
  • Results from Mission Critical Software Look
    Back study
  • Based on similar study at JPL
  • Estimates are made by manager of 7 ground
    software and 7 flight software projects What is
    the size of cost growth? What causes the cost
    growth?
  • Size of cost growth

5
Causes Of Cost Growth From Study
  • Contribution to Overall Cost Growth
  • (add to 100, w/ round off)

6
Causes Of Flight SW Cost Growth
  • Planning
  • Specific Project pressures to reduce cost
    estimates due to cost constraints, in some
    instances never allowing a true estimate to be
    baselined
  • Post-PDR mandatory cost reductions across the
    board
  • Unrealistic dependencies within delivery schedule
    (e.g., requirements not specified as scheduled,
    hardware not available as scheduled)
  • Requirements
  • New requirements often levied in late mission
    design or IT phase, as systems personnel become
    more knowledgeable
  • Incomplete requirements
  • Development and test ramifications are not fully
    understood
  • Reduction of Perceived Overhead
  • Minimized production or review of documentation
  • Minimized formal and informal review activities

7
Causes Of Ground SW Cost Growth
  • Planning
  • Project pressure to reduce overall cost estimates
  • COTS capabilities not functioning as
    planned/advertised
  • Requirements
  • Maturity of requirements limits accuracy of
    preliminary cost estimates
  • Disparity in completeness of functional
    requirements (e.g., telemetry and command
    requirements are often well understood, while
    science processing requirements are left
    incomplete until late in mission lifecycle)
  • Testing
  • Insufficient emphasis on test plan completeness
  • Application of additional testing as primary risk
    mitigation
  • Staffing
  • Launch delays result in funding and maintaining
    marching army
  • Lack of adequate dedicated staffing early in
    project lifecycle

8
Implications
  • It is tempting to cut back estimated costs, but
  • This implies implementing fewer features, or
  • This implies jettisoning proven engineering
    practices
  • THIS WILL COST MORE DURING TESTING!
  • Requirements always change, plan a reserve!
  • Provide best case, most likely case, and worst
    case
  • Measure development costs for
  • Tracking against plan
  • Refining estimation models for future projects

9
Cost EstimationHow Do We Estimate Costs?
  • General approach
  • Comparison with previous missions
  • Adjust based on technical management risk
    factors
  • Example Flight Software Branch
    feature-by-feature estimation
  • Break into subsystems
  • Estimates are made for each subsystem
  • Experienced developers rate complexity of each
    feature using previous missions as guide
  • Rationale for each rating should be documented
  • Points assigned to complexity correlate with
    effort
  • Subsystem efforts are totaled
  • Adjust total for project risk

10
Flight SW Cost EstimationACS subsystem
development
  • Estimator compares each required feature with
    prior missions creates a new rating (last
    column)
  • Scores are added to get overall ACS complexity
  • When 89 features are rated for ACS, estimation
    errors tend to offset each other
  • Linear regression or other technique is used to
    relate estimated cost to complexity estimates,
    e.g.
  • Effort (staff years) 0.2points 6

11
Flight SW Cost EstimationFlight Software Base
Cost
...
ACS Estimate
PSE Estimate
C DH Estimate
Instrument Estimate
Project Management Estimate
Dev. Test Environment Estimate
Subsystem Development Estimate
System Testing Estimate
Project HW Subsystem Sup. Estimate
Sustaining Engineering Estimate
FSW BASE COST ESTIMATE
12
Flight SW Cost EstimationAdjusting for Risk
  • For a set of risk factors, rate each risk as
  • Extremely high
  • High
  • Nominal
  • Low
  • Extremely low
  • Based on these ratings, adjust cost estimate
  • Models such as COCOMO have tables of multipliers
    for scale or exponent, corresponding to these
    ratings
  • Flight Software Branch uses expert judgment

13
Flight SW Cost EstimationRisk factors to use
  • Flight Data System Hardware selection
  • Distributed system or not
  • Mission Hardware selections and redundancies
  • Ground System Selections for FSW Test, IT and
    Operations
  • FSW Testbeds
  • Operations Concepts
  • Ground Contact Opportunities
  • Failure Detection and Response Philosophy
  • Deployment and in-orbit insertion plan
  • Experience of team lead
  • Quality of Tools available to the FSW Team
  • Schedule compression

14
Ground Systems Costing Practices
  • Lead Engineer costs all ground systems
    components, including component functional domain
    experts as necessary
  • Most Recent Missions Use Heritage Systems or COTS
    Solutions, e.g.,
  • ITOS on Triana
  • Epoch 2000 on NGST

15
Science Systems Cost Estimation
  • Systems Costed by Developing or Operational
    Organizations
  • Most Recent Mission Science Data Systems are
    Evolutions of Existing Data Processing Systems,
    e.g.,
  • GPMs PPS is an evolution of TRMMs TSDIS
  • In Some Cases, a Level of Effort is Provided to
    Science Labs for System Development
  • Process is Mostly Informal

16
HW/SW Maintenance CostCOTS Model of EOSDIS
Growth
  • EOSDIS is a long life system (to support the
    goal of a 15 year consistent climate data set)
  • Spreadsheet model for phased computational and
    I/O throughput and archive volume capacity based
    on
  • Mission launch schedules (Landsat, Terra, Aqua,
    Aura, et.al.)
  • Reprocessing and product deletion phasing
    assumptions
  • Assumptions about end user demand for product
    distribution
  • Translated into projected COTS (and COTS
    maintenance) costs and support capacity
    requirements
  • Based on current design and calibrated to recent
    COTS purchases
  • Incorporating estimated future price/performance
    curves for several equipment categories (e.g.
    CPU, disk, archive media)
  • Technology refresh trade-offs also included (vs.
    maintenance)

17
Wide-band Delphi Cost EstimationOverview
  • Iterative method multiple experts create
    estimates
  • Many variations of Delphi methods exist.
  • Estimators meet to analyze discrepancies between
    estimates
  • Meeting notes capture varying assumptions
  • Agree on assumptions for next round
  • Estimators create new estimates based on new
    assumptions
  • Process is repeated until estimates converge
  • Moderator Preparation
  • Provide materials
  • Provide initial assumptions
  • Select team
  • Conduct initial meeting

Round 1 by individuals Produce independent
estimates
  • Moderator produce estimatesummary
  • Individual median estimates
  • New assumptions

Round 2 Reconsider based on all infor-mation in
estimate summary
Consensus?
Meet to reachconsensus
Done!
18
Wide-band Delphi Cost EstimationTypical
assumptions are about
  • Development team experience
  • Requirements quality
  • Completeness
  • Clarity
  • Stability
  • Available tools
  • Reuse approach
  • Imposed constraints
  • Cost and/or schedule
  • Tools, programming language,

19
Wide-band Delphi Cost Estimation Estimation Team
  • Team must be
  • Experienced
  • In type of mission software
  • With GSFC software development processes
  • Representative experts in
  • Development
  • Testing
  • Software project management
  • Maintenance

20
Wide-band Delphi Cost EstimationEstimation by
individuals
  • Initial phase
  • No collaboration between estimators
  • Make best case, most likely case, and worst case
    estimate (can apply methods taught in class)
  • Document issues, questions assumptions
  • Revision phase
  • Moderator provides Estimation Summary
  • (anonymous) individual and median estimates
  • Summary of issues, questions assumptions
  • Individuals revise based on Estimation Summary

21
Software Management PlanCost Estimation in
context
Example from Flight SW Product Plan Template
22
GSFC Cost Estimation Summary
  • Where are we?
  • Information Systems Cost Estimates Rely on the
    Knowledge and Experience of a Few Experts
  • Estimates are as good as the expert
  • It is sometimes Difficult to Ascertain Actual
    Past Missions Costs
  • Where are we going?
  • Efforts are Underway to Document/Formalize Cost
    Estimation Practices
  • Some of what you are seeing in this course is
    likely to show up in the documented processes
    later
Write a Comment
User Comments (0)
About PowerShow.com