Cost Estimating Introductory Course

1

Cost Estimating Introductory Course
ATO Finance, and Investment Planning and
Analysis Maria DiPasquantonio, Manager
2
Class Participation Guidelines

• Participate freely in discussions
• Listen to other viewpoints
• Avoid distracting behaviors including sidebar
  conversations
• Try to relate your own experiences
• Ask questions when you need clarification
• Arrive on time
• Please turn your cell phone to either off or
  vibrate. Please do not interrupt the class by
  walking in or out of the classroom to answer the
  phone. Get it at the next break.
• Please dont use your e-mail on wireless devices
  during the class. It is obvious and distracting.

3
Topics to be Covered
Top Level View of Curriculum
Kickoff/Introductions/Course Objectives
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
4
More detailed view of Curriculum
Andrews Conference Room, BAE SYSTEMS _at_ the Aerospace 901 D St, Suite 850, Washington DC 20024. Time to Deliver (hours) Date for Instruction
Kickoff/Introductions/Course Objectives 0.5 27 Sept05
The Role of Cost Estimating and Analysis in FAA-ATO What is it? Why is it done? Who does it? When is it done? 0.75 27 Sept05
Cost Estimating Process Steps in the cost estimating process The FAA cost estimating process Work Breakdown Structures (WBS) Life Cycle Costs and the AMS The hallmarks of a good cost estimate 0.75 27 Sept 05
Data Sources and Collection Sources and Collection Index numbers, Inflation. Then year vs. Constant year Exercise Personnel Costs 2.5 27 Sept 05
5
More detailed view of Curriculum contd.
Andrews Conference Room, BAE SYSTEMS _at_ the Aerospace 901 D St, Suite 850, Washington DC 20024. Time to Deliver (hours) Date for Instruction
Cost Estimating Techniques Analogies Parametric Cost Factors ATO Below-the-line cost factors Proposed Exercise Pocket Estimating Guide (PEG Cost Factors 3.5 27 Sept 05
Sensitivity and Cost Risk 1.5 28 Sept 05
Software Cost Estimating Definition and Life Cycle Inherent difficulties Software Growth Software Development Software Maintenance Commercial models Data Sources and collection lessons COTS 1.5 28 Sept 05
Financial and Economic Analysis Benefit-to-cost ratios Discounting Present Value Exercise 2.0 28 Sept 05
References 0.5 28 Sept 05
Next Course Overview 0.5 28 Sept 05
6
Curriculum
Kickoff/Introductions/Course Objectives
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
7
Course Contents

• This course provides a broad-based understanding
  of the cost analysis activities involved in the
  acquisition and support of FAA programs. The
  course will introduce you to the methods used to
  perform cost estimates and help you to understand
  the fundamentals and terminology of the
  acquisition process.
• Topics include

• Inflation indices
• Time phasing of a cost estimate
• Risk and uncertainty analysis
• Time value of money
• Economic analysis

• FAA Acquisition and Management System (AMS) and
  Decision processes
• Development, use and analysis of estimating
  techniques
• Statistical and non-statistical cost estimating
  relationships ( Analogy, Parametric and
  Engineering Build-Up)

8
My Experience in the Cost Estimating Profession
POSITION MATH ECON MS CE/A
UNIVERSITY PROFESSOR X X
ARMY-OPS RES. ANALYST X X X X
EUROPE-CH.ECONOMIC ANALYSIS X X X
NAVAIR-DEP CH CE/A X X X
NAVAL CENTER FOR COST ANALYSIS-DIRECTOR X X X
BOOZ ALLEN HAMILTONPRINCIPAL X X X
NAVAL POSTGRAUATE SCHOOL-- VISITING PROFESSOR X X
Cost Estimating and Economics Seem Ubiquitous
9
Air Traffic Organization contd.
Cost estimates are developed within this
organization
10
Four Major Appropriations Fund the FAA

• Research, Engineering, Development (RED)
  supports research to improve safety, security,
  productivity, and capacity to meet the future
  needs of the national ATC system
• Facilities Equipment (FE) sustains current
  infrastructure and supports FAAs plan to
  modernize the NAS
• Operations Maintenance (OM) supports the
  day-to-day operation of the National Airspace
  System (NAS) and its personnel and support costs
• Airport Improvement Program (AIP) grants
  supports airport planning and development,
  safety, security, and noise mitigation

11
Four Major Appropriations Fund the FAA contd.

• Research, Engineering, Development (RED)
  supports research to improve safety, security,
  productivity, and capacity to meet the future
  needs of the national ATC system
• Facilities Equipment (FE) sustains current
  infrastructure and supports FAAs plan to
  modernize the NAS
• Operations Maintenance (OM) supports the
  day-to-day operation of the National Airspace
  System (NAS) and its personnel and support costs
• Airport Improvement Program (AIP) grants
  supports airport planning and development,
  safety, security, and noise mitigation

Cost estimates are developed for projects using
this appropriation
12
FAA Budget, FY 2000-2006 (est.)
13
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis in FAA-ATO What is it? Why is it done? When is it done? Who does it?
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
14
The Need for Cost Estimating Skills at FAA-ATO

• Cost Growth and Schedule Delays of Major
  Acquisitions Continue To Stall NAS Modernization
  Efforts
• FAA Needs To Develop a Comprehensive Strategy for
  Modernizing the NAS To Ensure Major Acquisition
  Programs Are Delivered Within Cost, Schedule, and
  Performance Milestones
• STATUS OF FAAS MAJOR ACQUISITIONS COST
  GROWTH AND SCHEDULE DELAYS CONTINUE TO STALL AIR
  TRAFFIC MODERNIZATIONFederal Aviation
  AdministrationReport Number AV-2005-061Date
  Issued May 26, 2005

15
Cost Estimating- What?

• Cost Estimating
• The process of collecting and analyzing
  historical data and applying quantitative models,
  techniques, tools, and databases to predict the
  future cost of an item, product, program or task
• The art of approximating the probable worth (or
  cost) of something based on information available
  at the time
• The act of developing, analyzing, and
  documenting (the process, data, techniques, and
  results of) cost estimates using analytical
  approaches and techniques

16
Definition Cost Estimating is the process of
collecting and analyzing historical data and
applying quantitative models, techniques,
tools, and databases to predict the future cost
of an item, product, program or task
What Can Cost Estimating Do?
Good Cost Estimating Drives Good Decisions

• Determine and communicate a realistic view of the
  likely cost outcome, which can form the basis of
  the plan for executing the work
• Make decisions on program viability, structure,
  and resource requirements
• Establish, Support and defend budgets
• Translate requirements associated with programs,
  projects, or processes into budget requests
• Assess reasonableness of the budget
• Quickly/accurately determine impacts of budget
  cuts on program baselines and associated
  functionality
• Assess resource implications of technology
• Assist in source selection/design trade-offs

• Provide standardized procedures for developing
  budgets and consistent basis of estimate (BOE) in
  developing budgets
• Enhance accuracy and credibility of budget
  estimates
• Make better decisions based on metrics, not
  feeling

17
Purposes of cost estimating

• Translate system/functional requirements
  associated with programs, projects, or processes
  into budget requirements
• Determine and communicate a realistic view of the
  likely cost outcome, which can form the basis of
  the plan for executing the work
• Develop a source of resource information for
  planning purposes
• Provide a quantitative basis for management
  decisions regarding optimal allocation of
  resources
• Facilitate the decision-making process of
  choosing one option from among several
  alternatives
• Enable review and evaluation of progress towards
  goals

A cost analyst helps to decide which of the
possible alternatives is more desirable and
recommends a course of action that will steer
decision makers towards it and away from
undesirable alternatives
18
When is Cost Estimating Done?

• Perform source selection
• Perform design trade-offs
• Comply with public law
• Satisfy oversight requirements
• Identify and objectively quantify the impact of
  program risks (technical and schedule risks)
• Evaluate proposals for cost reasonableness

• Make decisions on program viability, structure,
  and resource requirements
• Establish and defend budgets
• Assess technology changes
• Provide basis for evaluating competing systems
  and /initiatives (cost/benefit analyses and AoAs)
• Conduct analysis of alternatives (AoA)

Cost estimating, as part of a total systems
analysis, provides an analytic underpinning to
support decision makers
19
Why Credible Cost Estimates are Important

• Overestimates lead to planning and
• budget disasters
• Underutilized resources
• Excess capacity
• Wasteful, un-competitive pricing
• Underestimates lead to implementation disasters
• Insufficient resources
• Panicked decisions
• Unrealistic expectations
• Imprudent economics

At the Service, OMB and Congressional levels,
loss of credibility results in loss of program
and financial flexibility, such as Congress
legislating de facto cost caps and/or schedules
in attempts to avoid future cost growth
20
Skillsets and Environment for Cost Estimators and
Analysts
STATISTICIAN
ENGINEER
COMPUTER SCIENTIST/ MATHEMATICIAN

• Regression Analysis
• Forecasting
• Sensitivity Testing
• Learning Curve Applications
• Analysis of Commercial Models

• Production Process
• Scheduling
• Materials
• Performance Parameters
• System Integration
• Production Engineering
• Test Program Development

• Model Development
• CER Development
• Programming
• Analysis of Proposals
• Analysis of Commercial Model

ECONOMIST
ACCOUNTANT

• Inflation-vs-Foreign
• Exchange Rates
• Labor Agreements
• Break Even Analysis
• Present Value Analysis

COST ESTIMATOR/ANALYST

• Financial Analysis
• Cost Data Analysis
• Proposal Analysis
• Overhead Analysis

BUDGETING
PUBLIC AFFAIRS
SALESPERSON

• Appropriation Process
• Program Specific
• Internal Company (Industry)
• Defense Budget Appropriations

• Sell Estimate
• Sell Approach
• Sell Self as Knowledgeable

• Congressional Process//Motivations
• GAO
• General Public
• Press

21
The Context of Investment Analysis Applicable
Orders and Guidelines
Government
Best Practices

• OMB A-94
• Benefit-Cost and Cost-Effectiveness Analysis
• Alternatives Analysis
• Net Present Value
• Discount Rates

• DOT
• Policy Values
• Fatalities
• Injuries
• Passenger Time

• FAA/APO
• Aviation Forecasts
• Economic Values
• Aircraft Operating Costs
• Aircraft Replacement Costs

• FAA/ATO F P
• Probabilistic Analysis
• Work Breakdown Structure of Costs and Benefits
• Risk Analysis

• OMB A-11 (Exhibit A-300)
• New Budget Submittal Rqmts.
• (Capital Planning and Investment Control)
• Business Case Focus
• ROI NPV and Payback
• Risk Analysis
• Mitigation Plans

• Legislation/Congress
• Clinger Cohen Act (IT investment management)
• Government Performance andResults Act (GPRA)
• Chief Financial Officer's (CFO) Act

22
Why is Investment Analysis Needed and What is the
Motivation for Business Case Analysis?

• To match capital investments with agency
  strategy, goals, and objectives (e.g., new
  capabilities)
• To match capital investment with operating needs
• To support allocation of scarce capital resources
  (e.g., dollars, people) to the highest value
  opportunities
• To comply with governmental directions (e.g.,
  Congress Clinger-Cohen Act, Government
  Performance and Results Act (GPRA), Office of
  Management and Budget (OMB) Circular A-94, and
  OMB Circular A-11)

• Implement integrated corporate investment
  decision making process
• Baseline programs
• Shorten acquisition cycle
• Focus on life cycle management
• Establish system-wide decision criteria
• Involve FAA lines of business (LOB), users, and
  industry

23
FAA Investment Analysis Process

• Figure 2.4.1 illustrates the phases and decision
  points of investment analysis, which is conducted
  to ensure the agencys critical needs are
  satisfied by practical and affordable solutions.
  Initial investment analysis rigorously evaluates
  alternative solutions to mission need and
  determines which offers the best value and most
  benefit to the agency and its customers within
  acceptable cost and risk. Final investment
  analysis develops detailed plans and final
  requirements for the proposed investment program,
  including a lifecycle program baseline that
  establishes cost, schedule, performance,
  benefits, and risk-management boundaries for
  program execution.

Cost estimates are developed at this stage
of the IA process organization

• The Business Case Analysis (BCA) is at the heart
  of the decision process at FAA
• Business Case Analysis provides
  insights/decision-making structure to allocate
  FE funds

24
Explosion in Requirements and Motivation to do
Cost Estimating
GOVERNMENT COMMERCIAL
FAANeed to expand, enhance and professionalize cost estimating community DoD---by statute, every major program requires an independent cost estimate (ICE) Intelligence Community-- by statute, every major program requires an ICE NASA- ISSPO overruns, failed missions, Challenger tragedy, etc. Federal Acquisition Regulation (FAR) drives requirements for Cost Estimating System Competitive Environment forces need to understand and control costs Mergers and Acquisitions force focus on enterprise-wide consistency issues
25
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis
Cost Estimating Process Steps in the cost estimating process The hallmarks of a good cost estimate The FAA cost estimating process Work Breakdown Structures (WBS) Life Cycle Costs and the AMS
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
26
Understand the Proposed Program and Existing
Estimates
1.0 Understand the Proposed Program and
Existing Estimates
3.0 Collect Data for Each WBS Element
5.0 Time-Phase the Life Cycle Costs
in Then-Year Dollars
7.0 Conduct External Review and Coordinate
2.0 Cost Analysis Planning
2.0 Cost Analysis Planning
2.0 Cost Analysis Planning
4.0 Complete Cost Estimate
6.0 Conduct Internal Review
2.0 Cost Analysis Planning
2.0 Cost Analysis Planning
2.0 Cost Analysis Planning
8.0 Complete Final Reports and Documents
2.0 Cost Analysis Planning
As with any scientific undertaking, there is a
repeatable process at the core
From FAA Standard Cost Estimation Methodology,
April 2003, v.1
27
The hallmarks of a good cost estimate

• Good Cost estimating Practices
• Anchored in historical program performance
• Reflects future process and design improvements
• Understood by program and business leaders
• Cost leadership provides
• Confidence in cost estimates
• Understanding of financial issues and risks
• Major Attributes of Credible Cost Estimates
• Requirements Driven programmatic and system
  requirements documented
• Well-defined content and risk areas technical
  basis for estimating methods
• Can be validated by independent means within
  estimating/modeling accuracy
• Traceable and Auditable can be re-created from
  basis of estimates

• Major Attributes of Credible Cost Estimating
  Process
• Consistent WBS fixed reference for requirements
  and performance
• Consistent ground rules assumptions for annual
  estimates
• Validated estimating methodologies technically
  based models correlated to empirical data
• Cost data collection in sufficient detail and
  structure to support cost model development
• Explainable to variety of audiences

This is the Basic Guidance to Follow as You
Choose Methodologies for Developing the Cost
Estimate
28
Specify Ground Rules and Assumptions

• Scope And Limitations
• Schedule
• Base/Year
• Time Phasing
• Inflation Indices
• Transition from FE to OM funding
• Participants, including Contractors
• Relationship to/dependencies on other programs

Q What is the difference between a Ground Rule
and an Assumption? Ground Rules--specified by the
sponsor or agency represent standard procedures
or practices may clarify the boundaries between
programs or services Assumptions--made by the
estimator or sponsor provide a basis for needed
inputs in the absence of firm data or analysis.
NOTE Good cost analysts assess how varying
assumptions affect the outcome (sensitivity
analysis)
29
Why Cost Estimates Fail

• Lack of clear definition
• Requirements are misstated
  or
  misunderstood
• Lack of accurate size
  
  and functionality data
• Full scope of the job not
  captured
  (e.g.- integration, COTS, CM,
  QA, documentation)
• Inexperienced personnel
• Contractor counts on ECPs to get well
• Program changes, but estimates are not revised

The cost estimators job is not only to provide a
number, but to communicate how good the numbers
are.
30
What is a Work Breakdown Structure?

• A Work Breakdown Structure (WBS) is a
  hierarchical tree of all activities necessary to
  produce a product, accomplish a task, or provide
  a service
• A WBS identifies work elements and relates them
  to each other and to the end-product, task, or
  service
• The FAA Standard WBS captures 100 of a projects
  activities over its life cycle
• Contractor labor, material, travel, fees, etc.
• In-house labor, travel, etc.
• Costs are estimated for individual elements of
  the FAA WBS
• Cost estimating methods must be selected for each
  element

31
FAA Standard WBS Structure (Version 4.1)
The latest official version of the FAA Standard
WBS 4.0, can be found at http//faa.fast.gov
Project Specific Activities
Pre-Project Activities
FAA Standard Work Breakdown Structure
4.0 Implementation
3.0 Solution Development
1.0 Mission Analysis
2.0 Investment Analysis
6.0 Disposition
5.0 In-Service Management
5.1 Preventive Maintenance/ Certification 5.2
Corrective Maintenance 5.3 Modifications 5.4
Maintenance Control 5.5 Technical Teaming 5.6
Watch Standing Coverage 5.7 Program Support 5.8
Logistics 5.9 In-Service Training 5.10 Second
Level Engineering 5.11 Infrastructure
Support 5.12 Flight Inspections SIAP
Development 5.13 System Performance
Assessment 5.14 System Operations 5.15 Travel To
And From Sites
4.1 Program Management 4.2 Engineering 4.3
Environmental, Occupational Safety and Health
Compliance 4.4 Site Selection and Acquisition 4.5
Construction 4.6 Site Preparation, Installation,
Test and Checkout 4.7 JAI/Commissioning/
Closeout 4.8 Telecommunications 4.9
Implementation Training
3.1 Program Management 3.2 System Engineering 3.3
HW/SW Design, Development and Production 3.4
Physical and Airspace Infrastructure Design and
Development 3.5 Test and Evaluation 3.6 Data and
Documentation 3.7 Logistics Support
1.1 Identify Projected Demand for Services 1.2
Identify Technological Opportunities 1.3 Identify
Projected Supply of Services 1.4 Mission Needs
Analysis and Assessment 1.5 Initial Requirements
Definition
6.1 Program Management 6.2 Decommissioning 6.3
Engineering 6.4 Environmental Activities 6.5
Dismantle/Removal 6.6 Site Restoration/Closeout
2.1 Initial Investment Decision 2.2 Final
Investment Decision 2.3 Rebaselining Decision
Legend
Decomposes to lower level WBS elements
32
Cost Terminology

• Recurring vs Nonrecurring costs
• Direct Costs vs Indirect Costs
• Fixed Costs vs Variable Costs
• Overhead Costs
• Sunk Costs
• Opportunity Costs
• Life Cycle Costs

Cost is not a uniquely defined term
33
Cost Terminology

• Recurring Costs are those costs that are
  repetitive and occur when a company produces
  similar goods or services on a continuing basis.
  A fixed cost that is paid on a repeatable basis
  is a recurring cost (i.e., rent). For example,
  for a company that provides architectural
  services, office space rental - which is a fixed
  cost - is also a recurring cost. Can be tied to
  Quantity Produced.
• Nonrecurring Costs are those costs that are not
  repetitive, even though the total expenditure may
  be cumulative over a relatively short period of
  time. Nonrecurring costs typically involve
  developing or establishing a capacity to operate.
  For example, the cost of purchasing real estate
  upon which a plant will be built is a
  nonrecurring cost, as the cost of constructing
  the plant itself. Cannot be tied to Quantity.

34
Cost Terminology

• Direct Costs are those costs that can be
  reasonably measured and allocated to a specific
  output/product or work activity
• Typical direct costs include the labor and
  material costs directly associated with a
  product, service, or construction activity
• Indirect Costs are those costs that are difficult
  to attribute or allocate to a specific output or
  work activity. Costs that involve too much
  effort to allocate directly to a specific output
  instead, they are allocated through a selected
  formula (i.e., proportional to direct labor hours
  or direct material dollars). Cannot be tied to a
  specific product.
• Typical indirect costs include the costs of
  common tools, general supplies, equipment
  maintenance

35
Cost Terminology

• Fixed Costs are those costs which are unaffected
  by changes in output quantity over a feasible
  range of operations for the available production
  capability
• Typical fixed costs include insurance and taxes
  on facilities, general management and
  administrative salaries, and interest costs on
  borrowed capital. These are Non-Recurring Costs.
• Variable Costs are those costs associated with
  production that vary with quantity of output.
  Variable costs are the primary costs that should
  be considered when making an economic analysis of
  a proposed change to an existing operation.
• Typical variable costs include material and
  labor these are Recurring Costs

36
Cost Terminology

• Overhead Costs consist of plant operating costs
  that are not direct labor or direct material
  costs. (Indirect costs, overhead, and burden are
  terms that are sometimes used interchangeably)
• Typical overhead costs include electricity,
  general repairs, property taxes, supervision
• Various methods are used to allocate overhead
  costs among products, services, or activities
• Most commonly used methods involve allocation in
  proportion to direct labor costs, direct labor
  hours, direct materials costs, the sum of direct
  labor and material costs, or machine hours (refer
  to example)

37
Cost Terminology

• Sunk Costs are those costs that have occurred in
  the past and have no relevance to estimates of
  future costs and revenues for alternative course
  of action
• Common to all alternatives, not part of
  prospective cash flows in the future
• Not used in cost analysis but required in OMB-300
• Opportunity Costs are those costs incurred
  because of the use of a limited resource. The
  opportunity to use that same resource to monetary
  advantage in an alternative use is foregone.
• The cost of the best rejected or foregone
  opportunity
• Often hidden or implied

38
Types of Cost Estimates

• Life Cycle Cost Estimate (LCCE) Cradle-to-grave
  estimate includes RD, production,
  operations/support, and disposal
• Independent Cost Estimate (ICE) LCCE of most
  likely cost developed by an independent
  organization
• Budget Estimate For inclusion in budget to
  support current activities
• Rough-Order-of-Magnitude (ROM) Pre design
  effort, with very little specific information
  about the project
• Analysis of Alternatives (AoA) Evaluates costs,
  benefits, advantages and disadvantages of
  different alternatives
• Activity Based Costing (ABC) Accounting
  methodology that assigns resources and overhead
  costs to activities, products and services to
  support making decisions about pricing,
  outsourcing, capital expenditures and operational
  efficiency

There are many types of cost estimates, each with
a different purpose. Know the purpose of the
cost estimate before you start
39
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection Sources and Collection Index numbers, Inflation. Then year vs. Constant year Exercise Personnel Costs
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
40
Data Sources

• Three types we want to collect
• Cost/Resource Data (QUANTITATIVE)
• Product - NAS System, Facilities
• Functional - Engineering, Program Management
• Activity General Maintenance, Depot Repair
• Catalog Prices
• Cost Reports (EVM)
• Technical Data (QUANTITATIVE)
• Size
• Performance
• Technology
• Programmatic Data (QUALITATIVE)
• The Acquisition Environment
• Acquisition Schedule

41
Problems with Data Sources

• Differences in categories/accounting
  methodologies among contractors
• Information in the wrong format
• Sometimes previous program managers did not buy
  data
• The Matching up or Integration Problem
• The Influence of Temporal Factors
• Manufacturing Methods
• Technological Changes
• Comparability Problems

42
Information and Data Sources

• Product teams and Offices
• FASThttp//fast.faa.gov/ams/non_index/investab.htm
  
• NASA/JSC http//www1.jsc.nasa.gov/bu2/inflation/na
  sa/inflateNASA.html
• Society of Cost Estimating and Analysis (SCEA)
  http//www.sceaonline.net/
• US Army Corps of Engineers, Directorate of Civil
  Works (http//www.hq.usace.army.mil/cemp/e/ec/PAX/
  paxtoc.htm)

43
Data Normalization

• One of the most challenging and perennial
  problems confronting the cost analyst is the
  identification and normalization of cost data
• The adjustment of actual cost to a uniform basis
  has two objectives
• Reduces the dispersion of the data points
  Consistency
• Expands the number of comparable data points
  Homogeneity
• Also, since historic cost data involves the
  purchasing of goods and services in different
  time periods, we need to know how to compare the
  dollar cost of goods and services in one period
  with the dollar cost of comparable items in
  another period

44
Data Normalization contd.

• Normalization provides consistent cost data by
  neutralizing the impacts of external influences
• The three broad Normalization topics are
• Inflation
• Content
• Quantity
• Normalization efforts involve adjustments for
• Technology changes
• Data collection differences
• Production methodology enhancements
• Design Improvements
• Inflation and deflation

45
Inflation

• The consistent rise in the price of a given
  market basket of goods produced by an economy
• A rise in the general price level of goods and
  services produced in an economy
• Measured by the rate of rise of some general
  product-price index in percent per year
  examples?
• Many different measures of inflation are required
  because prices do not rise evenly
• Similarly, FAA uses different measures as well,
  e.g., FAA personnel at 5

46
Normalization for Inflation

• We do most of our normalization to account for
  inflation
• If System X costs 1M today, how much will that
  same system cost five years from now?
• A reflection of the fact that a dollar spent
  today buys more than it will in the future but
  buys less than it did in the past
• the effects of inflation over time

Of all the topics discussed in cost analysis,
none will be encountered more frequently than
inflation
47
Year XX ? Year YY

• Locate page for applicable appropriation with any
  base year
• On that page, locate the raw inflation index for
  year XX in column headed RAW INDEX
• Divide the dollar amount by this index
• On the same page, locate the raw inflation index
  for year YY in the same column.\Multiply the
  result of (3) by this index

48
Inflation Normalization Exercises
System Average Unit Cost (M) Reference FY07
System 1 540.1 FY95
System 2 890.1 FY90
Personnel Costs 146.5 FY06
Personnel Costs 841.9 FY05
49
Inflation and Escalation Terms
Term Definition
Base Year A point of reference representing a fixed price level
Constant Dollar Money or prices expressed in terms of the purchasing power prevailing in a specified base year
Current Dollar Money or prices expressed in terms of values actually observed in the economy at any given time
Then-Year Dollar Total budget that includes a slice of inflation to cover escalation of expenditures over a multiyear period
Inflation Rate Percentage change in the price of an identical item from one period to another
Outlay Profile In percentage terms, the rate at which dollars in each appropriation are expected to be expended based on historical experience
Raw Inflation Index A number which represents the change in prices relative to a base year of 1.000
50
Normalization for Content

• Is there an apples-to-apples comparison?
• This is largely a problem of mapping different
  data sets

51
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques Analogies Parametric
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
52
How is Cost Estimating Done?Three Essential
Costing Techniques

• Analogy Its like one of these subjectively
  compares the new system with one or more existing
  similar systems for which there is accurate cost
  and technical data.
• Parametric This pattern holds sometimes known
  as the statistical method, this technique
  generates an estimate based on system performance
  or design characteristics. It uses a database of
  elements from similar systems. It differs from
  analogy in that it uses multiple systems and
  makes statistical inferences about the cost
  estimating relationships.
• Build-Up Its made up of these bottom-up
  method of cost analysis that is the most detailed
  of all the techniques and the most costly to
  implement. Each WBS element must be costed to
  build the cost estimate for the entire program.
• Expert Opinion The other methods are not
  available

Costing Techniques rely on statistical
properties, logical relationships, emotional
appeal, and they are based on historical data
53
Analogy Estimating Technique

• Cost Estimating Method by which we assume our new
  system will behave cost-wise like a similar
  historical system
• Define the new system in terms of
• Design or Physical Parameters
• Performance Characteristics
• Known Similar System(s)
• Tailor the WBS for the New and Historical System
• Map Historical System WBS to New System WBS so
  they look similar
• Obtain Data on Historic Systems Design,
  Performance and Cost

54
Analogy Its like one of these

• Attribute Old System New System
• Engine F-100 F-200
• Thrust 12,000 lbs 16,000 lbs
• Cost 5.2M ?
• Q What is the unit cost of the F-200?
• A 5.2M (16,000/12,000) 6.9M

Tip The mischief in analogy most often arises
in the adjustment. Why do we so readily believe
a linear relationship on weight which passes
through the origin?
Source Society of Cost Estimating and Analysis
(SCEA)
55
How to Develop an Analogy

• Using a known items value, apply quantified
  adjustments to that item which measure the
  differences when compared to the new
• This requires good actual data and someone to
  quantify the differences
• Recent historical data should be similar not only
  in performance characteristics, but also similar
  from the standpoint of manufacturing technology
• Questions to ask when assessing the relative
  differences between the old and the new item
• How much different is the new compared to the
  old?
• What portion of the old is just like the new?
• How many components will be exactly the same?
• What is the ratio of complexity between the two
  systems?

56
Utility of Analogy Technique

• Many new programs consist of modified or improved
  versions of existing components, combined in a
  new way to meet a new need
• In the analogy technique we break the new system
  down into components (usually via a WBS) that can
  be compared to similar existing components
• The basis for comparison can be in terms of
  capabilities, size, weight, reliability, material
  composition, or a less well-defined, but often
  used, term, complexity
• When production and development cost estimates
  are needed, the analogy technique offers several
  approaches
• Separate development and production estimates,
  each based on data related specifically to
  development and production
• Production estimates based on production data,
  then use historical ratio factors to estimate
  development costs

57
Parametric Approach

• Parameter in the sense of a characteristic
• Cost f(physical and performance
  characteristics)
• Estimating relationships using explanatory
  variables such as weight, power, speed,
  frequency, thrust are used to predict cost at a
  higher level of aggregation
• Procedure consists of statistically fitting a
  line or function to a set of related historical
  data and then substituting the appropriate
  parameter of the new system into the resulting
  equation
• Developed from a set of sample points which
  realistically reflect the typical delays,
  problems, mistakes, redirection, and changing
  characteristics that occur during development of
  a new system

58
Examples of CERs
Building Construction
CER VARIABLES

• Floor Space
• Numbers of Floors
• Schedule

Cost is a function of

• Type, e.g., Sedan,
• SUV
• Doors
• Passenger Seating
• Cylinders/Horsepower

Passenger Car
Cost is a function of

• Empty Weight
• Speed
• Useful Load
• Wing Area
• Power
• Range
• Schedule

Aircraft
Cost is a function of
59
Parametric Estimating This pattern holds

• A mathematical relationship between a parameter
  and cost (Also called Cost Estimating
  Relationshipor CER)
• Parameter may be physical, performance,
  operational, programmatic, or cost
• Uses multiple systems to develop relationship
• Allows statistical inferences to be made

60
Analyze data for likely statistical relationships

• Visual Data Analysis
• Identify underlying trends
• Scatter Diagrams provide insight into the nature
  of the relationship exhibited by the data
• Linear
• Log/Linear
• Log/Log

61
How to develop a parametric CER
62
Example Max Take Off Weight vs. Max Payload
Weight
MAVW 174.6 5.116 x Payload Weight
Where MAVW is Max Air Vehicle Weight
Predator

• Chart Key
• ? actual data
• ? estimated data

Hunter
Max Take Off Weight (lbs)
R2 adj 83
Outrider
Skyeye
Pioneer
Max Payload Weight (lbs)
63
Utility of Parametric Approach

• Requires an extensive data base of historic cost
  and performance data
• Assumes that historic cost relationships will
  continue to hold true
• Regression analysis is the fundamental tool of
  parametric cost estimation
• Excellent for use early in program life cycle
  before a detailed design exists
• Used as the design progresses to capture changes
• Good as a cross-check for other methods

Source Society of Cost Estimating and Analysis
(SCEA)
64
Engineering Build-Up Approach Its made up of
these

• A detailed, bottoms-up application of labor and
  material costs
• Many detailed estimates are summed together to
  form the total estimate
• Used when you know detailed product information
  at the lowest level
• Data intensive, time consuming
• Typically expenive to produce

Build-Up could also be called Engineering
Build-Up, Industrial Engineering (IE),
Catalog/Handbook
Source Society of Cost Estimating and Analysis
(SCEA)
65
Build-Up Advantages/Disadvantages

• Advantages
• Easy to see exactly what the estimate includes
• Can include Time and Motion Study of actual
  process
• Variance Factors based on historical data for a
  given program or a specific manufacturer
• Disadvantages
• Expensive and requires detailed data to be
  collected, maintained, and analyzed
• Detailed specifications required and changes must
  be reflected
• Small errors can be magnified
• Omissions are likely

66
Other Cost Estimating Methodologies

• Expert Opinion Delphi technique
• Little, if any, analytical basis
• Extrapolation from actuals
• For systems that have been in production for
  some time
• Accurate historical cost data exists
• Used after production has already begun in order
  to estimate the cost of continued production
• Usually needed after a major change in quantity
  or performance

67
Program Maturity Influences Cost Estimating
Techniques
68
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
69
Cost Risk What is the Probability of Achieving a
Specified Cost Target?
Risk due to economic factors, rate uncertainties,
cost estimating errors, statistical uncertainty
and the manifestation of performance/schedule
risks
70
What is Cost Risk?

• Cost risk is the probability that the estimated
  costs will differ from the actual costs
• Unless each component of the estimate is known
  with certainty, the point estimate represents
  only one of several possible outcomes
• Cost Risk Analysis quantifies the possible
  outcomes and their likelihood
• Risk-adjusted cost estimates incorporate
  historical patterns based on data and expert
  judgment

71
What is Cost Risk? contd.

• Potential causes of cost risk include
• Technical Challenges that Require Additional
  Effort to Achieve Success
• Schedule Constraints that Result in Inefficient
  Resource Consumption
• Inherent Imperfection of Cost Estimating Methods
• Changing Programmatic Requirements or
  Organizational Conditions
• Schedule Risk

72
Why Estimate Cost Risk?

• Every program has some degree of risk and
  uncertainty
• Decision makers want to know about the risks in
  advance so they can make risk-adjusted investment
  decisions
• Realistic baselines must include funding to
  mitigate the risks and uncertainties (and this
  funding must be considered part of program cost)
• Accounting for risks lessens the likelihood that
  rebaselining will be necessary (and lessens its
  extent if it becomes necessary)

From Building Realistic Baselines Through
Cost-Risk Analysis, February 2002
73
Identify Key Cost Variables

• Concurrent with identifying data sources and
  estimating methodologies, identify, assess, and
  understand the relationships and risk factors
  between the variables

• Cost Drivers
• Labor Rates
• Material Costs
• Inflation/Discount Rates
• Economic Conditions

• Schedule Requirements
• Aggressive or Relaxed?
• Stable or Fluid?
• Staffing Experienced or Novice Available?

• Technical Requirements
• Stable or Fluid Design?
• Mil Std or Commercial?
• Technology Mature or Emerging?
• Staffing Experienced or Novice Available?

74
Estimating Cost Risk

• The cost estimator creates a cost model that
  represents cost elements as uncertainty variables
  with a range of possible values
• To obtain a risk estimate, the cost estimator and
  subject matter experts (SMEs) evaluate the
  programs risk at the WBS level by
• Identifying the costs of risk mitigation
  activities (costs that should be included in the
  risk-adjusted estimate)
• Assigning the appropriate probability
  distribution shape (typically triangular) and
  input parameters of the uncertainty variables
• Typically, FAA cost estimators then use Crystal
  Ball, a Microsoft Excel add-in, to automate the
  process of estimating the effects of risks on
  program cost

The FAA defines risk-adjusted estimates at the
80 confidence level
75
Estimating Cost Risk contd.

• Cost estimators typically generate a single,
  point cost estimate
• Typically all programs, public- and
  private-sector experience some cost growth

Cost Estimating
The question is how do these uncertainties, or
risks, affect cost?
Sources of Risk and Uncertainty
Technical/ Performance
Schedule
76
Sensitivity Analysis

• A process for assessing how sensitive a cost
  estimate is to change by varying non-cost
  parameters
• Tool for determining effect of changes for
  decision-making
• Addresses uncertainty of requirements more than
  cost uncertainty
• Results may be used to provide a range of
  possible outcomes

A good estimating practice is to check if the
estimated cost is sensitive to variations
in assumed inputs!
77
Illustration Apply Triangular Cost Distribution
Subsystem 1
A triangular distribution is defined by 3 values
minimum, maximum and most likely
1 Standard Deviation
Probability
The most likely value is not necessarily the
point estimate
0.108
0.108
0.90
1.03
1.15
1.28
1.40
Best-Case/ Minimum Value
Worst-Case/ Maximum Value
1.0
1.1
Mean
Mode
1.08
Most Likely Value
Median
Illustration from E. Kujawski, Notes on the
Triangular Distribution. http//www.engr.sjsu.ed
u/kujawski/labs/triang_dist.ppt
78
What is Monte Carlo Analysis?

• It is a technique for combining distributions,
  and thereby propagating more than just summary
  statistics
• It uses random number generation, rather than
  analytic calculations
• It is increasingly popular due to the
  availability of high speed personal computers

79
Why Perform Monte Carlo Analysis?

• Combining distributions
• With more than two distributions, solving
  analytically is very difficult or in some
  situations, impossible
• Simple calculations lose information
• Mean ?? mean mean
• 95 ile ?? 95ile ? 95ile!
• Gets worse with 3 or more distributions

80
Monte Carlo Analysis

• Takes an equation
• example Risk probability ? consequence
• Instead of simple numbers, draws randomly from
  defined distributions
• Multiplies the two, stores the answer
• Repeats this over and over and over
• Then the set of results is displayed as a new,
  combined distribution

81
Using Crystal Ball for Cost Risk
Most Likely Estimate

• Crystal Ball uses Monte Carlo techniques to
  repeatedly calculate program costs, sampling
  values from the probability distributions for the
  uncertainty variables and using those values for
  the cost elements
• Based on collecting the results of thousands of
  scenario runs, Crystal Ball generates a
  probability distribution of predicted program
  costs

The FAA typically budgets to the predicted cost
that will not be exceeded 80 percent of the time
(the risk-adjusted value)
82
Cost Risk What is the Probability of Achieving a
Specified Cost Target?
83
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating Definition and Life Cycle Inherent difficulties Software Growth Software Development Software Maintenance Data Sources and collection lessons Commercial models
Financial and Economic Analysis
References
Next Course Overview
84
What is Software?

• Software usually refers to a set of advanced
  computer modules that allow the user to plan
  efficient surveys, organize and acquire satellite
  navigation data, verify and download data,
  process and analyze measurements, perform network
  adjustments, and report and archive the final
  results
• The program or instructions that tell the
  computer what to do
• A generic term for computer programs, including
  systems programs which operate the computer
  itself, and applications programs which control
  the particular task at hand
• The entire set of programs, procedures, and
  related documentation associated with a computer
  system

85
Software Life Cycle Phases and WBS

• Interface Definition
• Requirement Specification
• Op Concept Development
• Algorithm Development

• Requirements Analysis
• Architecture Development
• Design, Code, and Unit Test Software
• Integration and Testing

• CM
• QA
• Software Environment Readiness
• Test Benches
• Security and Net. Admin.

• Test Planning
• Test Development
• Integration and Testing
• Acceptance Testing

• Project Management
• Supplier Management
• Personnel Management
• Team Building

86
Software Development Cost Estimating Process

• Estimate Costs
• Total
• Time-phased

87
Measuring Software Size

• Key software measurement Size
• How big is the software application or system
  being evaluated?
• Current sizing methods
• Source lines of code (SLOC)
• Oldest and most widely used
• Large programs sized in KSLOC (thousands)
• Function points
• Established in late 1970s
• Estimates size based on user-defined functionality

88
Software Cost-Estimating Concerns

• Hardware Cost-Estimating Philosophy is not
  Applicable to Software
• Software Development Tasks Are All Nonrecurring
  Development
• All Research and Testing, No Production
• Software Cost Is Uniquely Personnel-intensive
  Even Within Same Company or Workgroup,
  Productivity May Vary As Much As 100 to 1 Among
  Programmers

89
Software Cost-Estimating Concerns contd.

• Software Requirements Cannot Be Fully Captured in
  Any Finite List True List of Requirements Is
  Virtually Infinite
• Software Engineers High Self-esteem or
  Traditional Optimism Underestimates How Much Code
  Is Needed
• Initial Delivered Code Often Performs
  Inadequately and Fundamental Modification Costs
  Are Prohibitive
• Hardware Deficiencies That Cannot Be Fixed During
  Later Stages of Project Are Circumvented by
  Re-tasking Software

90
Software Cost-Risk Experience

• Cost Histories of Software-Development Projects
  Show a Definite Trend Toward Significant
  Underestimation of Number of Lines of Code and
  Cost
• Aerospace Corp. Study Found Lines-of-Code Growth
  of about 150 for Space-Related Ground-System
  Software Projects
• Naval Center for Cost Analysis Found Average
  Lines-of-Code Growth of 63 for Software Projects
  of Various Types (http//www.ncca.navy.mil/softwar
  e/handbook/software.htm)
• Developer Productivity, Measured in Lines of Code
  per Developer-Month, is Typically Overestimated
• This Results in Cost Growth, Even if
  Lines-of-Code Estimate is Accurate
• Data Collected Over Time Appear to Show Some
  Productivity Improvement, but not Enough to
  Overcome Estimating Optimism

91
Worst-Case for Software Costs

• Its 8 (times the Contractor Estimate)
• Number of Lines of Code Grows by Factor of 2.5
• Contractors Usually Assume Programmer
  Productivity to Be 300 Lines per Developer-Month
• Unfortunately, History Shows That Productivity
  Inevitably Slips to 85 Lines per Developer-Month
  as Project Moves Forward, which is Equivalent to
  Cost-growth Factor of 300/85 3.5
• 2.5 3.5 8.75
• Worst-Case Multiple is Applied to WBS Elements
  that Require Estimation of Software Costs
• Cost Distribution Will Be Right-triangular
• L M, but H 8M

92
USAF Space-Related Ground-System Software
Development Costs

• Definitions
• High Complexity Majority of Package Involves
  Operating System Software, Interactive
  Operations, Real-Time Command and Control, and
  Tight Timing Constraints
• Low Complexity Majority of Package Involves Data
  Storage and Retrieval, String Manipulation,
  Mathematical Computations, Routine Diagnostics
• Mission Operations Software Costs (FY02)
• High Complexity 314 per Line of Code ? 108 one
  sigma
• Neither High nor Low 254 per Line of Code ? 81
  one sigma Low Complexity 201 per Line of Code ?
  55 one sigma
• Mission Support Software Costs (FY02)
• High Complexity 281 per Line of Code ? 81one
  sigma
• Neither High nor Low 227 per Line of Code ? 55
  one sigma Low Complexity 141 per Line of Code ?
  27 one sigma

93
Post-Development Software Costs

• Software Maintenance Costs
• Generally Maintenance is Assumed to be Required
  over a 10-Year Life Cycle
• Maintenance Budget is Typically Set at 100 to
  150 of Software Development Cost
• Annual Budget for Software Maintenance Ranges
  from 10 of Development Cost to 15 of
  Development Cost

94
Curriculum
Kickoff/Introductions
Role of Cost Estimating and Analysis
Cost Estimating Process
Data Sources and Collection
Cost Estimating Techniques
Sensitivity and Cost Risk
Software Cost Estimating
Financial and Economic Analysis
References
Next Course Overview
95
Economic Analysis
Examples
Manned vs. Unmanned System
New vs. Upgraded system
Fix or Repair Car
Buy or Rent House
96
Structure of Economic Analysis

• Economic Analysis--A systematic approach to the
  problem of resource allocation, comparing two or
  more alternatives in terms of cost and benefits.
• Objectives of the action being considered
• Specification of assumptions / constraints
• Identification of alternatives
• Listing of benefits for all feasible alternatives
• Cost estimates for each feasible alternative
• A ranking of alternatives in terms of costs
• and benefits
• Risk / uncertainty analysis
• Conclusions / recommendations

Structure of an Economic Analysis
Economic analysis examines and compares the
benefits, costs, and uncertainties of each
alternative determine the most cost effective
means of meeting the objective
97
Business Case Analysis ProcessReturn On
Investment (ROI)
ROI Metrics

• Return on Investment Analysis
• Break-Even Analysis
• Internal Rate of Return

98
Definition of the Objective

• Should clearly define and quantify the function
  to be accomplished
• Must be as objective as possible
• Should not assume a specific means of achieving
  the desired result
• Must be worded to reflect a totally unbiased
  point of view

99
Assumptions and Constraints

• Assumptions Explicit statements used to
  describe the present and future environment upon
  which the economic analysis is based
• Required Assumptions
• (1) Economic life
• the period of time over which benefits from an
  alternative are expected to accrue. Usually
  constrained by physical, mission or technological
  life
• (2) Period of Comparison (period of analysis)
• begins when money is spent on the first
  alternative requiring expenditure of funds and
  ends when the alternative with the longest
  economic life ceases to produce benefits
• There is one period of comparison for an economic
  analysis

100
Limits on Economic Life

• Physical life
• The estimated number of years that an asset can
  physically be used
• Mission life
• The estimated number of years over which the need
  for the asset is anticipated
• Technological life
• The estimated number of years a facility, piece
  of equipment, or automated information system
  will be used before it becomes obsolete due to
  changes in technology

101
Constraints

• External factors that limit alternatives to
  problem solutions
• Physical
• a fixed amount of space
• Time-Related
• a fixed deadline
• Financial
• a limited amount of resources
• Institutional
• policy or regulation
• Conditions that are beyond the control of the
  decision maker that limit the number of available
  alternatives

102
List of Alternatives

• All reasonable ways of satisfying the objective
  should be documented and discussed
• At a minimum, each of the following alternatives
  must be considered
• Status quo
• the existing way of meeting the objective must be
  included even if its considered to be infeasible
• Modification