Army Cost Review Board Office

1
Cost Analysis for Non-Costers

• Army Cost Review Board Office
• Mort Anvari Mort.Anvari_at_US.Army.mil
• Steve Loftus Steve.Loftus_at_US.Army.mil

2
HQ Department of the Army
Secretary of the Army
Chief of Staff, Army
Under Secretary of the Army
Vice Chief of Staff, Army
Assistant Secretaries of the Army
Financial Management Comptroller
Acquisition, Logistics Technology)
Manpower Reserve Affairs
Installations Environment
Civil Works
Cost Economics
Budget
G8
3
DASA, Cost Economics
Director Programs Strategy
Director Acquisition Costing
Chief, Cost Review Board
Installation Costing
Cost Management
Unit Mission Costing
Weapon Systems
C4ISR Costing
Cost Policy Research
4
Cost Review Board (CRB)

• The ASA(FMC) Formed the Army Cost Review Board
  to review Cost Estimates for Major Weapon
  Information Systems
• Created in Response to the Need for
• A Comprehensive Army Cost Position, Acceptable to
  Acquisition Financial Management Communities
  Supports PPBES

The CRB Reviews Major Weapon Information
Systems at Their Critical Acquisition Decision
Points. All ACAT I Programs Programs of
Special Interest Must Have a Recommended ACP
Briefed to the CRB.
5
Cost Analysis Requirement
MS 0
MS I
MS II
MS III

O L D
Program Definition Risk Reduction
Engineering Manufacturing Development (EMD)
Production, Fielding/ Deployment
Concept Exploration
Cost Estimates Needed
Concept Exploration
Component Advanced Development
System Integration
System Demo
Production Readiness LRIP
Rate Production Deployment
N E W

Support
Concept Technology Development
System Development Demonstration
Production Deployment
Pre-Systems Acquisition
Systems Acquisition (Engineering and
Manufacturing Development, Demonstration, LRIP
Production)
Sustainment
6
Cost Estimating Process
Final Document
Data Collect Normalize
Estimate Formulation
Definition Planning
Review/ Presentation
DOCUMENTATION
7
Cost Analysis Model
Data/CERs LCC Estimates Cost Drivers
Develop Cost EstimateStructure and WBS
EstablishGround Rules and Assumptions
Prepare Cost Estimatesfor Each Element
Test TotalSystem Estimate
Prepare Documentation
Start
CompileData Base/ CERs/Models
Engineering Analogy Parametrics Expert Opinion
Reasonableness SensitivityAnalysis Cost-RiskAsse
ssment
8
Situation

• You have just been tasked to develop a cost
  estimate, that is, a professional opinion about
  the cost of an item, a service or a thing.
• Lets discuss a process for organizing and
  developing this estimate.

9
Definition and Planning

• Influences the success of the estimate
• Understanding the requirements and how you
  approach the process will establish the
  guidelines and procedures for the estimate.
• Ask lots of questionsThey help you understand
  the requirement.

10
Questions

• Why is this cost estimate needed?
• What decisions are pending on the results of this
  estimate?
• Will the estimate be briefed and to whom?
• Will the results be incorporated into some
  document?
• What does the recipient expect to have included
  or excluded?
• What excursions or variations from the baseline
  are anticipated?

11
Questions Continued

• What are the program and funding constraints
  especially if the program is a Joint Program?
• What are the time constraints for this estimate?
• What is the acquisition phase of the program?
• Is the program definition mature?
• Does technology exist today to design, develop,
  test and manufacture the system?
• What is the interrelationship with other systems?
• Are there previous contracts? How many? What
  type?
• How have the contractors performed to date?

12
Definition and PlanningKnow Purpose of the
Estimate

• Main purposes of estimates
• Budget Formulation
• Comparative Studies
• Source Selection

13
Purpose of EstimateBudget Formulation Estimates

• Program Office Estimate (POE)
• Component Cost Analysis (CCA)
• Independent Cost Estimate (ICE)
• What-if exercises
• Rough Order of Magnitude (ROM)
• Should Cost Estimates
• CAIV

14
Purpose of EstimateComparative Studies Estimates

• Making cost benefit comparisons between
  alternatives
• Economic Analysis (EA)
• Analysis of Alternatives (AoA)
• Force Structure
• Trade-off Studies
• Source Selection
• Prioritization

15
Definition and PlanningDefining the System

• Adequate description of the technical and program
  characteristics of the system
• What are the physical and performance
  characteristics?
• What are the development, production, and
  deployment schedules?
• How many systems are to be produced?
• How will the systems be supported contract,
  in-house, two or three levels of maintenance?

16
Defining the System Integrated System Schedule
17
Defining the System Work Breakdown Structure
(WBS)

• WBS defined in MIL-HDBK-881 as
• product-oriented breakdown of hardware,
  software, services, data and facilities that
  define the system.
• WBS breaks a total job down into manageable
  pieces portrays the way work is to be done.
• WBS displays a companys reporting structure.
• Program managers may cite MIL-HDBK-881 for
  guidance only in contract solicitations.

18
MIL-Handbook-881 (WBS)
19
Defining the System Cost Element Structure (CES)

• 1.0 Research Development Test Evaluation
  (RDTE)
• 2.0 Production
• 3.0 Military Construction (MILCON)
• 4.0 Military Pay and Allowances (MPA)
• 5.0 Operating and Maintenance Army (OMA)

20
Defining the System COST ELEMENT STRUCTURE
1.0 RDTE
CES ELEMENT FY00CM
TYM 1.01 DEV. ENG.
1.02
PEP 1.03 DEV. TOOL.
1.04 PROTO MFG.
1.05 SEPM 1.06
SYS TE
1.07 TRAINING
1.08 DATA
1.09
SUPP EQUIP. 1.10 DEV. FACILITIES
1.11 OTHER
39.039M 38.260M 0.408
0.386 0.457 0.450
110.421 107.724 78.266
76.363 11.112 10.927
1.989 1.954 3.439
3.413 4.897
4.758 0.0 0.0 0.968
0.928
21
Defining the System COST ELEMENT STRUCTURE
2.0 PROCUREMENT
CES ELEMENT
FY00CM TYM 2.01 NON REC PROD.
2.02 REC. PROD
2.03 ENG. CHG 2.04
SEPM 2.05 SYS TE 2.06
TRAINING 2.07 DATA
2.08
SUPP. EQUIP. 2.09 OPER./SITE/ACT.
2.10
FIELDING
2.11 TRAIN. AMMO/MSLS
2.12 WAR
RESV. 2.13 MODS
2.14
OTHER
16.110M 16.583M 1,169.348
1,312.377 0.0
0.0 116.637 132.258
12.564 14.437 28.880
31.499 2.073 2.300
146.460 158.304 0.0
0.0 89.525
101.635 59.001
79.482 0.0
0.0 236.619 280.729 51.739
64.129
22
Defining the System COST ELEMENT STRUCTURE 3.0
MILCON
CES ELEMENT
FY00CM TYM 3.01
DEVELOP. CONSTRUCTION 3.02 PRODUCT.
CONSTRUCTION
3.03 OPERATION/SITE
ACTIVATION 3.04 OTHER MILCON
23
Defining the System COST ELEMENT STRUCTURE 4.0
Military PayAllowances (MPA)
CES ELEMENT
FY00CM TYM 4.01 CREW 4.02
MAINTENANCE
4.03 SYSTEM SPECIFIC
SUPPORT 4.04 SEPM 4.05 REPLACEMENT
PERSONNEL 4.06 OTHER MPA
24
Defining the System COST ELEMENT STRUCTURE 5.0
Operating Maintenance Army (OMA)
CES ELEMENT
FY00CM TYM 5.01 FIELD MAINT.,
CIV LABOR 5.02 SYS. SPECIFIC BASE OPS
5.03 REPLENISHMENT DLRs 5.04
REPLEN. CONSUMMABLES 5.05 POL 5.06
END ITEM MAINTENANCE 5.07
TRANSPORTATION
5.08
SOFTWARE 5.09 SEPM
5.10 TRAINING

5.11 OTHER OMA


25
Defining the System Cost Analysis Requirements
Description (CARD)

• Source of a systems description
• Describes important features
• Is provided to other groups preparing cost
  estimates
• Helps ensure all groups are costing out the same
  program.
• Prepared by program office approved by
• DoD Component Program Executive Officer

26
Defining the System Cost Analysis Requirements
Description (CARD) Continued
27
Defining the System CARD Continued
28
Definition and PlanningGround Rules Assumptions

• State the conditions which must take place in
  order for the estimate to be valid
• Ground rules and assumptions must be documented
  since changes in these areas provide an audit
  trail for changes in the cost estimate.

29
Data Collection and Analysis

• Collection and analysis represent a significant
  amount of the overall estimating task in terms of
  time. The analysis will include decisions on
  what programs to include in the data set to
  whether to truncate lot data on a program for
  which you are calculating a learning curve.
• Document data in your analysis, and any
  assumptions you make

30
Data Collection and Analysis

• The direction we take in collecting historical
  data will be determined by our choice of
  estimating methodologies.
• This step may also dictate a change in estimating
  approach due to the availability or
  non-availability of certain data.
• Data collection is not limited to cost data. We
  must also collect technical and program data if
  we want the total picture of the historical
  systems. This will help us ensure the
  comparability of the systems that we are
  collecting data on with the system we are
  estimating.

31
Data Collection and Analysis Most Difficult Task
in Cost Estimating

• Data Sources
• Data Types Cost/Resource, Technical, Program
• Categories Primary, Secondary
• Data Problems
• Wrong Format Matching up Definition
• Temporal Factors - comparability
• Normalization
• Data Location

32
Army Tools for Total Ownership Cost Estimating
Life Cycle Cost Management Tools
33
Definition and PlanningSelect the Estimating
Approach

• Techniques available
• Analogy
• Parametric
• Engineering
• Extrapolation
• Expert Opinion
• Select the technique that is most applicable to a
  specific WBS element

34
Definition and PlanningEstimating Methods

• Analogy
• Basic Comparison
• Factors
• Parametric
• Regression Analysis
• Engineering
• Detailed
• Expert Opinion
• Committee
• Delphi

35
Life Cycle Cost Estimates

• Cost estimates based on confidence intervals
• Parametric analysis based on similar systems and
  similar attributes (regression)
• Engineering data based on reliability projections
  used for bottoms-up estimate
• Actual system costs used to extrapolate future
  system costs
• Cost estimate revised every two years after
  production
• Weights associated with non-actuals decrease as
  system matures
• MS B is a true hard stop for systems

36
Cost Estimating MethodsAnalogy Method

• Based on direct comparison with historical
  information of similar existing activities,
  systems, or components.
• Compares new system with one or more existing
  similar systems where there is accurate cost and
  technical data.
• Analyst must show validity of comparison.

37
Cost Estimating MethodsAnalogy Method

• Based on known costs of a similar program
• Adjustments for complexity, technical, physical
• Strengths
• Based on representative experience
• Less time consuming than others
• Can be used as a check on other techniques
• Weaknesses
• Small sample size
• Heavy reliance on judgment
• Sometimes difficult to identify analogy and
  associated costs

38
Analogy Estimating with Factors
Cost(New) Cost(Old) x Adjustment Factor
Element Old Sys1 Old Sys2 Old Sys3 New Sys
Airframe 500/lb 250/lb 750/lb 1.25S1
Engine 2M/Unit 3M/Unit 5M/Unit .8S3
Avionics 3K/lb 2K/lb 4K/lb 1.0S2
Payload 6M/Unit 8M/Unit 7M/Unit .65S1
39
Cost Estimating MethodsParametric Method

• Known as Statistical Method or Top Down Method
• Relates cost to physical attributes or
  performance characteristics
• Uses database of elements from similar systems
• Uses multiple systems
• Most beneficial in earlier stages of the system
  or project life cycle

40
Cost Estimating MethodsParametric Method

• Statistical relationships between cost and
  physical or performance parameters of past
  systems.
• Strengths
• Captures major portion of cost
• Quick what if type estimates
• Weaknesses
• Less detailed
• Getting accurate data

41
Cost Estimating MethodsParametric
Method(Extrapolation)

• Use historical values to establish a trend for
  the future.
• Example problem Given the actual productivity
  and labor rates in the given table. How much
  will it take to complete a 3-year software
  development project of 10K lines of code, if 50
  is completed in the second year and 25 is
  completed in first and third years?

42
Cost Estimating MethodsEngineering Method

• Known as bottom up method
• Requires extensive knowledge of system
  characteristics
• Divide into segments estimate costs for each
  segment
• Combine segments plus integration cost
• Uses a combination of cost estimating methods
• Detailed knowledge of new technologies may not be
  available.

43
Cost Estimating MethodsEngineering Method

• Strengths
• Detailed
• Best when long stable production process
• Weaknesses
• Requires a lot of time
• Cost
• Cannot be used until system well defined

44
Cost Estimating MethodsExpert Opinion Method

• Subjective judgment of an experienced individual
  or group
• Use if time does not permit a more thorough
  analysis
• Document source(s) of opinion of experts
• List attributes of the source(s)
• Example Delphi Technique

45
Cost Estimating MethodsExpert Opinion Method

• Consulting with one or more experts who use their
  knowledge and experience to arrive at an estimate
• Group techniques include
• Consensus (Committee)
• Delphi
• Strengths and weaknesses

46
Expert Opinion MethodDelphi Technique

• Query expert opinion from group
• Seek information from each expert
• Summarize the results
• Send report to each expert
• Gather second opinion after each individual
  reviews report
• Summarize results
• Iterative process continues until the experts
  reach a consensus, or near-consensus.

47
Expert Opinion Method Example
Labor type Hours Needed Hourly Rate of Total Hrs rate
Senior Engineer 1000 13 10.5 1.37
Design Engineer 3000 11 31.6 3.48
Tool Die 500 11 5.3 .58
Machinist 5000 9 52.6 4.73
Totals 9500 10.16
48
Learning Curve Theory
As the quantity of a product produced doubles,
the man-hours- per-unit expended to produce the
product decreases at a fixed rate or constant
percentage (usually 10 to 15).
49
Learning Curve Theory Factors Contributing to
Efficiency

• Job familiarization by both production workers
  and supervisory personnel.
• Changes in product design which do not
  materially affect the product, but result in
  increased ease and speed of production.
• Changes in tooling, machinery, and equipment
  which simplify or speed up the production
  process.
• Improved production planning and scheduling, and
  improvements in production techniques and
  operational methods.
• Improvements in shop organization, engineering
  coordination and liaison.
• Improvements in the handling and flow of
  materials, and in the materials and parts supply
  systems

50
Learning Curve Theory
The table is based on the assumption that the
first unit required 100 person-hours to produce.
The table indicates a constant rate of reduction
of 20 for each doubling of the unit number the
value of the second and each succeeding item in
the table is 80 of the value of the preceding
item.
51
Learning Curve Theory
80 Unit Curve on Arithmetic Paper
                                                  
              
52
Learning Curve Theory
53
Learning Curve TheoryUses

• Evaluating contract production costs.
• Assessing impact of production interruptions,
  product changes and production rate change.
• Rate of improvement experienced by a particular
  contractor on a prior product may be indicative
  of rate of improvement expected on new product
  of similar size, complexity, and construction.
• Improvement curve pattern experienced in the
  production of past item can be extended to
  calculate costs of future items.

54
Estimate Formulation

• We have defined our tasks, planned the estimate,
  assigned cost responsibilities, and performed
  data collection and analysis.
• Here we apply our estimating methodologies and
  tools develop the factors, analogies, CERs, and
  learning curves.
• We will aggregate the various cost elements into
  development, production, and OS estimates,
  fiscally spread the costs, and apply inflation.

55
Cost Estimating MethodsLaying Out the Estimating
Approach

• Aircraft System sum of level II elements
  (cross-check with analogy)
• Air Vehicle sum of level III elements
• Airframe CERs
• Air Vehicle Software Expert Opinion
• Propulsion CERs
• Avionics Analogy
• Armament Catalog Price
• System Eng/Program Mgt Factor of Air Vehicle
• System Test Evaluation Factor of Air Vehicle
• Training Factor of Air Vehicle
• Data Factor of Air Vehicle
• Peculiar Support Equipment Factor of Air
  Vehicle
• Initial Spares Factor of Air Vehicle

56
Review and Presentation

• We want to ensure that the estimate is
  reasonable, realistic, and complete.
• Reasonableness addresses areas such as using
  appropriate and acceptable methodologies
  presenting methodologies systematically the use
  of relevant data and ensuring that assumptions
  are valid and clearly stated.

57
Review and Presentation (Continued)

• The realism test checks to see if the assumptions
  and ground rules are consistent with the
  statement of work and if the costs are in line
  with historical data.
• We evaluate completeness by determining whether
  the estimate includes all the work stated in the
  request for proposal and whether the costs are
  traceable and reconcilable.

58
Risk and Uncertainty Analysis

• Risk Analysis Approaches

• Detailed Network Stochastic

• Discrete Technical, Schedule, and Estimating
  Risks

• Detailed Monte Carlo Simulation (each WBS)

Effort

• Bottom Line Monte Carlo Simulation

• Add a Risk Factor/Percentage

Detail
59
Risk and Uncertainty Analysis

• Probability Distributions

WBS Cost Distribution
Total Cost
60
Risk and Uncertainty Analysis

• Probability Density Function

61
Risk and Uncertainty Analysis

• Cumulative Distribution Functions

62
Reasons for Risk
63
Cost Schedule Curve
Fixed Cost Technology Outdate
Parallel Effort More ECP Less Mature Design
Life Cycle Cost
Typical
Min
Optimal
Development Schedule
64
Development Schedule
Mean Cycle Time to IOC
DoD Pre-1992 Starts 132 months (11
years) Post-1992 Starts 89 months (7.4
years) on-going F-22
216 months (18 years) IOC 2004 Comanche
264 months (22 years) IOC
2006 Commercial Boeing 777 54
months (4.5 Years)
65
Schedule Goals

• Commercial Drivers
• Technology Drives Schedule
• Constraint Schedule
• Goal is ROI Maximization

Defense
Cost

• DoD Drivers
• Funding Drives Schedule
• Unconstrained Schedule
• Goal is Cost Reduction

Commercial
Development Schedule
66
DoD Program Schedule Drivers
Funding Allocation
Cost
Acquisition Process
Program Execution
Cycle Time Reduction Goal
Development Schedule
67
Final Documentation

• Provide the means for other analysts to get the
  same results that we have in our cost estimate.
• Providing good directions and a clear trail to
  follow are essential in having an estimate that
  can be replicated.
• Provide step-by-step documentation of the
  methodologies, supporting data, ground rules, and
  assumptions, equations, examples, etc.,
• Ability to interpret or evaluate someone elses
  cost documentation is as important as ability to
  prepare good cost documentation.

68
Life Cycle Management Changes
Current State
ICD
Multiple bill payer drills to offset actual costs

Defined, Refined, Fairly Accurate
Program forEquipment
ST
RDTE
Initial Cost Estimate
Revised Cost Estimate
Disposal
Revised Cost Estimate
MS A
MS B
MS C
SRR
Procurement
OPA
MCA
Construction
MPA
Personnel
Training
OS Cost Estimate
Program forOS Costs
Operations
OMA/MPA
Support
Only compete procurement costs against Army
Investment Strategy over the EPP. Impact of
sustainment on Army program ignored.
Develop SSP
Unbalanced process that emphasizes procurement
over sustainment Large unplanned costs
Future State
ICD/DOTMLPF Change Request
Periodic Cost Estimate
Develop Supportability Sustainability Plan
Program forEquipment
ST
RDTE
Revised Cost Estimate
Disposal
Initial Cost Estimate
Revised Cost Estimate
MS A
MS B
SRR
OPA
Procurement
FOC
MCA
Construction
Capabilities Based Programming
Personnel
MPA
Plan for Sustainment
Training
Compete total cost against Army TOA extended over
EPP.
Operations
OMA
POM for Sustainment
Support
Balanced process that considers investment and
sustainment Informed decision makers
69
CAIV Process

Set realistic but aggressive cost objectives
early in acquisition program

Manage risks

Track progress using appropriate metrics

Motivate Government and industry
managers to achieve program objectives

Incorporate incentives to reduce OS costs for
fielded systems
70
Cost As An Independent Variable
(CAIV)

• Best time to reduce cost is early in the process.
• Involves the stakeholders in the process.
• Cost tradeoffs must be addressed early in the
  acquisition process and embedded in program
  requirement documents, Request For Proposals
  (RFPs), contract provisions, and source selection
  process.

71
Cost Analysis Domain
Types of Cost Studies
Performance Based and Design Based Cost Models
Currently do not Exist
72
Analysis of Alternatives

Set realistic, aggressive cost objectives
early in development

Manage risks

Track progress with appropriate metrics

Motivate government/industry managers to achieve
program objectives

Incorporate incentives to reduce OS costs for
fielded systems.
73
The Coster's Challenge
Cost Analysis Art or Science?