Performance Based Logistics PBL Metrics

1
Performance Based Logistics (PBL) Metrics

• PBL Metrics Sub-IPT Member View
• Mark Weston-Dawkes CPL
• 26 Sept 2006
• Presentation to GS SOLE Chapter

2
Rationale Reaction to PBL Metrics
3
Thoughts from the Past.Measurement, Incentives,
Agreements

• "In physical science the first essential step in
  the direction of learning any subject is to find
  principles of numerical reckoning and practicable
  methods for measuring some quality connected with
  it. I often say that when you can measure what
  you are speaking about, and express it in
  numbers, you know something about it but when
  you cannot measure it, when you cannot express it
  in numbers, your knowledge is of a meagre and
  unsatisfactory kind it may be the beginning of
  knowledge, but you have scarcely in your thoughts
  advanced to the state of Science, whatever the
  matter may be.
• Lord Kelvin ( Sir William Thomson ) PLA, vol.
  1, "Electrical Units of Measurement", 1883-05-03
  
• It is not from the benevolence of the butcher,
  the brewer, or the baker that we expect our
  dinner, but from their regard to their own
  interest. Adam Smith
• Man is an animal that makes bargains no other
  animal does this - no dog exchanges bones with
  another. Adam Smith

4
PBL Definition and Guidance

• The Office of the Secretary of Defense (OSD) has
  defined PBL as a strategy for weapon system
  product support that employs the purchase of
  support as an integrated performance package
  designed to optimize system readiness. It meets
  performance goals for a weapon system through a
  support structure based on performance agreements
  with clear lines of authority and
  responsibility.
• Army Guidance AR 70-1
• The extent and level of detail to which PBL will
  be applied will be based on a business case
  analysis (BCA).
• PBL will be implemented on all ACAT I and ACAT
  II programs, where PBL is proven to be
  economically and operationally feasible. PBL
  will be applied to ACAT III programs at the
  discretion of the PM, with approval from HQDA.
• When a PBL strategy is used, the PM will
  designate a product support integrator (PSI) to
  integrate all product support for the system.
• PBL must be tailored to individual system
  needs.
• PBL will be executed through performance based
  agreements (PBAs) with, at a minimum, the using
  MACOM(s) and the product support integrator
  (PSI).
• Army Guidance includes constraints with theme of
  need to integrate with standard Army systems and
  structures

Mark Weston-Dawkes Sept 2006
5
PBL Metrics Importance

• PBL Concept Links Outcomes, Agreements
  Incentives
• PBL Metrics are an integrating performance
  thread.
• Use of at least one of the overarching metrics
  is preferred. However if a sub-element is better
  suited for use in evaluating a PBL strategy for a
  particular system, use of that element is
  acceptable as well. The metric or sub-element
  selected should be able to measure success or
  failure of the Product Support Providers (PSPs)
  and/or the Product Support Integrator (PSI).
  When properly applied, the metrics will ensure
  successful evaluation of Army PBL efforts. From
  Draft HQDA PBL Metrics Policy Letter
• PBL Business Case Analysis rests on performance
  proposition being worth cost. Performance
  proposition infers Metrics.
• PSI must be able to manage PSP performance
  through Metrics
• Metrics must be identified, defined and
  incentivized in PBL PBAs.
• Otherwise---the result is either a performance
  strategy or a logistics strategy but not a
  Performance Based Logistics Strategy.

Mark Weston-Dawkes July 2006
6
PBL Overarching Metrics

• Logistics Response Time (LRT)
• How responsive is Sustainment System
• Logistics Footprint
• What is my Logistics burden to deploy, move
  sustain?
• Cost Per Unit Usage
• What does it cost to operate, maintain
  sustain?
• Operational Availability
• What is my Materiel Readiness?
• Mission Reliability
• How Reliable will this be in use?
• Many Sub Elements to these Metrics!!!
• Note these are my characterizations see PBL
  Metrics Guide for definitions and formulas

Mark Weston-Dawkes /sept 06
7
PBL Metrics Implementation

• Implementation Concerns (From Draft Policy Guide)
• Early Planning for Data Collection
• Data Sources and Limitations
• Burden on Field Units (Metrics Implementation
  burdens)
• Automated Sources and Recording
• Negative Analysis (what could go wrong in Metrics
  process)
• Policy and Doctrine (Metrics Implementation vs
  Policy)
• Review Boards (Data)
• Early Determination of System Definition and
  Usage Factors
• Accounting for Modeling and Configuration
  Differences
• Sample Data versus Total Population
• Funding Resources
• Scope of Implementation Concerns vary by System
  and Metric !!!!

Mark Weston-Dawkes July 2006
8
Ao DataDefinition and Collection Challenge
Operational Availability (Ao) Ao MTBM / (MTBM
MDT)
MTBM
MDT
Maintenance Events
MTTR
MALDT
Mission Duration / Measurement Base
Maintenance Time
M-ADT
M-LDT
Care Service
Conditions Qualifications
Organization / Time (1)
Organization / Time (1) (eg PLL)
Organization / Time (2)
Organization / Time (2)
Organization / Time (N)
Organization / Time (N)
Mark Weston-Dawkes July 2006
9
PBL Overarching MetricsChallenges

• Operational Availability
• Complex Metric
• Many Different PSP players.
• Operational/Mission Reliability
• Dont confuse with Readiness
• Data Collection in Field vs Test Environment
• Logistics Footprint
• Optimal vs Minimal
• Whats behind a larger Footprint (e.g Mission vs
  Reliability)
• Cost Per Unit Usage
• Attributable Costs vs Common Items vs Overhead
• Whats behind higher costs? (e.g who controls?)
• Logistics Response Time
• Which part of Logistics/Supply Chain?
• Consider all actions for all situations measured

Mark Weston-Dawkes July 2006
10
PBL Metrics Implementation Issues and Cautions

• PBL must not become merely a matter of least cost
  against minimum readiness criteria.
• It is important to be able to distinguish the
  nature of each metrics requirements for deployed
  versus those engaged in normal peacetime
  training.
• When considering and overall Logistics Strategy
  Warfighter benefits in responsive and robust
  reliability design and support approach are not
  always quantifiable in metrics. Metrics are
  important but when the benefits are not always
  quantifiable, logisticians should be ready to use
  analytical judgment in determining strategy.
• Dont confuse PSI and PSP in Metrics
  Implementation.
• Dont underestimate the difficulty of Field Data
  Collection or potential incompatibility of STAMIS
  Data to PBA Metric usage.
• Need PBAs with all Key contributors/participants
  in Metrics (Multiple PSPs for Single Metric?)
• Dont promise more in your PBL Strategy and BCA
  than there is the practical ability to deliver in
  Implementation.

Mark Weston-Dawkes July 2006
11
PBL MetricsStuff happens in Field!!

• Operating Timeaccuracy?
• Maintenance Time-accuracy?
• Logistics DelaysPrompt Posting Receipts
• PMCS Performance
• Operating Conditions
• Accidents

• OopsI just thought it was broke!
• Technical Manual and Training Discipline
• Operator and Maintainer Headspace
• COMBAT
• Acts of God

How do we measure and control these?
Mark Weston--Dawkes Sept 06
12
Metrics Implementation No Easy Thing!!
Mark Weston-Dawkes July 2006
13
Design Impact on MetricsPBL or Not Dont forget
Logistics Design
Good Design Practice helps drive Logistics
Performance !
Mark Weston-Dawkes Sept 06
14
PBL Metrics Concluding Thoughts

• Plan for Metrics Strategy Upfront Iteratively
• Dont Underestimate Complexity of PBL Metrics
  Implementation
• Understand the PBL Issues as relates to Metrics.
• PBL is still very new for Tactical Army in real
  practice.
• Be very careful of any product that promises
  trouble free Metrics measurement.
• For a long time it puzzled me how something so
  expensive, so leading edge, could be so useless,
  and then it occurred to me that a computer is a
  stupid machine with the ability to do incredibly
  smart things, while computer programmers are
  smart people with the ability to do incredibly
  stupid things. They are, in short, a dangerously
  perfect match.
• Bill Bryson Im a Stranger Here Myself ,
  Chapter Lost in Cyberland, p168 .

Mark Weston-Dawkes July 2006
15
PBL Metrics Backup Slides

• DOD Army Definitions
• Army PBL Policy AR 70-1
• PBL PBL Metrics Connection
• PBL Overarching Metrics Definitions and Formulas

16
DOD PBL

• Background
• On September 30, 2001, the Quadrennial Defense
  Review (QDR) mandated implementation of
  Performance-Based Logistics (PBL) and modern
  business systems with appropriate metrics to
  compress the supply chain, eliminate
  non-value-added steps, and improve readiness for
  major weapons systems and commodities. PBL
  delineates outcome performance goals of weapon
  systems, ensures that responsibilities are
  assigned, provides incentives for attaining these
  goals and facilitates the overall life cycle
  management of system reliability, supportability,
  and total ownership costs.
• The Office of the Secretary of Defense (OSD) has
  defined PBL as a strategy for weapon system
  product support that employs the purchase of
  support as an integrated performance package
  designed to optimize system readiness. It meets
  performance goals for a weapon system through a
  support structure based on performance agreements
  with clear lines of authority and
  responsibility.
• DOD Acquisition Policy
• The DoD Directive 5000.1 states The PM shall
  develop and implement performance-based logistics
  strategies that optimize total system
  availability while minimizing cost and logistics
  footprint. Sustainment strategies shall include
  the best use of public and private sector
  capabilities through government/industry
  partnering initiatives, in accordance with
  statutory requirements
• DOD 5000.2 3.9.2.6 States Sustainment
  strategies shall evolve and be refined throughout
  the life cycle, particularly during development
  of subsequent increments of an evolutionary
  strategy, modifications, upgrades, and
  reprocurement. The PM shall ensure that a
  flexible, performance-oriented strategy to
  sustain systems is developed and executed.

17
Army PBL

• Army Acquisition
• PBL Extract From Page 4-5 AR 701 31 December
  2003 (Para 1.5 x)
• x. Performance-based logistics. Performance-based
  logistics (PBL) is the preferred support strategy
  for materiel systems. The Army will implement PBL
  on weapon systems in order to provide the
  warfighter increased operational readiness
  increased reliability enhanced logistics
  response times enhanced deployment reduction in
  the logistics footprint and reduction in
  logistics costs.
• Army Performance Based Logistics (PBL) Concept
  (AR 700-127 1.5 a)
• Performance-based logistics (PBL) is the
  Department of Defense (DoD) preferred product
  support strategy for weapon system product
  support that employs the purchase of support as
  an integrated performance package designed to
  optimize system readiness. PBL is the delineation
  of output supportability requirements for
  acquisition systems and the assignment of
  responsibilities and incentives for the
  attainment of these requirements.

18
PBL Defined Graphic
19
Army PBL Guidance (AR 70-1)

• The extent and level of detail to which PBL will
  be applied will be based on a business case
  analysis (BCA).
• PBL will be implemented on all ACAT I and ACAT
  II programs, where PBL is proven to be
  economically and operationally feasible. PBL
  will be applied to ACAT III programs at the
  discretion of the PM, with approval from HQDA.
• When a PBL strategy is used, the PM will
  designate a product support integrator (PSI) to
  integrate all product support for the system.
• PBL must be tailored to individual system
  needs.
• PBL will be executed through performance based
  agreements (PBAs) with, at a minimum, the using
  MACOM(s) and the product support integrator
  (PSI).

20
PBL constraints

• The following will serve as constraints in the
  design and implementation of a PBL strategy.
• The approach must be transparent to the field
  user. Transparency will be determined by any
  changes in how the field user is trained to
  perform logistical activities.
• Contractors on the battlefield policy will be
  followed unless a written waiver is obtained as
  defined in AR 7159.
• The PSI will integrate with existing and future
  logistics systems, that is, STAMIS and WLMP.
• Total asset visibility (TAV) will be maintained.
• Standard Department of Defense (DOD)
  distribution hubs will be used for all
  contingency and wartime operations.
• Note the theme of integrating into Army Logistics
  Systems/Structures

21
PBL PBL Metrics

• PBL What does this mean - in practice.
• Seeking Logistics Performance Outcomes
• Gaining Outcomes Requires Incentives
• Inclusion of both in Performance Based Agreements
  (PBA)
• Contracts
• MOA/MOU
• All above infer Measurement/Metrics
• Make the connection between PBL strategy and
  Metrics
• In PBL Business Case Analysis (BCA)
• In PBL Product Support Integrator (PSI) Selection
• In PBL Product Support Provider (PSP)

Mark Weston-Dawkes Sept 2006
22
PBL Overarching Metrics

• Logistics Response Time (LRT)
• Definition Logistics Response Time is the
  period of calendar time from when a
  failure/malfunction is detected and validated by
  the maintainer to the time that the
  failure/malfunction has been resolved. This
  includes the time from when a need is identified
  until the provider satisfies that need, all
  associated supply chain and maintenance time, and
  delivery times of parts
• Formula LRT (Date (or time) of satisfaction
  of the logistics demand) minus
• ( Date (or time) of issue
  of logistics demand)
• Logistics Footprint
• Definition The government/contractor size of
  logistics support required to deploy, sustain,
  and move a weapon system for a given mission
  profile. Measurable elements should include but
  not be limited to inventory/equipment,
  personnel, facilities, transportation assets,
  supply, and real estate. Measures should
  quantify the footprint, i.e. weight, area,
  volume, and personnel etc. as appropriate.
• Formula Logistics Footprint encompasses a wide
  variety of elements that having one specific
  formula will not envelop the entire embodiment of
  logistics support. However, each element can be
  quantified, measured, and assessed individually.
  These individual assessments can then be
  integrated as an overarching logistics footprint
  analysis.
• Logistic Footprint is a function of various
  elements to include area (a), volume (v), weight
  (w), and support personnel (sp)

23
PBL Overarching Metrics (cont.-1)

• Cost Per Unit Usage
• Definition The total Operating and Support
  costs, to include overhead and management costs,
  for a weapon system usage attributable to a given
  unit of usage under established conditions.
  Usage can be measured in terms of unit density or
  individual weapons system usage factors include
  miles, rounds, launches, flight hours, time,
  systems, etc.
• Formula Total Operating Support Costs
• Miles/Rounds/Launches/Flight hours
• Operational Availability (Ao)
• Definition The percent of time that a weapon
  system or SoS is mission capable
• Formula Over any period of time, the directly
  measured Ao (post-fielding) isAo Up Time /
  Total Time Up Time / (Up Time Down Time)
• The expected long-term, steady-state Ao
  (throughout the life cycle) is determined from
  the classic formula
• Ao MTBF/ (MTBF MTTR MLDT)
• Note that Preventative Maintenance must not be
  ignored in overall Assmt. Also be careful about
  standby time.

24
PBL Overarching Metrics (cont.-2)

• Mission Reliability
• Definition The measure(s) or ability of a system
  to achieve Operational Performance (OP) for a
  defined mission or specified mission profile.
• Formula ( Number of successful missions) Divided
  by
• (Number of attempted missions)
• Alternate Use of Formula  this may be used where
  MR success is best measured in terms where
  discrete mission success does not provide best
  meaning for this metric.  This is the formula
  where OP is measured as a percentage of
  Mission Duration (MD).  
• Therefore  MR   (Total Operational Performance
  for MD) Divided By (Total Mission Duration)
• Many Sub-elements possible for Each Overarching
  Metric