Performance Based Services Acquisition Procedures

1
Performance Based Services Acquisition Procedures
2
Performance Based Services Acquisition Procedures

• Learning Objectives
• 1. Understand the DOD policy of having 50 of all
  acquisitions in both dollars and actions be
  accomplished using performance based methods.
• 2. Develop a working knowledge of each of the
  following as they apply to the QASP (Quality
  Assurance Surveillance Plan)
• Performance Objectives
• Performance Standards
• Acceptable Quality Levels
• Methods of inspection
• Frequency of inspection
• Incentives

3
Performance Based Services Acquisition Procedures

• Learning Objectives
• 3. Understand how to develop and use the QASP for
  contract monitoring, completing a past
  performance report and as a source selection
  tool
• 4. Gain a basic understanding into the
  functioning of Award Fee Review Board (AFRB)
  process and the critical role COR contractor
  oversight plays.

4
Performance Based Services Acquisition Procedures

• PBSC TOP-LEVEL GUIDING PRINCIPLES
• To the maximum extent practicable, agencies shall
  use performance-based methods for acquiring
  services. Fixed-price, performance-based
  commercial service acquisitions are complementary
  strategies that encourage commercial contractors
  to conduct business with DoD.
• Utilize a multi-functional acquisition team to
  the maximum extent practicable.
• (2)

5
Performance Based Services Acquisition Procedures

• PBSC TOP-LEVEL GUIDING PRINCIPLES
• Early planning is essential in determining
  requirements and assessing market conditions, and
  it should include the user and as many relevant
  acquisition team members as possible.
• To maximize returns for all stakeholders,
  acquisition strategies should be tailored on the
  basis of
• experience
• market research
• and risk.
• Performance-Based Services Acquisition (PBSA) is
  not a one size fits all process.
• (3)

6
Performance Based Services Acquisition Procedures

• PBSC TOP-LEVEL GUIDING PRINCIPLES
• Strive to define requirements in clear, concise
  language.
• Focus on specific work outcomes and ensure that
  they are measurable to the greatest extent
  practicable.
• Templates are only a partial solution.
• Sample work statements should be individually
  tailored to the requirement, and for more complex
  requirements, work statements should be uniquely
  crafted. (4)

7
Performance Based Services Acquisition Procedures

• PBSC TOP-LEVEL GUIDING PRINCIPLES
• Need to
• Seek industry comment and suggestions regarding
• performance objectives
• standards
• and incentives
• How
• Through market research
• public meetings
• and draft solicitations
• (5)

8
Performance Based Services Acquisition Procedures

• PBSC TOP-LEVEL GUIDING PRINCIPLES
• Incentives
• Incentives should motivate a contractor to
  achieve performance levels of the highest quality
  consistent with economic efficiency.
• Ensure that incentives are
• Effective
• That reflect value both to the government and to
  the contractor.
• (6)

9
Performance Based Services Acquisition Procedures

• PBSC TOP-LEVEL GUIDING PRINCIPLES
• Contractor performance assessments (the process
  known as quality assurance) should focus on
  outcomes rather than on contractor processes.
• Focus on insight of the contractor performance,
  not oversight.
• Periodic assessment of contractor performance
  should emphasize clear communication, with the
  objective of encouraging and maintaining high
  standards of performance, and it should be
  consistent with past performance assessments.
• (7)

10
Performance Based Services Acquisition Procedures

• PBSC Overview
• CONTRACTOR PERFORMANCE MANAGEMENT
• Introduction
• Performance Assessment Plan
• Quality Control Plan
• Assessment Methods
• Performance Assessment Plan Development
• Some Considerations for Determining the
  Appropriate Assessment Method
• Suggested Performance Assessment Plan
  Outline (12)

11
Performance Based Services Acquisition Procedures

• Policy
• It is the policy of the Department of Defense
  that, in order to maximize performance,
  innovation and competition, often at a savings,
  performance based strategies for the acquisition
  of services are to be used wherever possible.
  While not all acquisitions for services can be
  conducted in a performance-based manner, the vast
  majority can. Those cases in which
  performance-based strategies are not employed
  should become the exception.
• (15)

12
Performance Based Services Acquisition Procedures

• Policy con't
• "In order to ensure that the Department
  continually realizes these savings and
  performance gains, the DoD establishes, at a
  minimum, that 50 percent of service acquisitions,
  measured in both dollars and actions, are to be
  performance-based by year 2005. Under
  Secretary of Defense, Acquisition, Technology
  Logistics (USD (ATL)), April 5, 2000."
• (16)

13
Performance Based Services Acquisition Procedures

• Performance Based Service Acquisitions
• PBSA Definition
• PBSA involves acquisition strategies, methods,
  and techniques that describe and communicate
  measurable outcomes rather than direct
  performance processes.
• It is structured around defining a service
  requirement in terms of performance objectives
  and providing contractors the latitude to
  determine how to meet those objectives.
• Simply put, it is a method for acquiring what is
  required and placing the responsibility for how
  it is accomplished on the contractor.
• (17)

14
Performance Based Services Acquisition Procedures

• Performance Based Service Acquisitions
• To be considered performance-based, an
  acquisition should contain, at a minimum, the
  following elements
• 1.Performance work statementDescribes the
  requirement in terms of measurable outcomes
  rather than by means of prescriptive methods.
• 2.Measurable performance standardsTo determine
  whether performance outcomes have been met,
  defines what is considered acceptable
  performance.
• (18.a)

15
Performance Based Services Acquisition Procedures

• 3.RemediesProcedures that address how to manage
  performance that does not meet performance
  standards. While not mandatory, incentives should
  be used, where appropriate, to encourage
  performance that will exceed performance
  standards. Remedies and incentives complement
  each other.
• Performance Assessment PlanDescribes how
  contractor performance will be measured and
  assessed against performance standards. (Quality
  Assurance Plan or Quality Assurance Surveillance
  Plan)
• (18.b)

16
Performance Based Services Acquisition Procedures

• Objectives of PBSA
• By describing requirements in terms of
  performance outcomes, agencies can help achieve
  the following objectives
• 1. Maximize performance Allows a contractor to
  deliver the required service by following its own
  best practices. Since the prime focus is on the
  end result, contractors can adjust their
  processes, as appropriate, through the life of
  the contract without the burden of contract
  modifications provided that the delivered service
  (outcome) remains in accordance with the
  contract. The use of incentives further motivates
  contractors to furnish the best performance of
  which they are capable.
• (19)

17
Performance Based Services Acquisition Procedures

• Objectives of PBSA, con't
• 2. Maximize competition and innovation
  Encouraging innovation from the supplier base by
  using performance requirements maximizes
  opportunities for competitive alternatives in
  lieu of government-directed solutions. Since PBSA
  allows for greater innovation, it has the
  potential to attract a broader industry base.
• 3. Encourage and promote the use of commercial
  services The vast majority of service
  requirements are commercial in nature. Use of FAR
  Part 12 (Acquisition of Commercial Items)
  procedures provides great benefits by minimizing
  the reporting burden and reducing the use of
  government-unique contract clauses and similar
  requirements, which can help attract a broader
  industry base. (20)

18
Performance Based Services Acquisition Procedures

• Objectives of PBSA, con't
• 4. Shift in risk Much of the risk is shifted
  from the government to industry, since
  contractors become responsible for achieving the
  objectives in the work statement through the use
  of their own best practices and processes.
  Agencies should consider this reality in
  determining the appropriate acquisition
  incentives.
• 5. Achieve savings Experience in both
  government and industry has demonstrated that use
  of performance requirements results in cost
  savings.
• (21)

19
Performance Based Services Acquisition Procedures

• The Business Relationship with the Contractor
• Business relationships should seek to create a
  cooperative environment to ensure
• Effective communication between the parties
• Teamwork
• Cooperation
• Good-faith performance.
• These are important for meeting mission
  objectives and resolving conflicts and problems.
• Each party should clearly understand the goals,
  objectives, and needs of the other. It is
  essential that government and industry work
  together as a team to communicate expectations,
  agree on common goals, and identify and address
  problems early on to achieve desirable
  outcomes. (30)

20
Performance Based Services Acquisition Procedures

• PWS Standards
• DEVELOPING A PERFORMANCE-BASED WORK STATEMENT AND
  ESTABLISHING MEASURABLE PERFORMANCE STANDARDS
• Introduction
• The key to using performance-based methodologies
  is
• Describing requirements as outcomes
• not
• in terms of how to accomplish the requirement.
• Therefore, a performance based work statement
  must be carefully structured to ensure that the
  requirement is articulated in this manner.
  Accordingly, the acquisition team will conduct a
  series of in-depth analyses to understand the
  requirement fully in order to be able to
  articulate the desired outcomes.(42)

21
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis
• Developing a performance work statement involves
  a series of analysis-oriented steps to help
  identify and define the requirement.
• Step 1 - Define the desired outcomes. What must
  be accomplished to satisfy the requirement?
• To define desired outcomes, list what needs to be
  accomplished in order to satisfy the overall
  requirement, from a top-level perspective. Use an
  interview or brainstorming approach with the
  customer (user) to determine all dependent
  variables (what, when, where, who, quantity,
  quality levels , etc.) to ensure that all unique
  requirements have been considered. For some
  requirements, you may simply be reviewing
  previous requirements for validity and
  accuracy. (43)

22
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• Example
• For - Custodial requirement
• clean office administration building
• clean office training building
• clean office physics lab
• For - Pre-hardware development
• conceptual design component development
• design fabrication
• testing reporting requirements
• (44)

23
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• Step 2 - On the basis of the desired outcomes
  defined in step 1, conduct an outcome analysis to
  identify performance objectives. What tasks must
  be accomplished to arrive at the desired
  outcomes?
• An outcome analysis is the process that
  identifies specific performance objectives for
  those outcomes defined in the previous step.
• Performance objectives are the specific services
  that you want performed and delivered by the
  contractor, defined in terms of the outcomes.
• (45)

24
Performance Based Services Acquisition Procedures

• Performance Requirement Analysis con't
• Performance objectives
• This step differs from the previous step in that
  it goes into greater detail and expands the
  analysis beyond the top-level perspective.
• The goal is to describe adequately what is
  expected in accomplishing the requirement (not
  how to accomplish it).
• (46.a)

25
Performance Based Services Acquisition Procedures

• Performance Requirement Analysis con't
• Performance objectives
• You should start by segregating desired outcomes
  into lower task levels and linking those tasks
  together into a logical flow of activities.
• Using a tree diagram to outline each of the basic
  outcomes (those top-level perspectives) is
  recommended in order to ensure that you have
  considered all critical elements of the
  requirement. Appendix A of the DOD Guide shows a
  tree diagram that illustrates the intent of this
  step.
• (46.b)

26
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• TIP Keep in mind that there may not have to be a
  performance objective (a need to get into greater
  detail) for each task specified, especially those
  tasks at the lower task levels. The performance
  objective of lower tasks may be inherent in the
  higher level tasks and thus may be captured at
  that higher level. You should go only as low as
  you have to in order to describe the requirement
  adequately.
• (47.a)

27
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• TIP Ensure that each performance objective is
  necessary and carefully chosen. Try not to make
  any performance objective unduly burdensome or
  restrictive. Restrictive performance objectives
  are often pricey and may result in delivery of
  unnecessary requirements at a high price.
  However, if it is absolutely necessary in order
  to meet the mission, proceed accordingly. (47.b)

28
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Standards Analysis
• Step 3 - On the basis of the performance
  objectives identified in step 2, conduct a
  performance analysis to identify the appropriate
  performance standards and acceptable quality
  levels (AQLs).
• When or how will I know that the outcome has been
  satisfactorily achieved, and how much deviation
  from the performance standard will I allow the
  contractor, if any?
• A performance analysis is a process that
  identifies how a performance objective should be
  measured and, thus, what performance standards,
  (e.g., timeliness or quality levels) are
  appropriate and reasonable for that particular
  performance objective.
• (48)

29
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• In this step, you may also identify AQLs, also
  known as thresholds.
• Performance standards and AQLs are very important
  in that they identify acceptable levels of
  performance.
• Developing and setting performance standards and
  AQLs are judgment calls based on the needs of the
  mission, available expertise, and market
  research. (49)

30
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• AQLs constitute a minimally acceptable level of
  performance and are typically stated as a
  percentage of required conformances (e.g., clean,
  95 of the time) or as a number of permissible
  deviations (e.g., 1 error per x time period).
• In developing AQLs, you are asking, What minimum
  level of quality do I need to meet my mission
  needs?
• Keep in mind that not every performance standard
  may have an AQL. When a performance standard does
  not include an AQL, then you are stating that no
  deviations are allowed in meeting the performance
  outcome (see the tips below).
• (50)

31
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• TIP Exercise care in establishing AQLs for
  performance standards.
• The closer to 100 percent perfection, then the
  higher the price.
• Ensure that performance standards are measurable
  if possible.
• Think carefully when establishing a 100 percent
  standard (or not establishing deviations), since
  this threshold can directly affect the price
• 100 percent AQLs (thresholds) can be unreasonable
  or unrealistic for some performance objectives
• If the performance objective is critical (for
  instance, if it covers a life-threatening
  matter), 100 percent is acceptable
• If the AQL is set too low, it could act as a
  disincentive to good contract performance. (5
  1)

32
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• Examples of Performance Standards
• Response times, delivery times, timeliness
  meeting deadlines or due dates, adherence to
  schedule.
• Error rates number of mistakes/errors allowed
  in meeting the performance standard.
• Accuracy rates similar to error rates, but most
  often stated in terms of percentages.
• Completion milestone rates x percent complete
  at a given date.
• Cost control keeping within the estimated cost
  or target cost. Applies in cost-reimbursement
  contract arrangement. (52)

33
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• Performance Requirements Summary
• The desired outcomes, performance objectives,
  performance standards, and AQLs, that have been
  developed through the previously explained
  analyses are then documented in a Performance
  Requirements Summary (PRS). See Appendix B of the
  DOD Guide for a sample PRS matrix.
• The PRS will be the baseline for the performance
  work statement
• PRSs should be brief and should capture the
  salient elements of the requirement
• (53)

34
Performance Based Services Acquisition Procedures

• PWS Standards
• Performance Requirement Analysis con't
• In the actual performance work statement, the
  acquisition team will elaborate on and describe
  the requirement in greater detail. The ultimate
  goal is to describe the requirement in a way that
  allows an offeror to understand fully what will
  be necessary to accomplish it. Appendix C of the
  DOD Guide contains several samples of PRSs.
• (54.a)

35
Performance Based Services Acquisition Procedures

• PWS Standards
• NOTE Many of the samples in Appendix C come from
  various agencies and other sources. Performance
  objectives and performance standards are
  sometimes combined in one tasking sentence.
  Likewise, performance standards and AQLs can also
  be combined. There is no preferred format.
  Agencies should ensure only that PRSs and
  performance work statements fully communicate the
  expectation using the appropriate
  performance-based service acquisition
  elements. (54.b)

36
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Contracts, by their very nature, motivate
  successful performancecontractors that fail to
  per form satisfactorily dont get paid.
• Contracts are incorporating specified incentives
  designed to encourage superior performance
• (63.a)

37
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES, cont
• The government collects, maintains, and uses
  information on past performance. An exceptional
  track record gives a contractor a greater
  competitive edge in future source selections and
  thus a stronger assurance of future work.
• Also, contract clauses such as liquidated damages
  provide a negative incentiveif the contractor
  causes harm or damage to the government as a
  result of failure to perform, the contractor must
  compensate the government in accordance with the
  contract clause.
• (63.b)

38
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Incentives can be
• Monetary
• non-monetary
• positive
• negative
• They can be based on
• cost
• schedule
• quality of performance
• The goal is to encourage and motivate the
  best-quality performance. (64)

39
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Cost-based incentives Cost incentives are
  designed to relate profit or fee to results
  achieved by the contractor in relation to
  identified cost-based targets.
• Performance-based incentives Performance
  incentives are designed to relate profit or fee
  to results achieved by the contractor in relation
  to identified performance-based targets.
• (65.a)

40
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Performance-based incentives
• For services such as maintenance of equipment,
  typical measures would be mean time between
  failures (MTBF), mean time to repair (MTTR) or
  system availability rates (in-commission rates).
• Regardless of the measure, performance incentives
  must be quantified and within a reasonable range
  (high-target-low)
• (65.b)

41
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Award-fee contract arrangements Using evaluation
  factors established in an award fee plan,
  award-fee contracts are a tool for subjectively
  assessing contractor performance for a given
  evaluation period.
• They allow contractors to earn a portion (if not
  all) of an award-fee pool established at the
  beginning of the evaluation period.
• The agency unilaterally determines the amount of
  earned fee.
• In the context of PBSA, the award-fee evaluation
  will be based on a subjective assessment of how
  well the contractor meets or exceeds the
  applicable performance standards.
• (66)

42
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Award-term contract arrangements Award-term
  arrangements are very similar to award-fee
  contracts, however, instead of money as
  compensation for quality performance, the
  contractor is awarded additional periods of
  performance.
• Or, if performance is habitually below standard,
  the period of performance can be shortened.
• (67.a)

43
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Award-term arrangements are most suitable when
  establishing a long-term relationship is valuable
  both to the government and to the potential
  contractor.
• They differ from options in that award terms are
  based on a formal evaluation process and do not
  entail the regulatory procedures associated with
  priced options.
• Award-term arrangements are relatively new. See
  Appendix I of the DOD Guide for more information.
• (67.b)

44
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Schedule incentives Schedule incentives focus on
  getting a contractor to exceed delivery
  expectations. They can be defined in terms of
  calendar days or months, attaining or exceeding
  milestones, or meeting rapid-response or urgent
  requirements.
• Past performance Past performance information
  can affect decisions to exercise options or to
  make future contract awards. Thus past
  performance assessments are a quick way for
  motivating improved performance or to reinforce
  exceptional performance. Keep in mind that the
  integrity of a past performance evaluation is
  essential.
• (68)

45
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Considerations When Contemplating Incentives
• Will enhanced performance provide additional
  value to the mission?
• Which areas of the requirement would benefit most
  from enhanced performance?
• Which areas do not need added incentives (or
  which areas can do without than)?
• How much is the agency willing to pay to achieve
  a level of performance beyond the performance
  standard? (69)

46
Performance Based Services Acquisition Procedures

• PWS Standards
• INCENTIVES AND REMEDIES
• Is there a potential for using cost-sharing?
• Do contractors within the particular industry
  prefer additional performance periods (award
  terms) in lieu of monetary incentives (award
  fees)?
• Is the incentive affordable?
• Will it affect timelines or schedules in a
  positive way? Adversely?
• (70)

47
Performance Based Services Acquisition Procedures

• PWS Standards
• CONTRACTOR PERFORMANCE MANAGEMENT
• In a performance-based environment, it is the
  contractor that is contractually responsible for
  quality assurance, further motivated through
  various kinds of incentives such as award-fee and
  past performance assessments
• Agencies are still responsible for ensuring that
  they get what they are paying for by periodically
  evaluating performance through the appropriate
  assessment methods
• (76)

48
Performance Based Services Acquisition Procedures

• PWS Standards
• CONTRACTOR PERFORMANCE MANAGEMENT
• Performance Assessment Plan
• A performance assessment plan describes how
  government personnel will evaluate and assess
  contractor performance
• It is intended to be a living document that
  should be revised or modified as circumstances
  warrant
• (77.a)

49
Performance Based Services Acquisition Procedures

• PWS Standards
• CONTRACTOR PERFORMANCE MANAGEMENT
• Performance Assessment Plan
• It is based on the premise that the contractor,
  not the government, is responsible for managing
  and ensuring that quality controls meet the terms
  of the contract
• If the performance assessment plan is not
  incorporated into the contract, it should
  nevertheless be furnished to the contractor.
• (77.b)

50
Performance Based Services Acquisition Procedures

• PWS Standards
• Assessment Methods
• Random sampling
• Random sampling is a statistically based method
  that assumes receipt of acceptable performance if
  a given percentage or number of scheduled
  assessments are found to be acceptable
• The results of these assessments help determine
  the governments next course of action vis-a-vis
  the contractor, if necessary, and whether
  adjustments in this method of assessment are
  necessary
• If performance is considered marginal or
  unsatisfactory, the evaluators should document
  the discrepancy or finding and begin corrective
  action (79)

51
Performance Based Services Acquisition Procedures

• PWS Standards
• Assessment Methods
• Random sampling, con't
• If performance is satisfactory or exceptional,
  they should consider adjusting the sample size or
  sampling frequency
• Random sampling is the most appropriate method
  for frequently recurring tasks
• It works best when the number of instances is
  very large and a statistically valid sample can
  be obtained. (80)

52
Performance Based Services Acquisition Procedures

• PWS Standards
• Assessment Methods
• Periodic Sampling Periodic sampling is similar
  to random sampling, but it is planned at specific
  intervals or dates
• It may be appropriate for tasks that occur
  infrequently
• Selecting this tool to determine a contractors
  compliance with contract requirements can be
  quite effective, and it allows for assessing
  confidence in the contractor without consuming a
  significant amount of time. (81)

53
Performance Based Services Acquisition Procedures

• PWS Standards
• Assessment Methods
• Trend analysis
• Trend analysis should be used regularly and
  continually to assess the contractors ongoing
  performance over time
• It is a good idea to build a database from data
  that have been gathered through performance
  assessment
• Additionally, contractor-managed metrics may
  provide any added information needed for the
  analysis
• This database should be created and maintained by
  government personnel. (82)

54
Performance Based Services Acquisition Procedures

• PWS Standards
• Assessment Methods
• Customer feedback
• Customer feedback is firsthand information from
  the actual users of the service
• It should be used to supplement other forms of
  evaluation and assessment, and it is especially
  useful for those areas that do not lend
  themselves to the typical forms of assessment
• However, customer feedback information should be
  used prudently
• (83)

55
Performance Based Services Acquisition Procedures

• PWS Standards
• Assessment Methods
• Customer feedback, con't
• Sometimes customer feedback is complaint-oriented,
  likely to be subjective in nature, and may not
  always relate to actual requirements of the
  contract. Such information requires thorough
  validation.
• Sometimes complaints from customers who are not
  familiar with the PWS, can expect performance
  that was not called out for in the PWS, making
  complaints baseless and unenforceable
• (84)