Performance Contracting

1
Performance Contracting

• Presentation to theSoutheast Evaluation
  Association
• February 2, 2006

2
Walter Sachs Staff Director, Contracts Office of
Contracted Client Services (850)
921-8983 walter_sachs_at_dcf.state.fl.us

3
Success Stories
The State of Practice

• California Earthquake Repair
• Jersey City Water System
• Oklahoma Rehabilitative Services
• North Carolina Adoption Program
• Illinois Family Services Foster Care
• Ontario Realty Corporation

4
California Earthquake Repair
The State of Practice

• Financial Incentives
• 200k/day (bonus or penalty)
• 13.8 million given in performance bonuses
• Savings
• 74 million to local economy (end-outcome)
• 12 million on contract administration
• Construction Time
• 2 months vs. an estimated 9-24 months

5
Jersey City Water System
The State of Practice
Contractor Incentive - Percent of additional
collections above current rate of 66
6
OK Community Rehabilitation Services Unit
The State of Practice

• Paying contractors at predefined milestones
• Determination of consumer needs (10 of bid)
• Vocational preparation completed (10 of bid)
• Job Placement (10 of bid)
• Four weeks of job retention (20 of bid)
• Job stabilization of 10-17 weeks of retention
  with minimal support contacts (20 of bid)
• Consumer rehabilitated no support contacts 90
  days after stabilization period (30 of bid)
• Costs per placement declined by 51, waiting
  lists for clients dropped by 53, and the number
  of individuals who failed to get a job fell by
  25.

7
North Carolina Dept. of HHSAdoption Program
The State of Practice

• Paying contractors at predefined milestones
• Placement for the purpose of adoption (60
  percent of average placement cost)
• Decree of adoption (20 percent of average
  placement cost)
• One year of intact placement after decree (20 of
  average placement cost)
• Adoptions rose from 261 in 1994 to 364 in 1996,
  660 in 1998, and 1338 in 2001.

8
Illinois Dept. of Children and Family Services
Foster Care
The State of Practice

• Peg Costs to Average Performance
• Each caseworker is assigned 33 cases per year
• They are paid for 25 cases automatically
• If they place 8 children of the 33 assigned, they
  breakeven
• If they place less than 8 children of the 33,
  they lose in that they must service 26-33 cases
  and get paid for 25
• If they place MORE than 8 children, they pocket
  the difference
• In the first year, the number of placements
  increased by 120. By year two, the increase
  was 390. Relative Home Care caseload
  declined by 41, Traditional Foster Care doubled
  the number of placements, and the
  disruption rate remained the same.

9
Ontario Realty CorporationFloating Incentive
Structure
The State of Practice

• PBC for land management services
• Covers 65,000 acres 3,500 properties 1,500
  leases 1.6 million square feet of
  commercial/industrial space
• Five Performance Objectives
• 13 broad measures, 30 performance measures and
  112 performance requirements
• Roughly one dozen (12) only monitored and
  rewarded/penalized at one time
• Contractor does not know which ones, so has to be
  compliant at all times

10
(No Transcript)
11
Critical Success Factors Designing
The State of Practice

• Strategic and Program Logic for the Agency is
  clear
• Clear scope of work
• Clear performance measures and expectations
• Define baseline and expected level of performance
• Canvass providers/contractors
• What measures would they propose?
• What incentives would they want? How?
• How would they want to report performance data?

12
Critical Success FactorsDesigning
The State of Practice

• Motivate with incentives
• make sure you can afford the incentives!
• Allow for flexibility in exchange for results
• Measure, Monitor and Report
• Establish Acceptable Quality Levels (AQL)

13
Critical Success Factors Managing
The State of Practice

• Monitor
• Adjust
• Benchmark and compare
• create a race to the top culture
• Communicate and reward success!

14
What makes a contract Performance-Based?

• Results vs. Activities
• Performance Measures
• Incentives, bonuses and rewards
• Flexibility
• Monitoring and Measuring Performance

15
Streamlining the Contracts Terms and Conditions

• Benchmark the purpose, objective, and projected
  outcome of the contracted services
• Determine the key components of each task and
  related outcome

16

• Performance Work Statement (PWS)
• Outputs are clear and agency wants to purchase
  specific outputs
• Performance incentives tied to cost, timeliness,
  quality and impact of outputs
• Narrow and limiting to the contractor
• Statement of Objective (SOO)
• Agency defines results to be achieved, and
  solicits a wide variety of technical solutions
  from contractors
• Outputs are not pre-determined by the agency
• Performance incentives tied to achievement of
  performance results (impact of outputs) and may
  include cost, timeliness, quality and impact of
  outputs associated with contractors technical
  solution
• Provides maximum flexibility to contractor on
  what work is to be done provides room for
  innovation

17
Remember

• Delete unnecessary requirements
• Eliminate how statements
• Let the contractor solve the problem
• Dont spec the requirement so tightly that you
  get the same solution from each offeror

18
Outcome Analysis

• Answers the questions
• What is the objective of the service?
• What outcomes do the users need?
• What must be accomplished to satisfy these
  requirements?
• What are the subtasks that must be performed to
  accomplish the task?
• What are the relationships between the outcomes?

19
More on Questions

• Align contract to agency mission and outcome
  goals
• Key Question Is the acquisition even needed?
• Provides the context to all team members on what
  impact contract will have to the agency
• Baseline What is the current level of
  performance?
• Identify desired level of results
• Where do I want to go?
• Identify desired level of success
• How will I know when I get there?
• Context What intermediate outcome issues must be
  confronted by the contractor?

20
PWS Format

• 1.0 Scope
• 1.1 Background Tells the contractor where this
  contractual effort fits in the history,
  chronology, or context of related or preceding
  work
• Conveys the broad picture
• 1.2 Objectives Main goals of the government
• Example
• Objectives of this effort are
• 1. Develop an automated data system to support
  5,000 employees who require word processing,
  graphics, and communications capability
• 2. Install the system
• 3. Train personnel in its use
• 4. Support and maintain for three years after
  installation

21
PWS Format (Cont)

• 2.0 Applicable Documents Federal, State or
  Program standards or specifications that are
  mandatory
• 3.0 Tasks (requirements) - Use action words,
  followed by the target of the action Example
  Test the program
• Each task should have a deliverable or output
• Essential to establish the quantitative and
  qualitative requirements that are the
  contractors obligations
• 4.0 Government-furnished property
• 5.0 Government-furnished facilities
• 6.0 Deliverable data

22
Writing the PWS (cont)

• PWS agrees with the rest of Contract
• Need to prepare standards, AQL, surveillance,
  when developing the PWS
• Include only tasks not covered in a cited
  specification
• Arrange PWS in a chronological order

Specify WHAT you want done, NOT HOW you want
it done
23
SOO Methodology

• The SOO is a Government prepared document
  included in an RFP that states the overall
  acquisition objectives
• The SOO methodology requires competing
  contractors to develop the statement of work,
  performance metrics, performance measures, and
  quality assurance plan
• It is used in solicitations to provide the
  maximum flexibility to offerors to propose
  innovative solutions

24
Developing the SOO

• Describe the scope
• Establish size and range of services needed
• Consider available funds to fund the acquisition
• Make sure that they requirements are realistic
  and competitive
• Incorporate performance objectives
• Mission related, measurable
• Let the contractor submit the metrics and quality
  assurance plan especially since the solution is
  not known
• Make sure the government and contractor share
  objectives
• Work in partnership toward a common goal
• Make the contractor part of the team

25
Keys to Success

• Conduct an outcome analysis
• Capture the results of the analysis
• Draft the performance requirements
• Let the contractor solve the problem

26
SOO The Final Product

• Purpose
• Scope
• Period of Performance
• Place of Performance
• Background
• Program objectives
• Constraints

REMEMBER Eliminate anything that is not
essential Experiment and Innovation! Not how, but
what!
27
Designing Performance Measuresand Including them
in RFPs

• Design performance measures as part of the RFP
  and educate potential recipients of the role of
  performance measures
• Develop RFPs to measure appropriate outcomes

28
Examine Private and Public Sector Solutions

• Commercial Options Identify commercial options
  available off-the-shelf or with modification
• Government Options Streamlined vehicles and
  coordinated procurement opportunities
• Competitive Sourcing Is this part of a managed
  competition?
• Market Research Tactics
• Public Information Industry associations, trade
  groups, corporate profiles, etc.
• Government Procurement Vendors of other programs
• Solicitation Industry days, requests for
  information, etc

Make your agency a smarter shopper
29
Market Research

• Market research is the continuous process of
  collecting information to maximize reliance on
  the commercial marketplace and to benefit from
  its capabilities, technologies, and competitive
  forces in meeting an agency need.
• Market research is essential to the government's
  ability to buy best-value products and services
  to solve mission-critical problems.
• Market research should be undertaken by the
  integrated solutions team long before attempting
  to write a performance work statement.

30
Market Research

• Key part of the early stage of acquisition, not
  just Contracting Officers Responsibility
• Obtain Requirements Information
• Identify potential vendors
• Assess the competitiveness of the market
• Identify commercial practices
• Identify successful procurements of others

31
Market Research

• Market research techniques
• contact industry and government experts
• review the market research results of others
• publish requests for information
• access Internet databases
• review catalogs and trade journals
• presolicitation conferences

32
Communicate with Industry

• Open Door
• Equal Opportunity
• Web sites
• Industry Days
• One on One or Due Diligence Meetings with
  Companies
• Promote more input from Industry

33
Talk to Private Sector Companies

• Acquire information on commercial offerings,
  capabilities, and practices before structuring
  the acquisition
• Contact with vendors and suppliers for purposes
  of market research is encouraged
• FAR 15.201(a) promotes the exchange of
  information "among all interested parties, from
  the earliest identification of a requirement
  through receipt of proposals.
• Begin market research before a procurement is
  underway.
• Limitations that apply (once a procurement is
  underway)
• All prospective contractors must be treated
  fairly and impartially
• The standards of procurement integrity must be
  maintained.

34
Learn from Public-Sector Counterparts

• Talk to counterparts in other agencies
• This may avert problems that could otherwise
  arise
• Cross-agency cooperation and collaboration
• Another agency may have "solved" your problem
• Services may be available through an interagency
  agreement
• Two or more agencies may need to team on a
  solution that will provide seamless services to
  the public
• Agencies with similar needs may be able to
  provide lessons learned and best practices

35
Government-Industry Interface

• A few alternatives for the government
• Search the marketplace through a Request for
  Information (RFI) for sources and other
  information
• Conduct an early-on general conference to explain
  intentions
• In a formal solicitation - provide for a
  pre-proposal conference

36
Using the Logic Model to Develop Performance
Measurements
Input Activities and Outputs
Intermediate Outcomes End Outcomes
FTE FTE FTE


Input Activities and Outputs
Intermediate Outcomes End Outcomes
37
Selecting Performance Measures

• There are few perfect measures!
• Emphasize a few key strategic
• indicators of success!

38
Significant Technical Challenges

• Too Many Measures
• Wrong Kinds of Measures
• Too process and activity oriented
• No clearly defined Logic Model
• No measures of strategy
• Few measures of end outcome
• Dumbing-down of Measures
• Measuring only the things you can count rather
  than things that are strategically important

39
Performance Measure CriteriaThink SMART

• S PECIFIC
• M EASUREABLE
• A CCOUNTABLE
• R ESULTS-ORIENTED (1)
• T IME-BOUND

40
Applying SMART

• End-Outcome Reduce Smoking-related deaths,
  illness and costs
• Intermediate Outcome Reduce the number of new
  youth smokers (10-18) by 2 each year
• Results-oriented Youth smoking is where you can
  stop the habit before it takes hold and has a
  lasting health impact
• Specific number of new youth smokers (10-18)
• Accountable You have the ability to make it
  happen
• Measurable reduce by 2
• Time-Bound per year

41
Applying SMART to Activities and Outputs

• Output 1-- of publications produced on dangers
  of smoking per year
• Activity 1.1Research on age of new smokers
• Specific-publications on smoking dangers
• Measurable-- of publications
• Accountablethis is an output you dothe
  activities lead to the output
• Results-Orientedtiers to your Intermediate
  Outcome
• Time BoundPer year
• Output 2-- of restrictions passed on
  availability/selling of cigarettes per year
• Activity 2.1Identify of public areas with
  cigarette vending machines
• SpecificAvailability of cigarettes to minors
• Measurable-- of restrictions
• AccountableThis is an output you dothe
  activities lead to the output
• Results Orientedtiers to your Intermediate
  Outcome
• Time Boundper year

42
Determining End Outcomes Questions to Ask

• What is the bottom line of your program?
• If you had to defend your programs value/benefit
  before a grand jury, what 2-3 pieces of evidence
  would prove you were a success rather than a
  failure?
• What is the end benefit to the taxpayer or
  society from your program?
• Brainstorm How will you know you have been so
  successful that you can shut your program down?

43
Intermediate Outcomes

• Given the end outcomes you seek
• What must change in the status quo to create the
  conditions necessary for goal attainment? Who
  are the targets of change and what must they do?
• What causes the outcomes NOT to be achieved?
• What are your strategies?
• What must you influence?
• Magic Wand What are your wishes?

44
Program Outputs

• For each intermediate outcome
• What specific things can this agency do to cause
  that change to happen? to influence that target
  to change?
• What products could you produce?
• What services could you provide?
• What is the actual workload that is to be
  handled?
• (Note Dont include administrative items inside
  your program. Think of what things actually leave
  the four walls of your program.)

45
Work Process Mapping


Output 1 -Activity A -Activity B -Activity
C -Activity D Output 2 -Activity E -Activity
F -Activity G -Activity A -Activity B Etc.
Etc. For some activities, process measures can
be devised.
Business Line 1 -Output 1 Workload
Measure -Output 2 Workload Measure Business
Line 2 -Output 3 Workload Measure -Output 4
Workload Measure -Output 5 Workload
Measure Etc. Etc.

FTE

Strategy 1 -IO Performance Measure -IO
Performance Measure Strategy 2 -IO Performance
Measure -IO Performance Measure -IO Performance
Measure Strategy 3 -IO Performance Measure Etc.
Etc.
Outcome Goal Outcome Measure 1 Outcome
Measure 2 Outcome Measure 3
46
Separating Activities from Outputs
Activity Definition Template The purpose of
(Specific Program Work
Activity) Is to produce/provide (Output) To
(Target of Change) So that they
can (Intermediate Outcome Change)
47
Clarifying the Logic of the Program
Logic Model Template The mission of the
(Program) Is to produce/provide
(Products or Services) To (Target of
Change) So that they can
(Intermediate Outcome Change) Resulting
ultimately in (End Outcome Goal)
48
Vendor Selection

• Key part of any program
• Evaluation criteria and proposal instruction
  should be tied to the requirement
• Should lead to successful outcome?
• Use only areas\factors that provide true
  discriminators
• Too many areas and factors dilute the evaluation
• Make sure Industry understands what is important
  regarding your requirement

49
Two Types of Risk in Evaluations

• Proposal risk
• Technical and price risk associated with the
  proposed approach for meeting requirements
• Performance risk
• An offerors likelihood of success

50
Proposal Risk

• Technical Elements
• Contractor understands performance-based approach
• Contractor understands the requirement
• Contractor has processes and resources to perform
• Demonstrates reasonable due diligence
• Price
• Competitive market prices
• Subject to cost analysis

51
Price and Cost Analysis

• The process of examining and evaluating proposed
  price without evaluating its separate cost
  elements.
• Objective is to verify that the overall price is
  fair and reasonable (Market Price).
• Actions
• Compare proposed prices (ITB, RFP, Quotes, etc.)
• Compare proposed price to previous prices or to
  prices of similar programs.

• The process of reviewing and evaluating each of
  the cost elements.
• Objective is to determine that proposed costs are
  allowable, reasonable, and necessary.
• Actions
• Determine that proposed costs are allowable
• Evaluate necessity for and reasonableness of
  proposed costs
• Compare to actual costs previously incurred for
  the same services.

52
Mitigating Proposal Risk

• Communicate early and often
• Discuss requirements in open forum
• Compete the Solution
• Use down-selection and due diligence
• Use oral presentations
• Emphasize past performance
• Use best-value techniques
• Assess solutions for conflicts of interest

53
Performance Risk

• Performance risk
• An offerors likelihood of success, or
• an offerors likelihood of achieving performance
  excellence
• Mitigating Performance Risk
• Review Relevant Past Performance
• Evaluate offerors against Criteria for
  Performance Excellence

54
Source Selection Basics

• General Population Characteristics
• Risk Adverse
• Dislikes change
• Rarely thinks outside the box
• Questions
• How do you encourage innovative offers?
• How do you communicate to offerors that you dont
  want the same level of service described
  differently?

55
Keys to Success

• Develop source selection criteria (traditional)
• Emphasize best-value approach
• Cast wide net for maximum competition
• Emphasize past performance based on your criteria
• Examine conflicts of interest
• Examine financial stability (particularly with
  shift away from fee-for-service)
• Dont be afraid to re-compete if ideal solution
  not submitted!!

56
Evaluating the Progress of Your Vendor

• Determine the type of follow-up required
• Assess the grantees operational and financial
  capacity to overcome performance obstacles, take
  corrective action and report progress

57
Assisting with Implementing Effective Performance
Measures

• Link performance measures to budgeting and
  staffing
• Make performance measurement part of everyday
  functioning and improvement

58
Using External Evaluators to Assess Performance

• Develop value-added approach to using third party
  evaluators
• Balance the need for an expert evaluator with the
  need to incorporate evaluation into capacity
  building for grantees

59
Post-Award Monitoring, Oversight and Review

• Determine whether positive changes have been
  demonstrated
• Devise clear measures for performance and
  accountability

60
Manage and Monitor Performance

• Management starts with the incentive structure
• Maintain team-based management approach to PBC
  after award
• Develop capacity for collecting and analyzing
  performance information on contracts
• Establish mechanisms for validity and
  verification checks on performance information
• Manage change Kick-off meeting, transition
  period, and roll-out
• Benchmark and comparecontinually push for
  improvements
• Exercise corrective mechanisms when performance
  is off track
• Report contractor's performance regularly

61
Keys to Success

• Keep the team together
• Adjust roles and responsibilities
• Assign accountability for managing contract
  performance
• Regularly review performance in a Contract
  Performance Improvement Working Group
• Ask the right questions
• Report on the Contractors past performance

62
Ask the Right Questions

• Are we measuring the right thing?
• How are we doing?
• Is the acquisition achieving its cost, schedule,
  and performance goals?
• Is the contractor meeting or exceeding the
  contract's performance-based requirements?
• How effective is the contractor's performance in
  meeting or contributing to the agency's program
  performance goals?
• Are there problems or issues that we can address
  to mitigate risk?

63
Monitoring through Acceptable Quality Levels

• Number of unacceptable outcomes that government
  will accept
• Recognize that unacceptable work happens and zero
  tolerance drives costs
• Set a limit on how much will be accepted before
  penalizing the contractor (established in
  contract)

64
Acceptable Quality Levels Checklist

• Are they realistic?
• Do they represent true minimum levels of
  performance?
• Do they consider the cost trade-offs?
• Use market research to determine this
• If a percentage error, is the basis well
  understood?

65
Not Meeting Standards

• Establish value of task to total contract
• If AQL is breached, determine the number of
  errors as a percent of the total events measured
  or observed during the period
• Deduct an amount equal to the percent of errors
  multiplied by the value of the task

66
Performance Requirements Summary

• The tasks that the contractor must perform
• The standards and AQLs for those tasks
• The surveillance method/measure used by the
  government to monitor those tasks
• Usually listed as an exhibit or appendix to the
  RFP and resulting contract

Indicators, performance standards, AQL, and value
of tasks are contained in Performance
Requirements Summary
67
Using Results and Lesson Learned

• Did the project achieve its stated objectives?
• What other effects (good or bad) did the project
  have?
• What were the projects costs (financial,
  political, or social)?
• How does past performance effect future funding?
• Does documentation support results?
• What lessons learned can be used in future
  awards?

68
Seven Crucial Elements

69
Seven Crucial Elements

70
Constructing the Contract
Internal Consitency
71
StatementOf Work
Scope of Work
72
Specificity

• All of the activities and services that the
  provider is to perform
• All of the products and equipment the provider is
  to deliver
• The time period within which the services are to
  be provided

73
Clarity

• Directly related to the goals and objectives of
  the program
• No interpretation required
• Provide limits where needed

74
Measurable

• Activity-based performance requirements such as
  staffing levels, staff qualifications, number of
  clients to be served
• Acceptable quality level of services
• Acceptance of deliverables

75
Whats Right/Whats Wrong
Vague
More Specific

• The provider is to offer an adult education
  program for up to 30 adults. Program activities
  may include tutoring, counseling or other
  activities. The provider is responsible for
  providing staff to sufficiently operate the
  program.

Provider shall conduct an evening adult
education program for 30 eligible adults (as
defined in paragraph A.3.) at the providers
location Monday through Friday 630PM to 830PM
for the period of 7/1/04 to 6/30/05. The
providers staff will consist of a minimum of 1
supervisor and 6 tutors that meet the minimum
qualifications defined in paragraph B.3.a.
76
Major Program Goals

• Description of the major goals of the program
  that the contract will support

77
Major Program Goals must ...
78
Deliverables

• Events that trigger payments

79
Deliverables must be...
80
Deliverables
81
Fixed Price Deliverables
Not Related To Scope of Work
82
Fixed Price Deliverables
Related To Scope of Work
83
Fixed Rate Deliverables
Not Related To Scope of Work
84
Fixed Rate Deliverables
Related To Scope of Work
85
Cost Reimbursement Deliverables
Not Related To Scope of Work
86
Cost Reimbursement Deliverables
Related To Scope of Work
87
Performance Specifications

• The departments method for determining a
  providers performance including the impact the
  program has on the people it serves

88
Performance measures...
89
Writing the Section
What gets measured, gets done.
B. Manner of Service Provision 5.
Performance Specifications a.
Performance Measures b. Description
of Performance Measurement Terms c.
Performance Evaluation Methodology
90
Seven Crucial Elements

91
Evaluation

• The department has the responsibility of
  providing a reasonable and objective evaluation
  of the providers performance

92
Logic Model for Developmental Child Care