Measuring Efficiency

1
Measuring Efficiency

• CRJS 4466EA

2
Introduction

• It is very important to understand the
  effectiveness of a program, as we have discovered
  in all earlier chapters
• But, it is equally important to be able to inform
  stakeholders about program costs, and more
  importantly, how program outcomes compare to
  program costs

3
Cost-benefit analysis

• Expressed in monetary terms
• E.g. a cost-benefit analysis of a program to
  reduce cigarette smoking would focus on the
  difference between the dollars expended on the
  anti-smoking program and the dollar savings from
  reduced medical care for smoking-related
  diseases, days lost from work, and so on (Rossi,
  Freeman and Lipsey, 1999)

4
Cost-effectiveness analysis

• Outcomes are expressed in substantive terms
• E.g. a cost-effectiveness analysis of the same
  smoking program would estimate the dollars that
  had to be expended to convert each smoker into a
  non-smoker (Rossi, Freeman and Lipsey, 1999)

5
Acceptance of efficiency assessment

• Derives from business applications
• Very acceptable in human services to assess
  efficiency, but concerns remain about acceptable
  procedures
• Unfamiliarity with procedures
• Debate regarding the appropriateness of applying
  monetary values to social program outcomes
• An unwillingness to let go of what may not be
  working

6
Timing of efficiency analysis

• Most commonly undertaken at the planning stage or
  post program
• Before analysis is based on estimates of the
  future
• Presumes a given magnitude of positive net impact
• Not based on empirical information therefore over
  and under-estimates can occur
• Most important when the program is unlikely to be
  abandoned once implemented
• Important when resource commitments are
  anticipated to be high

7
After analysis

• Most commonly efficiency analysis occurs after
  outcome evaluation completion
• in comparative terms, the issue is to determine
  the differential payoff of one program versus
  another for example, comparing the reduction in
  arrest rates for drunken driving brought about by
  an educational program with that of a program
  that pays for taxis to take people home after
  they have imbibed too much. In ex post analyses,
  estimates of costs and outcomes are based on
  studies of the types described in previous
  chapters on program monitoring and impact
  evaluations (Rossi, Freeman and Lipsey, 1999)

8
The concepts of cost-benefit and
cost-effectiveness analyses

• Cost-benefit analyses
• Requires tangible and intangible estimates of
  program costs, both direct and indirect
• Once the benefits and costs are specified, they
  are translated into a common measure (usually a
  monetary unit)
• A particular economic perspective is adopted
• Controversy around procedures used to convert
  inputs and outputs into monetary values
• clearly, the assumptions underlying the
  definitions of the measures of costs and benefits
  strongly influence the resulting conclusions.
  Consequently, the analyst is required, at the
  very least, to state the basis for the
  assumptions that underlie the analysis (Rossi,
  Freeman and Lipsey, 1999)

9

• Cost-effectiveness analyses
• Less controversial
• Merely requires monetizing the programs costs
• Benefits are expressed in outcome units
• an ex ante cost-effectiveness analysis allows
  potential programs to be compared and ranked
  according to the magnitudes of their effects
  relative to their estimated costs (Rossi,
  Freeman and Lipsey, 1999)
• inputs and outputs replace, to a considerable
  extent, estimates and assumptions (Rossi,
  Freeman and Lipsey, 1999)

10
The uses of efficiency analyses

• Cant be done when outcomes are unknown or
  unestimable
• Senseless for outcome evaluations that show no
  significant net effects
• Senseless for programs in progress that do not
  yet have program impact information

11
Methodology of cost-benefit analysis

• benefits and costs must be defined from a single
  perspective because mixing points of view results
  in confused specifications and overlapping or
  double counting(Rossi, Freeman and Lipsey, 1999)
• Accounting perspectives
• Individual target
• Program sponsor
• Community aggregates (communal)

12
Individual target accounting perspective

• Takes the point of view of the units that are the
  program target
• Can provide for the highest benefit-to-cost
  results because this group rarely bears the cost
  of the intervention

13
Program sponsor accounting perspective

• Takes the point of view of the funding source in
  valuing benefits and specifying cost factors
• analysis from this perspective is designed to
  reveal what the sponsor will pay to provide a
  program and what benefits (or profits) should
  accrue to the sponsor (Rossi, Freeman and
  Lipsey, 1999)
• Most appropriate perspective when choices need to
  be made by the sponsor between alternative
  programs in the face of fixed budget

14
The communal accounting perspective

• Takes the perspective of the community or society
  as a whole (usually in terms of total income)
• Most comprehensive approach but also most complex
• Special efforts are made to account for secondary
  or indirect project effects effects on groups
  that are not directly involved in the
  intervention (Rossi, Freeman and Lipsey, 1999)
• Can also be expanded to include equity
  considerations, that is the distributional
  effects of programs among different subgroups
  (Rossi, Freeman and Lipsey, 1999)

15

• Communal perspectives include costs and benefits
  from the other types of perspectives but monetize
  and value items differently
• But communal perspectives include opportunity
  costs
• Generally the communal perspective is the most
  politically neutral

16
Case Studies

• Exhibit 11-F Components of Cost-Benefit Analyses
  from Different Perspectives
• Exhibit 11-G Hypothetical Example of
  Cost-Benefit Calculation from Different
  Accounting Perspectives (Rossi, Freeman and
  Lipsey, 1999)

17
Measuring costs and benefits

• Potential problems
• Identifying and measuring all program costs and
  benefits
• Expressing all costs and benefits in terms of a
  common denominator (translating them into money
  values)
• Social programs do not often produce results that
  can be valued accurately by means of market
  prices
• Sometimes cost-effectiveness analysis should be
  used because with this form of analysis benefits
  neednt be valued

18
Monetizing outcomes

• Money measurements
• Estimate of direct monetary benefits
• Market valuation
• Econometric estimation
• Hypothetical questions
• Observing political choices
• Opportunity costs
• Secondary effects (externalities)
• Shadow prices
• Derived prices meant to reflect their true
  benefit and cost

19
Comparing costs to benefits

• Simple method (net benefit)
• Subtract costs from benefits
• Ratio of benefits to costs
• More difficult to interpret