Measuring%20and%20understanding%20public%20sector%20productivity

1
Measuring and understanding public sector
productivity

• Helen Simpson
• IFS and CMPO

2
Outline

• Why measure productivity?
• Measuring outputs and inputs
• Productivity measurement techniques
• Understanding the drivers of productivity

3
Why measure productivity?

• Productivity volume of output relative to volume
  of inputs
• Not the only concern from point of view of
  welfare quantity and mix of outputs provided by
  public sector matter
• Measures
• Changes over time in efficiency with which
  resources are used
• Differences across organisations in efficiency
  with which resources are used
• Use as a comprehensive measure of performance to
  understand why some organisations get more out
  given what they put in

4
Measuring outputs

• Want to measure the volume of output
• Private sector
• Typically have data on value of output
• Value of output quantity of output x price of
  output
• Prices are serving as weights in well
  functioning private sector markets prices reflect
  marginal benefits and marginal costs
• Deflate value of output using a price index
• Often far from perfect e.g. capturing quality
  change
• Note for some private sector services, e.g.
  banks, problems are akin to those for public
  sector

5
Measuring outputs

• Public sector
• Want to capture the full range of outputs valued
  by society
• E.g. doctors surgery
• Amount and quality of treatment
• Equality of distribution of treatment across
  different types of individuals and illnesses
• Ease of registering and getting an appointment
• Insurance services
• Direct versus indirect or collective provision
• Good measure of output of fire service would
  capture fires extinguished and fire prevention
• Collective services outputs inputs

6
Measuring outputs

• Public sector
• Measures of activities, outputs and outcomes may
  all provide useful information
• Quality
• Activities or outputs, number of lessons provided
  or pupils taught wont capture quality, but
  qualifications or the impact of education on
  employment outcomes and earnings might help
• Isolating effects of providers on outcomes
• Even if all relevant outputs can be measured how
  to add up?
• In principle weights should reflect marginal
  social benefits associated with each output

7
Measuring outputs

• How to weight different outputs together?
• Relative costs?
• Often used in practice, e.g. education sector
  primary and secondary schools pupil attendance
  measures weighted together by costs
• Attribution of costs
• Surveys, implicit valuations?
• Private sector prices?
• Different characteristics of services provided
• Different characteristics of users

8
Measuring outputs

• What if preferences and valuations change over
  time?
• Measuring performance of an organisation over
  time
• Changes in marginal social valuations captured in
  increase in volume output to extent that reflect
  quality improvement, but not changes in
  valuations that are not driven by the actions of
  the service provider
• ONS consultation
• Increased value of public service provision in an
  economy with rising real incomes?

9
Measuring inputs

• For many inputs, measurement no greater problem
  than in private sector
• Full-time equivalent employees, value of capital
  stock (capital services provided),intermediate
  inputs
• Price indices
• Some public services can be thought of as
  involving joint production
• Characteristics of individuals using the service
  can affect measure of gross output
• Effort expended by an individual may also affect
  output
• Latter can be influenced by provider a
  characteristic of provision that will be valued
  by society
• Value-added measures
• Or only compare like with like

10
Measuring outputs and inputs

• Measuring output of public sector services very
  tricky
• Important to capture as much of the output that
  is valued as possible if productivity measurement
  is to be informative
• Accurate measurement of inputs important too
• Taking account of characteristics of individuals
  using the service could be crucial when comparing
  across organisations
• But measurement of outputs and inputs for private
  sector not perfect either

11
Techniques for measuring productivity

• Index measures
• Estimating a production or cost function
• Measuring productivity relative to an efficiency
  frontier
• Stochastic Frontier Analysis
• Data Envelopment Analysis
• Partial productivity measures

12
Index measures

• Relate volume of outputs to volume of inputs
• Require weights to add up
• Productivity growth
• Straightforward to interpret
• Adding in more inputs to explain away
  differences in labour productivity
• Comparing across organisations
• Total factor productivity as a measure of our
  ignorance
• Need for accurate data

13
Estimating a production function

• Uses aggregate measure of output
• Comparing across organisations
• Relative total factor productivity as a residual
• Add in more characteristics to try and explain
  this residual
• Estimation issues how to accurately estimate
  parameters
• Can also estimate a cost function doesnt
  require aggregation of outputs

14
Measuring efficiency relative to the frontier

• Techniques have been used to measure efficiency
  in public sector
• Avoid the need to aggregate outputs
• Stochastic Frontier Analysis (SFA)
• Data Envelopment Analysis (DEA)

15
Stochastic Frontier Analysis
16
Stochastic Frontier Analysis
17
Stochastic Frontier Analysis
18
Data Envelopment Analysis
19
Data Envelopment Analysis
20
SFA and DEA

• How robust are these techniques?
• SFA productivity rankings may turn on
  assumptions made about shape of frontier, and the
  distributions of noise and inefficiency
• DEA sensitivity of results to measurement error
  in the data
• DEA possibility of comparing to hypothetical
  organisations
• DEA the data decide the weights, but how
  appropriate will these be?
• Empirical studies, e.g. for hospitals, have shown
  sensitivity of productivity rankings to technique
  used

21
Partial productivity measures

• Measure productivity for a single output or
  outcome, e.g. survival rates following treatment
  for a particular illness
• Might provide very accurate measure of output
• But might be concerns about accurate attribution
  of inputs
• Unlikely to be representative of productivity of
  organisation as a whole
• Potentially even more so if output is targeted
• Main message ensure robustness of any
  productivity scores or rankings to different
  measurement techniques

22
Understanding the drivers of productivity

• Approach in private sector studies
• Try and explain the total factor productivity
  residual by adding in further information
• E.g. firms RD stocks, ownership characteristics
  
• Two issues that have received attention in both
  private and public sectors
• Effects of competition on productivity
• Conditional on inputs (input growth), is output
  (growth) higher when product market competition
  is more intense?
• Effects of performance incentives on productivity
• Conditional on inputs, do firms that use
  performance incentives produce more output?

23
Effects of competition

• Evidence for private and public sectors differs
• Not necessarily surprising given differences in
  incentives and constraints
• But are differences in measures used in studies
• Key issue is picking up causal effects some
  studies exploit reforms that affect degree of
  competition
• Private sector
• Positive effect of competition on productivity
  growth (innovation), e.g. Nickell, (1996)
• Positive effect of competition on level and
  growth rate of productivity, e.g. Disney et al.
  (2003)
• Role of restructuring and exit of relatively poor
  performers

24
Effects of competition

• Public sector
• Evidence more mixed - education and health
  (Burgess et al., 2005)
• Some for US healthcare sector (Kessler and
  McClellan, 2000) finds some positive effects
• Some for UK healthcare sector finds small
  negative effects on outcomes (Propper et al.,
  2004)
• Single outcome measures, or look at outcomes and
  costs separately
• Can capture single aspect of provision
  accurately, but difficult to make direct
  comparisons to private sector evidence
• Expansion, contraction and exit in the public
  sector?

25
Effects of performance incentives

• Private sector
• Studies of performance-based remuneration
  (employee or executive share ownership),
  typically find positive relationship with
  productivity, (e.g. Conyon and Freeman, 2004)
• Tricky to establish causality
• Some evidence on incentive schemes such as piece
  rates - lead to increased productivity, (Lazear,
  2000)
• Public sector evidence
• More in line with that for the private sector
• Two studies that find positive effects on
  outputs, (Burgess et al., 2004 and Atkinson et
  al., 2004)
• Look at multiple output measures
• But important in this case to try and use
  organisation-level measure diversion of effort
  away from non-targeted activities

26
Understanding the drivers of productivity

• Evidence on competition not very consistent
  across two sectors
• Evidence on performance incentives pointing in
  same direction
• But, studies for the two sectors use different
  measures
• Good to try and use (robust!) organisation-level
  measures for public sector
• Especially as public sector presents
  opportunities for identifying effects
• Constraints on behaviour
• Reforms

27
Conclusions

• Accurate measurement of output of public service
  providers problematic
• Capturing all elements of output
• Aggregating
• Measurement difficulties arising from individuals
  as inputs
• Deriving organisation-level measures of
  productivity could be valuable if can be
  demonstrated to be robust
• Help to understand what underlies productivity
  improvements
• Reforms to public sector services and use of
  pilot schemes present opportunities to pick up
  driving factors