Composite Indicators

1
Composite IndicatorsData Envelopment Analysis

• Brussels, Oct. 24
• Tom Van Puyenbroeck

2
Structure

• Background why DEA?
• Essence of (formal) approach
• Incorporating (expert) opinion
• Extensions

3
Composite Indicators

• Indicators are observed (measured in different
  units)
• We want an overall score in order to compare
  countries
• but weights for aggregation are not known with
  certainty

4
Disagreement among experts proposed weights for
IMI
5
Classical DEA-problem

• Inputs and Outputs are observed (measured in
  different units)
• We want an overall expression of productivity in
  order to compare firms
• Productivity (weighted sum of outputs) /
  (weighted sum of inputs)
• but weights for aggregation are not known with
  certainty
• (production function, right prices, unknown)

6
 The typical composite indicator 

• For country c, given j 1,,m policy dimensions
• Xcj (normalised) variable of country c in
  dimension j
• wj weight . (Normally 0 ? wj ? 1 and sum wj
  1)

7
Towards DEA composite indicatorStep 1 

• Weighted sum of indicators country total
• Since the eventual purpose of a composite
  indicator is to compare (with other countries),
  we will express this country total RELATIVE TO
  A SIMILARLY WEIGHTED SUM of BENCHMARK
  SUB-INDICATORS

8
Towards DEA composite indicatorStep 2 

• Which benchmark should we choose?
• Look WITHIN SAMPLE of COUNTRIES for the one that
  yields the HIGHEST POSSIBLE TOTAL (given the
  weights)
• Best Practice notion

9
The DEA composite indicatorStep 3
Benefit-of-the-doubt weighting

• Which weights should we choose?
• CHOOSE the weights such that the evaluated
  country has a MAXIMAL COMPOSITE INDICATOR VALUE
• Benefit-of-the-doubt-notion

10
Advantages

• Qua model better description of reality than
  (e.g.) equal weighting
• Embedded concern for (MS) diversity no other
  weighting scheme yields higher composite
  indicator value (political acceptance)
• Principle is easy to communicate
• Since we are not sure about the right weights, we
  look for  benefit of the doubt  weights (such
  that your overall relative performance index is
  as high as possible)
• If ANOTHER country gets a higher overall score,
  using YOUR weighting scheme, we regard that
  country as outperforming you.

11
An aside normalising not needed

• A feature of DEA-models unit invariance
•  expressing an output in tons rather than kilos
  will leave final result unaffected 
•  expressing one output in tons, another one in
  litres, is perfectly possible 

• You need not worry (so much) about normalization
  stage
• It is well-known that rankings can be affected by
  normalization options in most other approaches!

12
Benefit-of-the-doubt expert opinion

• The BoD approach seems especially valuable when
  individual expert opinion is available, but when
  experts disagree about the right set of weights
• Foster and Sen  On Economic Inequality 
•  While the possibility of arriving at a unique
  set of weights is rather unlikely, that
  uniqueness is not really necessary to make agreed
  judgments in many situations, and may indeed not
  even be required for a complete ordering 

13
Example performance in the KBE

• Ingredients 5 sub-indicators
• These are transformed (z-scores)
• Recall NOT NEEDED in BoD
• Transformed scores are added, using a fixed
  weighting system
• Meaning ?

14
Aside 2 consensus on what?

• When experts give weights, how do they read these
  weights themselves?
• E.g. Budget Allocation-method,
• We propose to ask them about Pie Shares

15
Ex. 1 consenus on order of pie shares
16
Results for KBE-performance
17
Ex. 2 pie shares limited flexibility

• Example use above weights, /- 25

18
Results for KBE-performance
19
(No Transcript)
20
Extensions

• Fixed (set of) benchmark(s) rather than
  endogenous benchmark
•  Common  weights for all countries
• Dynamics
• Best practices move. Others slower/faster?
• Geometric index (?)

21
Further reading

• JRC/OECD Handbook on Composite Indicators (2005)
• Cherchye, Moesen and Van Puyenbroeck (2004)
  Journal of Common Market Studies
• Despotis (2005) Journal of the Operational
  Research Society (Common Weights)
• Cherchye, Moesen, Lovell and Van Puyenbroeck
  (2005) econpapers.repec.org/paper/WPEPapers/CES05
  13.htm (Dynamics)