Composite Indicators for the Measurement of Economic Performance

1
Composite Indicators for the Measurement of
Economic Performance
P. Roberti F. Oropallo ISTAT Productivity,
Competitiveness and the New Information
Economy Business , Systemic and Measurement
Issues NESIS FP5 ISTAT Rome June 26, 2003
2
We live with too many Indicators

• A host of indicators that have been proposed to
  monitor this goal.
• They can be grouped in five areas
• General economic background
• Employment
• Economic Reform
• Social Cohesion
• Environment

• LISBON OBJECTIVES
• to become the most competitive and dynamic
  knowledge-based economy in the world, capable of
  sustainable economic growth ...and social
  cohesion

3
Indicators can beone-dimensional
ormulti-dimensional

• Composite indicators have received increasing
  attention in recent years.
• Various methodologies have been developed to
  handle aggregation and related problems
• Aggregation systems
• Deciding on the phenomenon to be measured
• Selection of sub-indicators
• Assessing the quality of the data
• Assessing the relationships between the
  sub-indicators
• Testing for Robustness and Sensitivity

4
State of the Art Methodological Issues

• A number of methodologies can be applied for the
  development of composite indicators. They
  include
• Multiple linear regression models
• Principal components analysis and factor
  analysis
• Cronbach alpha
• Neutralization of correlation effect
• Efficiency frontier
• Distance to targets
• Experts opinion (budget allocation)
• Public opinion
• Analytic Hierarchy Process
• JRC EC (2002) - State-of-the-art Report on
  Current Methodologies and Practices for Composite
  Indicator Development Joint research Centre
  European Commission - Institute for the
  Protection and Security of the Citizen
  Technological and Economic Risk Management
  I-21020 Ispra (VA) Italy - Prepared by the
  Applied Statistics Group June 2002

5
Is there a clear Framework yet?

• How can a framework be developed?
• By defining an analytical framework with precise
  properties!

6
Property of Indicators

• Micro founded
• Scope fulfilling
• Purpose oriented
• Well- behaved
• Consistent
• Decomposable
• Multidimensional - Composite

7
MICRO FOUNDED

• that is, based on a comprehensive database that
  embraces all aspects of enterprise features (ad
  hoc surveys can cover some aspects)
• Integration of different data sources of micro
  data
• Quality test of the integration process
  (matching procedures / estimation)
• In the second part of this presentation examples
  of the opportunities opened up by an Integrated
  Database are shown (Diecofis Project
  www.istat.it/diecofis Year of reference is
  1998-2000)
• Sources are (1) Structural Business Statistics
  (2) Administrative data (Foreign Trade,
  Commercial account, Fiscal and Social security
  data)

8
SCOPE FULFILLING

• Indicators can measure size, change and
  dispersion
• Changes in an Indicators value can have
  different causes and lead to different
  conclusions depending on underlying combinations
• Socio-economic phenomena have different and
  complex dimensions.
• Different indicators can serve different
  purposes, i.e. measure
• - heterogeneity/dispersion
• ?-performance (moving toward the mean)
• - overall systemic performance
• ß-performance (generalised move
  upward/downward)
• overall comparative performance (e.g. catching
  up/lagging behind)
• stratification (to evaluate differences in
  systemic structures and whether they represent a
  stratum - as with Yitzhakis decomposition)

9
PURPOSE ORIENTED

• Socio-economic phenomena may have many facets and
  change can result from a combination of different
  patterns
• Appropriate indicators may be needed in different
  circumstances

10
WELL-BEHAVED AND CONSISTENT

• Indicators inconsistency may arise for different
  reasons
• Reference to condition of
• - Lorenz dominance (focusing on relative
  differences)
• - Pareto superiority (focusing on levels)
• - stochastic dominance (focusing on both
  dimensions)

11
DECOMPOSABLE MULTIDIMENSIONAL

• To be able to
• study patterns
• take into consideration more than one
  dimension/aspects

12
Developments in poverty analysis are a good
example of the possible problems and avenues to
solve them
Number of people ? Income gaps ? Welfare
dimensions ? Multidimensional aspects Headcoun
ts ? Gaps ? FGT indices (squared gaps, etc.) ?
Multidimensional indices
13
Developing a similar framework is important for
the analysis of systemic performance and the
benchmarking of economic textures

• Since many factors and forces are at work to
  determine, condition and produce different
  outcomes.
• The quote that follows can serve to grasp the
  problem and possible approach to address it

14
The fact that many of the smaller EU economies
do either better or worse than the larger ones is
partly due to larger EU economies contributing
more to the overall EU mean than smaller
economies, which means that they are less able to
diverge from the mean. A second explanation is
due to structural conditions. The industrial
distribution of small economies is often
concentrated in a few sectors, while larger
economies are more diverse. This can shift the
scores towards the mean for many indicators in
large economies, while small economies can
exhibit either a high or low innovative capacity,
depending on the sectors that dominate the
economy. Of course, this shift towards high or
low technology sectors is not accidental, but
reflects both public and private institutions
seeking out areas of comparative advantage and
high profitability. (EU Commission)
15
The analytical framework and indicators that
can serve to study and benchmark different
levels/dimensions of systemic features

• Systemic maps (whole jigsaw)
• The overall aggregate picture EU level
• The national, regional, local picture
• The sectional picture
• The occupational picture
• Systemic strength and weakness, at a point in
  time (cross-section analysis) and overtime
  (longitudinal analysis)
• Map transitions features, patterns and
  evolution (New vs. Old)
• Systemic change and its features, at the
  aggregate/disaggregate levels

16
Decomposable indicator how

• The GINI index measures the concentration of a
  particular phenomenon (0no concentration,
  100maximum concentration). It can be divided
  into three elements

17
Distributions and Overlap
When overlap in the decomposition is high, it is
very hard to judge which group is the
best/worst, because distributions cross
18
Distributions and Overlap
When overlap in the decomposition is low it is
easy to determine which group is the best/worst
performer
19
Mono-dimensional Analysis of performance
(Overall)(GINI index calculated on exports)
20
Mono-dimensional analysis of performance
(Exports) (By NACE Sectors)
21
Multi-dimensionalanalysis of performance

• Decomposition for each dimension

Decomposition of the between component for each
dimension across K classes
22
To make composite indicators easy to use and
interpret

• Requires Normalization

Requires Aggregation
23
Benchmarking performance levels or gaps?

• How far away from best performers enterprises
  are?

24
Weighted Composite Gap

• How far away from best performers enterprises
  are?

25
Composite Indicator (Breakdown by NACE sector)

• Three dimensions
  ? One Dimension

26
NACE sectors
The Sectors in italic are defined ICT sectors
by OECD
27
Composite Indicator (Breakdown by NACE sector)

• Three dimensions of enterprises performance
  (1)Value Added (2)Employment (3)Exports
• a) one dimension-three
  areas

28
Composite Indicator (Breakdown by NACE sector)
b) 3 dimensions into one indicator
Short fall from top or average
29
Composite Indicator (Breakdown by regions)
Three dimensions of enterprises performance
(1)Value Added (2)Employment (3)Exports
a) 3 mono-dimensional
indicators
30
Composite Indicator (Breakdown by regions)
b) from 3 to one 3-dimensional indicator
Short fall
31
Composite Indicator (by size of firm)

• Three dimensions of enterprises performance
  (1)Value Added (2)Employment (3)Exports

Short fall
32
Conclusions

• The analysis which has been presented draws from
  research results under two related FP5 projects
  DIECOFIS and NESIS that deal with different
  mapping and benchmarking aspects.
• A strong investment in the design and development
  of a complex and wide ranging system of
  enterprise micro data which have been integrated
  and systematised into one single hub.
• The analysis is founded on micro-data drawn from
  the integrated and systematised enterprise SIS,
  which gives high flexibility and allows to
  aggregate and disaggregate indicators a la carte.
  

33
Thank You