What makes indicators successful? Lessons from practitioners

1
What makes indicators successful? Lessons from
practitioners

Funded by
2
The Project

• BRAINPOoL (Bringing alternative indicators into
  policy) is an EU-funded project aimed at
  identifying and overcoming the barriers to
  Beyond GDP indicators being used in policy.
• During the project we are carrying out research
  and interviews, conducting workshops and
  knowledge-brokerage seminars and carrying out
  various action research case studies to explore
  ways to improve uptake of Beyond GDP indicators.

3
The Project (work package structure)
4
Understanding the supply of Beyond GDP indicators

• Work Package 1 outcomes
• Catalogue of over 100 Beyond GDP indicator
  initiatives
• Understanding of intentions of indicator
  producers/promoters
• Documenting of impact, including media impact
• Fact sheets for 16 indicator initiatives
• Understanding of success factors for Beyond GDP
  indicators.
• Report available at www.brainpoolproject.eu/resea
  rch

This presentation
5
The initiatives studied
Domestic Material Consumption Happy Life
Years OECD Handbook of Subjective Well-Being UN
Commission for Sustainable Development
6
Success factors
Policy / Context factors
User
Indicator
Indicator factors Salience Legitimacy Credibil
ity
Relationship factors
User factors
From the perspectives of
The factors that determine the extent to which an
indicator is used by users cluster into four
categories (the light blue spheres). Of these,
indicator factors need to be understand from
three different perspectives.
7
Indicator Factors salience for policy makers
aenimation
8
Indicator Factors salience for policy makers

• Fit with a vision or organisational strategy
  this is particularly relevant for those
  initiatives promoting new indicators so as to
  shift priorities or assess progress differently.
• Measure things that can be influenced by policy
  this can be problematic for alternative
  indicators seeking to measure overarching
  concepts such as progress or well-being.
• Low-cost or money-saving e.g. minimal expensive
  data collection, providing clues for low cost
  policies or ways to save money.

9
Indicator Factors salience for policy makers

• Links with other outcomes links between what the
  indicator measures and other outcomes (e.g.
  subjective well-being being related to reducing
  staff turnover).
• Reaching multiple audiences this can ensure
  indicators do not sit within particular silos and
  can achieve cross-cutting outcomes.
• Perceived need this is particularly important
  where initiatives are bringing together data
  rather than creating new measures.

10
Indicator Factors salience for public/broad
audience
Knivesout
11
Indicator Factors salience for public/broad
audience

• Simplicity initiatives are effective when they
  allow one to produce a simple and attractive
  message.
• Ease of understanding while what they measure
  may be complicated, successful indicators manage
  to illustrate a complex reality using
  understandable concepts.
• Engagement with communications experts close
  collaboration, rather than simply handing over
  data, can ensure that both communicability and
  accuracy are maintained.
• Avoiding taboo words in the UK happiness is
  considered woolly or unscientific, while in
  the USA, practitioners have avoided the mention
  of climate change instead referring to air
  quality.

12
Indicator Factors Credibility
GuySie
13
Indicator Factors Credibility

• Data quality a particular concern was whether
  subjective well-being data changes over time.
• Concerns regarding composite indicators
  concerns over methodology and the weighting of
  different components of composite indicators can
  elicit strong resistance. The OECDs Better Life
  Initiative has managed to secure acceptance of a
  composite indicator by allowing users to decide
  for themselves how to weight the different
  dimensions of the measure

14
Indicator Factors Legitimacy
United Nations Photo
15
Indicator Factors Legitimacy

• Being (or appearing) neutral some indicator
  initiatives work within a framework of simply
  providing neutral information, while others are
  clearly connected to political agendas, such as
  social cohesion or respecting environmental
  limits. Mechanisms used to ensure neutrality
  included monitoring funding mix and barring staff
  involvement in political parties.
• Institutional power governmental bodies or
  supra-governmental bodies like the Council of
  Europe and the OECD often carry greater
  legitimacy than NGOs.

16
Relationship and process factors
Relationship and process factors
Rabanito
17
Relationship and process factors

• Engage ones audience from the start
  fundamental to the success of local initiatives,
  it was also seen in terms of getting
  policy-makers involved in large-scale
  initiatives.
• Direct contact with audiences while not all
  initiatives can or want to engage their audience
  from the beginning, all the most successful
  initiatives had direct contact with the people
  they were trying to influence.
• Small is beautiful to date, local initiatives
  have been able to achieve more impact than
  larger/national ones, with local bodies tending
  to be more flexible and responsive.

18
Relationship and process factors

• Partnership working aside from allowing a
  greater network to be reached and a greater skill
  base to be marshalled, partnerships allow
  different organisations to take on different
  roles. This can ensure an initiative is not too
  associated with a particular agenda.
• Picking ones audience On the one hand, some
  initiatives worked with individuals within
  organisations who could be seen as allies, or
  organisations who are overall supportive. On the
  other hand, several initiatives highlighted the
  need to reach those bodies potentially least
  sympathetic to their initiative ministries of
  finance, treasuries or economic departments.

19
User factors
AEN Foto
20
User factors

• Users capacity to use social and environmental
  indicators Beyond GDP initiatives typically
  involve a rebalancing towards social and
  environmental indicators and away from economic
  ones. This is not just a matter of calculating
  different things but of grasping different
  disciplines, and valuing different academic
  perspectives.
• The OECDs approach of using economic techniques
  with subjective well-being may be a fruitful
  technique for convincing economists, by using
  their own language.

21
Policy and context factors
Ryan Fitton
22
Policy and context factors

• The Stiglitz/Sen/Fitoussi Commision Seen as the
  biggest positive factor.
• The economic crisis - On the contrary, the
  financial crisis is seen by many as hindering
  Beyond GDP efforts, leading people to view
  well-being as a distraction. Others, however, see
  an opportunity in highlighting the role of the
  fixation on GDP in causing the crisis.

23
Policy and context factors

• Ideology as a barrier For example, subjective
  well-being has been criticised from a libertarian
  perspective as not being something government
  should influence.
• Vested interests might Beyond GDP efforts have
  a negative effect for certain groups?
• Public pressure support for the idea of
  alternative indicators required from the bottom
  up.
• Indicator initiatives take time a last sobering
  lesson is that it can take generations for an
  indicator to become sufficiently embedded in the
  system to maximise its impact.

24
For the full report, visit www.brainpoolproject
.eu/research For more information please
contact Alistair Whitby, World Future
Council alistair.whitby_at_worldfuturecouncil.org
Saamah Abdallah, nef (the new economics
foundation) saamah.abdallah_at_neweconomics.org
Tomas Hak, Charles University Environment
Centre tomas.hak_at_czp.cuni.cz
James Jordan