Decade of Roma Inclusion Progress monitoring

1
Decade of Roma Inclusion Progress monitoring

• Challenges, problems and possible approaches
• Andrey Ivanov, UNDP

2
Why evidence is crucial?

• The Decade MDGs for one of the most vulnerable
  groups, the Roma
• Clear commitment from governments with strict
  time-frame set (2005-2015)
• Major priority areas - poverty, employment,
  education, health and housing
• National action plans for Roma Decades developed
  and Decade secretariats established in 2003-2005

3
The Decade as data challenge

• Intends to address the needs of fluid population.
  Unclear what to put in the denominator when
  computing various indicators
• NAPs need reliable estimate to allocate
  resources.
• Both sides governments and Roma often speak
  different languages. High level of mistrust
  (pre-divorce phenomenon)
• Susceptible to manipulation by both sides
• Government has incentives to report success
• Roma CSO have incentives to report failures

4
The approaches so far

• NAPs usually lack clear indicators. Input,
  output, outcome and impact often confused (not
  clear deliberately or not)
• The monitoring role taken over by
  non-governmental actors
• The Decade Watch published in 2007, assessment
  of the inputs to the Decade implementation as
  seen by Roma CSOs
• WB/OSI/UNDP Decade indicators working group
  working since 2007 on internationally comparable
  indicators
• UNDP two rounds of data collection (2002 and
  2004) and increasing supporting national-level
  efforts for NAPs implementation
• The current pattern may reinforce the mutual
  distrust of parties directly involved
  (governments and Roma civil society)

5
UNDP approach to Decade monitoring

• It is neither possible nor reasonable to invent
  and implement Roma indicators.
• For the monitoring of the Decade a standard sets
  of socio-economic and human development
  indicators should be applied
• Standard indicators must be fed with ethnically
  disaggregated data and the challenge boils down
  to availability of such data
• Indicators should follow the policy purpose
• Anything too simple is inevitably misleading
• Choosing between national adequacy and
  international comparability the former is a
  priority (following the MDGs pattern)

6
UNDP approach to data process

• National actors should be supported and not
  substituted in data issues
• International organizations can test the grounds
  but not take over
• There is a lot of data available, the challenge
  is to use it in creative way. Some possible
  approaches include
• Disaggregating hard statistics using personal
  identification numbers as a common denominator
  (key link) for mutually complementing data sets
• Disaggregating hard statistics using territorial
  tags as ethnic markers combining quantitative
  data with qualitative information. Gives estimate
  of the real needs (adequate for policy-making)
• Extending the samples of regular sample based
  surveys with ethnic boosters
• Custom on the spot surveys conducted among
  recipients of different social services
• Collecting data at a community level by
  Community-based data collectors and monitors
• Using measurable proxies that are strongly
  correlated with Roma identity to estimate
  progress in sectoral priorities of the Decade

7
UNDP approach to NAPs

• Consistently distinguish input, output, outcome,
  and impact indicators
• Put different focus on different levels of
  monitoring (central and local)
• Integrate the monitoring functions into the whole
  NAP implementation strategy (so far it is
  detached)
• Support local governments in data collection,
  data analysis and data application capacities
  (skills to understand and use data are even more
  important than the existence of data per se)
• Regularly update of the NAPs with ME components
  with necessary indicators
• Include Roma in the whole cycle
  (collection-processing-analysis-policy
  modification)

8
Comparing different approaches do data
disaggregation
9
Comparing different approaches do data
disaggregation
10
Conclusions

• Defendable indicators for progress monitoring are
  crucial both for policy design and policy impact
  assessment. Without such indicators the Decade
  may fail in its promises
• Disaggregating statistical data by ethnicity is
  possible even when exact number of Roma
  population is unclear. Constructing ethnically
  sensitive indicators is possible both national
  and internationally comparable
• Problems exist, however they are less of
  methodological, technical or financial but rather
  of political nature
• Given the concerns regarding individual data
  integrity, such disaggregation and construction
  of indicators should be done by specially
  appointed agency operating in line within clear
  legislation on the matter. National statistics
  are best suited for such a role.