Integrity, independence and accountability: The European Statistical System Peer Reviews

1
Integrity, independence and accountability The
European Statistical SystemPeer Reviews

• Martina Hahn, Solveiga EidukynaiteEurostat
• Statistical governance, quality and evaluation
  Unit
• 11 July 2008

2
Content of the presentation

• Quality framework for the ESS peer reviews
• Quality assessment of the ESS peer reviews as an
  audit
• Conclusions

3
1. Quality framework for the ESS peer reviews
Peer reviews audit of the compliance status
with the Code of Practice
in line with the ISO 190112002 definition of an
audit
pre-definedcriteria
obtaining evaluatingaudit evidence
systematic
independent
documented
4
2. Quality assessment of the ESS peer reviews
as an audit (1)
Principles of performance auditing by the
International Association of Supreme Audit
Institutions

• Significance (importance of the matter)?
• Areas of the CoP covered crucial for
  independence, integrity, credibility
• Other areas of the Code could be better assessed
  using different tools

• Reliability (accurate reflection of actual
  conditions)?
• Findings depend on the information provided by
  the NSI
• Statistical analysis confirms validity of results
  (correlations, coherence, principle 3 best
  predictor for overall results)
• Indicator differ in suitability for supporting an
  audit

5
2. Quality assessment of the ESS peer reviews
as an audit (2)

• Objectivity (impartial and fair review)?
• Matching of PR teams
• Independent review
• Low neutralisation effects within teams
• Stricter evaluation during the peers second or
  later review
• Eurostat comprehensive quality assurance for
  whole process

• Scope (are all elements needed addressed)?
• Peer reviews conducted on a strategic,
  organisation-wide basis
• Other national producers of European statistics
  not included

6
2. Quality assessment of the ESS peer reviews
as an audit (3)

• Timeliness (of reports)?
• Average reporting rate 2.5 months
• Last report published on the Eurostat website
  April 2008

• Clarity (of reports)?
• Reports concise and clear following a common
  structure
• Summary reporting needed to address main
  stakeholders

7
2. Quality assessment of the ESS peer reviews
as an audit (4)

• Efficiency (resources reasonable for
  significance/complexity)?
• 51 peers (31 SPC, 18 Eurostat, 2 IOs)
• impressive results during relatively short time
  (3 days preparation and reporting), heavy
  investment on both sides but positive
  cost-benefit assessments

• Effectiveness (appropriate response from
  addressees)?
• auditee agreed improvements, momentum for
  boosting implementation of the Code
• EP/Council response from governance authorities,
  enhanced credibility?

8
3. Conclusions

• ESS peer reviews contribute to enhancing ESS
  integrity, independence and accountability
• carried out in line with criteria of a high
  quality audit
• signalling that the self-regulatory approach
  works
• fostering themselves the implementation of the
  Code
• Some trade-offs to consider
• Comparability Independence
• Reliability Burden/Costs
• Scope Burden/Costs
• Transparency Credibility (?)
• Need to review certain (few) indicators of the
  Code
• Refrain from mere quantitative interpretation
  and from country rankings