Assessing the Capacity of Statistical Systems

1
Assessing the Capacity of Statistical Systems

• Development Data Group

2
Summary

• Overview of the assessment process
• Some tools and frameworks
• Assessing organization and management
• Indicators of statistical capacity building

3
Part 1 Overview of the assessment process
4
Assessing statistical capacity

• The statistical system
• Inputs
• Financial and human resources
• Legislative and regulatory framework
• Statistical and physical infrastructure
• Intermediate processes
• Statistical operations and procedures
• Organization and management
• Outputs
• Statistical products and services

5
Looking at outputs

• Assessing data quality
• The Data Quality Assessment Framework (DQAF)
• Data coverage and dissemination
• Comparison with international frameworks and good
  practice
• General Data Dissemination System (GDDS)
• Meeting users needs
• Balance between supply and demand
• Anticipation of new needs and demands

6
Intermediate processes

• Reviewing statistical operations and procedures
  (DQAF and GDDS)
• Appropriateness and correspondence with good
  practice
• Communications with providers and actions to
  reduce data burden and protect privacy
• Quality awareness and control
• Assessing management and coordination
• Financial management and control
• Human resource management
• Effectiveness of logistics

7
Inputs

• Financial and human resources
• Levels and trends in recurrent and development
  budgets
• Numbers and levels of skills/training
• Legislative and regulatory framework
• Compliance with fundamental principles
• Statistical infrastructure
• Adequacy of registers, sampling frames etc,
• Physical infrastructure
• Adequacy of buildings, computers and
  communications equipment

8
Part 2 Some tools and frameworks
9
Data Quality Assessment Framework

• Monitors the quality of economic and social data
• Quality of the statistical product
• Quality of the statistical agency
• Used by IMF for data part of Reports on Standards
  and Codes (ROSCs)
• Monitors extent to which observed procedures
  follow good practice

10
Coverage

• General DQAF as well as separate frameworks for
• Main economic statistics frameworks
• National accounts Balance of payments
  Government finance Money and banking Consumer
  price index
• Socio-demographic statistics (being prepared by
  World Bank)
• Income poverty (completed) Education Health
  Population (in preparation)

11
Structure

• Six dimensions of quality
• 0. Prerequisites of quality
• Integrity
• Methodological soundness
• Accuracy and reliability
• Serviceability
• Accessibility
• Hierarchical structure
• Dimensions
• Elements
• Indicators
• Focal issues and key points

12
GDDS

• Sets out objectives for data production and
  dissemination in four dimensions
• Data coverage, periodicity, and timeliness
• Quality
• Integrity
• Access by the public
• Provides a framework for development
• National authorities set their own priorities and
  timing to achieve their objectives

13
Participation

• Voluntary and involves three actions
• 1.  Commitment to use the GDDS as a framework for
  statistical development
• 2.  Designation of a country coordinator
• 3. Publication of metadata, descriptions of
• current statistical production and dissemination
  practices
• plans for short- and longer-term improvements
• need for support including technical assistance

14
Coverage

• Economic and financial data responsible
  agencies and main data series
• Real sector
• Fiscal sector
• Financial sector
• External sector
• Socio-demographic data responsible agencies and
  main data series
• Population
• Health
• Education
• Poverty

15
Part 3 Assessing the organization and management
of statistical agencies
16
One approach

• Effectiveness of a statistical system is
  determined by
• The products it produces and the services it
  provides
• Its functional and organizational structure
• Carry out a SWOT analysis of
• The internal organization
• The external environment in which the system
  operates

17
Internal organization

• Structure
• Coordination
• Human resources
• Infrastructure
• Management systems

18
External environment

• Statistical legislation and regulations
• Budgets
• Accountability and reporting
• Relationships with users
• Public image

19
Part 4 Indicators of statistical capacity
building
20
Assessing capacity

• 16 quantitative indicators
• Resources
• Inputs
• Statistical products
• 18 qualitative indicators
• Environment
• Core statistical processes
• Quality of statistical products

21
The quantitative indicators

• Resources
• Annual budget - recurrent and development,
  locally and externally funded
• Inputs
• Data sources censuses, surveys and
  administrative data
• Statistical products
• Media and topics covered

22
Using quantitative indicators

• Provide rough measure of extent of statistical
  activities
• Usefulness limited by
• Lack of benchmarks
• Do not measure efficiency or effectiveness
• Need to be interpreted using contextual
  information provided by qualitative indicators

23
Qualitative indicators

• Cover a broader view of factors determining
  capacity
• Based on DQAF Framework
• Six indicators on institutional prerequisites
• Two indicators on data integrity
• One indicator on methodological soundness
• Four indicators on accuracy and reliability
• Three indicators on serviceability
• Two indicators on accessibility

24
Coverage

• Legal and institutional environment
• Professional and cultural setting
• Methodological expertise
• Adequacy of data sources
• Analytical and processing capacity and quality
  control
• Relevance of products to users needs
• Effectiveness of dissemination

25
Measurement and recording

• Quantitative indicators use four point assessment
  scale
• Level 1 largely underdeveloped
• Level 2 developing but with observed
  deficiencies
• Level 3 moderately well developed
• Level 4 highly developed, in line with good
  practice