OECD Statistical Quality Framework Paul Schreyer OECD

1
OECD Statistical Quality FrameworkPaul
SchreyerOECD

• Assessing and Improving Statistical Quality
• OECD/IMF Workshop
• Paris, 5-7 November 2003

2
Background (1)

• Improvement in the quality of OECD statistics is
  one of the main objectives of the current OECD
  Statistics Strategy launched by Enrico Giovannini
  two years ago.
• Until then, no formalised approach to quality
• However, informal quality assurance process
  already in place in several Directorates
• Treatment and validation of questionnaire replies
• Cross-checking with national publications
• Compilation of additional information from OECD
  and other international sources
• Referral of data queries and other quality issues
  back to source agency, interpretation, etc.

3
Background (2)

• OECD Quality Framework is internal tool
• Focuses on improving the quality of data
  collected, compiled and disseminated by the OECD
  through an improvement in the Organisations
  internal statistical processes and management
• Hopefully, positive spillovers on the quality of
  national data.
• OECD quality initiative is similar to those
  developed by Statistics Canada and other NSOs,
  which also encompass statistical managerial and
  technical processes.

4
Background (3)

• The OECD Quality Framework has four elements
• a definition of quality and its dimensions
• a set of broad principles on which OECD
  statistical activities are to be conducted and
  quality guidelines covering all phases of the
  statistical production process
• a procedure for assuring the quality of proposed
  new statistical activities
• a procedure for evaluating the quality of
  existing statistical activities on a regular
  basis.

5
This presentation

• Lists the quality dimensions of OECD framework
• Summarises principles of quality management as
  they arise from the OECD and from other
  frameworks
• Makes a few points about procedures

6
Dimensions of quality

• Quality Fitness for use
• Quality multi-dimensional concept
• Dimensions
• Relevance value contributed by data to users
• Accuracy degree to which data correctly
  estimate true value
• Timeliness length of time between event and
  availability of data
• Punctuality existence of and adherence to
  publication schedule

7
Dimensions of quality

• Accessibility suitability of media form,
  pricing delivery
• Interpretability/clarity information
  environment, metadata availability
• Consistency/coherence within, across datasets,
  over time and space
• Credibility/Transparency/Integrity the
  confidence that users place in data products
• Cost efficiency

8
Principles of quality management

• Achieving credibility
• Maintaining relevance of output
• Entertaining effective relationships with
  respondents
• Putting in place processes for high quality
  output
• Methodologies (e.g. supply-use framework)
• Information systems (software)
• Reviewing and evaluating statistical activities
• Hiring and keeping skilled and motivated staff

9
OECD procedures

• Self-assessment by database managers
• Review of results by representatives of
  statistics, publications and IT departments
• Review of results by peers, i.e., the group of
  OECD statisticians (Statistical Policy Group)
• Final conclusions and recommendations, sometimes
  presented to country representatives
• Allocation additional and re-allocation of
  existing resources to put in place
  recommendations
• As of today reviews of 4 databases completed