Standard Quality Indicators SQIs of statistical process and product.

1

Measuring statistical quality at the Spanish
National Statistical Institute
C. ARRIBAS, D. LORCA, A. SALINERO A. COLMENERO
2
SUMMARY

• 1.-Global Quality assessment of all surveys at
  NSI
• 2.-Implementation of systematic quality
• management
• 3.-Identification calculation of SQIs of
  outputs processes
• 4.-Production of SQRs of outputs(users
  oriented) and processes (internal uses)
• 5.-Conclusions

• Quality

3
1. Global Q assesment of all Surveys

• 1.-Calculation of synthetic Indicator of Global Q
  of the survey
• 2.-Calculation of synthetic Indicator of Global Q
  for survey output and process
• 3.-Calculation of synthetic Indicator of Q
  criteria of outputs or processes phases
• 4.-Calculation of SQIs of outputs and processes

3
4
2. Implementation of systematic quality
management

• I.-Objectives monitoring in a continuous
• basis the quality assessment of the most
  important statistical surveys
• Review the statistical processes and outputs
• Identify and Evaluate Standard Quality Indicators
• Produce Standard Quality Reports
  

5
2. Implementation of systematic quality
management

• II.- Procedures followed
• Self-assessment by Survey Managers
• (using DESAP, european checklist)
• Audit teams for checking
• Information provided by Survey managers
• Evaluation results of SQIs of processes outputs
• Content and presentation of SQRs of processes
  outputs in order to harmonize the format

5
6
3. Identification Calculation of SQIs of
output processes

• I.- Identification
• SQIs of outputs
• Selected set of Indicators, defined by
  Eurostat and based on Q criteria
• SQIs of processes
• Selected set of Indicators, defined by NSI and
  based on processes phases

7
EUROSTAT Q criteria

• Relevance
• Accuracy
• Timeliness Punctuality
• Accessibility Clarity
• Comparability
• Coherence

7
8
TABLE 1.- STANDARD QUALITY INDICATORS FOR
STATISTICAL OUTPUTS
8
9
NSI set of selected process phases

• Sampling Frame
• Coverage
• Data Collection
• Response Burden
• Editing Imputation
• Sample Estimation
• Documentation

9
10
TABLE 2.- STANDARD QUALITY INDICATORS FOR
STATISTICAL PROCESSES
10
11
3. Identification Calculation of SQIs of
output processes

• II.- Calculation
• 1.-DESAP checklist, was filled for all surveys by
  Survey
• managers and the answers were audited
• 2.-Link each SQI to one or several DESAP
  questions
• 3.-Associate the answers to a measurement scale
• for each SQI weighted average of selected
  questions is obtained
• for each Q criteria or process phase weighted
  average of selected indicators is obtained
• Global Q for output or statistical process is
  obtained as the weighted average values of Q
  criteria or process phases

11
12
3. Identification Calculation of SQIs of
output processes

• II.- Calculation (cont)
• 4.-Measurement scale represent Q degrees for
  outputs and processes
• 5.-Transformation rules transform original
• answers into values or scores of
  the
• measurement scale
• the scale is the same to all questions of the
  Indicator
• the transformation rule of questions into scale
  scores must be coherent, so they can be comparable

12
13
4. Production of SQRs of outputs processes

• Grouped 33 main surveys
• I. Short-term statistics and indicators
• II.- Structural Surveys
• III.-Household Surveys
• Survey managers produced
• SQRs of outputs users oriented
• I.-Description of the Survey
• II.-Q Evaluation (based on Q criteria)
• SQRs of processes internal uses for top
  management and improvement actions
• I.-Description of the Survey
• II.-Q Evaluation (based on process phases)

14
5. Conclusions

• Implementation of systematic Q management using
  the same tools methods to assess Q of surveys
• Survey analysis was obtained
• Numerical values of SQIs of outputs processes
• Permit compare Q between different surveys
• Permit compare Q changes over time
• Calculation system permits introduce Q
  improvements and obtain the new numerical value
  of SQI automatically
• The SQRs approach the NSI to users
• enable users a
  better understanding and
• use of survey data

15
5. Conclusions

• Possibility to repeat the exercise (review the
  whole
• process) in some years time
• Selection of particular domains for improvements
• Standardization of the documentation of the
  surveys
• Improve and introduce generalized systems for
  data editing and imputation
• Improve dissemination of survey data
• Identify best practices and disseminate them
  among staff
• Concluding remark Statisticians should quantify
  social and economic features of a society.
• Our first duty should be to try to measure, in
  numerical values, the Q of our own work.

15