Quality Indicators in Data Collection

1
Quality Indicators in Data Collection
Q2010 Helsinki Gustav Haraldsen Øyvin Kleven,
Anne Sundvoll gha_at_ssb.no, kle_at_ssb.no,
asu_at_ssb.no Department of Data Collection Statisti
cs Norway
2
Outline

• Strategy for data collection main objectives
• Quality management system (check lists and
  indicators)
• Components/indicators - examples
• Challenges in implementing the information system

3
Main objectives Strategy for data collection

• Motivated respondents
• Data collection of higher quality
• Identify strategic focus areas and key process
  indicators
• Measure effects of actions taken

4
Strategic focus areas
Professional support service
Me
Motivated respondents
Increased product quality
Adapt data collection to respondents Information
system and preferences
Reduce errors in processing
Develop and utilize quality indicators
Develop information system for data collection
5
Key process indicators
Paradata
Percentage electronic reporting
Me
Motivated respondents
Increased product quality
Rapid data reporting indicator
Check list/ process indicators
Complete data reporting indicator
Data collection specification form
Standard and new product quality indicators
Customer evaluation form
6
Examples key process indicators
Infrastructure performance indicator
7
Examples key process indicators
8
Rapid data indicators
Reminder Imposed fine
Amount of units reported before reminder (week
9) 51 Amount of units imposed compulsory fine
(week 19) 18 Amount of units followed up by the
state agency for the recovery of fines 4
9
Data Collection Information System (DCIS)
Expectations
Specification form
Customer evaluation
Documentation
Sample management
Product
Checklist
Response burdens and gratifications
Response process
Feedback
Accomplishment
Quality
Satisfaction
Design
Paradata
10
Order specification
Specification form

• Title, objectives data collection
• Client information
• Budget

• Sampling information

• Design
• Questionnaire development

• Post survey tasks
• Deliveries

11
Activities

• Declaration of the work being done in the steps
  of developing and collecting the data.
• Comprises 70 questions related to work done in
  the following processes
• Questionnaire design
• Sample design
• Training of staff (interviewers and support
  services)
• Data collection procedures
• Data reception
• 1st line data editing

12
Response burden
13
Product quality

• Response rate
• R-indicator
• S-indicator
• Item nonresponse
• Validity indicators
• Reliability indicators

14
Response rate
15
R-indicator
16
Response rate, net sample result and trend line.
Perceived health
17
Customer evaluation

• The delivery compared with objectives stated in
  the order
• Satisfaction with the communication during the
  project
• Satisfaction with the work done

18
Data Collection Information System (DCIS)
Expectations
Specification form
Customer evaluation
Documentation
Sample management
Product
Checklist
Response burdens and gratifications
Response process
Feedback
Accomplishment
Quality
Satisfaction
Design
Paradata