Management Tools for Enhancing the Composition of Survey Response

1
Management Tools for Enhancing the Composition of
Survey Response

• Q2008
• European Conference on Quality in Official
  Statistics
• Rome, July 2008
• Anne Sundvoll, Bengt Oscar Lagerstrøm
• Øyvin Kleven, Tora Löfgren,
• Statistics Norway

2
Outline

• A growing focus on how to improve response rates,
  minimize response bias and monitor fieldwork
  costs during the fieldwork period
• Combination of easily retrieved administrative
  data and process data make continuous monitoring
  of survey variable of interest possible
• The task is to maximize the result, given certain
  constraints as time or costs

3
SIV A system for managing multi mode data
collection

• Co-ordinates CATI, CAPI, CAWI and Paper
  Questionnaires
• Master Management system
• Updates, reports and graphs
• Quick and continuous re-scheduling of actions

4
Architecture example
CATI
In House Case Management
Blaise CATI Service
Deliver Cases
CAWI
Blaise Input Data Store
Blaise Data Server
Progress Report
CASI
In House Data Store
Extract Data
CAPI
Third Party Data
CADI
4
5
Examples of Administrative Data

• Mode(s)
• Data collection period and dates of actions
• Budget
• Number of Interviewers
• Interviewer characteristics

6
Examples of Process Data

• Response rate
• Composition of nonresponse
• Refusal rate
• Non-contact rate
• Telephone coverage
• Workload
• Cost

7
Management Tools - description

• Snapshots of the data collection process
• Intervention must be possible
• Transparent and predictable management system

8
Data we would like to monitor

• During the data collection process, we would like
  to monitor the composition of..
• Process data
• Auxiliary data
• Target variables

9
Examples of Quality Indicators

• Response rate
• Non-response rate
• Response bias
• Response burden
• Revisions
• Errors

10
Monitoring of Process Variables
11
Monitoring of Auxiliary Variables
12
Monitoring of Target Variables
13
Post stratification by background variables
14
Benchmarking by data from previous rounds
15
Examination of multiple bias
16
Management Tools - Challenges

• Get used to communicate through data displays
• High skilled project managers
• Demand of quick re-sceduling procedures

17
Management Tool - Limitations

• May draw attention away from serious measurement
  problems
• May indicate more than one issue of concern
• May add new bias

18
Further work

• Development of SIV
• Experiences from real-time data collections
• Development of more sophisticated management tools