Workshop 4c Data Collection

1
Workshop 4cData Collection

• Stephanie J. Battles
• Energy Information Administration
• Dhaka, Bangladesh
• September 22, 1999

2
Data Collection

• Mail Survey
• Personal Interview
• Telephone Interview Computer-Assisted Telephone
  Survey (CATI)
• Other Modes of Data Collection
• Multiple Modes of Data Collection
• Survey Supplement Using A Subsample
• Nonsampling Error
• Nonresponse
• Response

3
Mail Survey(Self-Reported Data)

• Advantages
• Cost Effectiveness
• Small Staff
• Same Mailing Cost for 1 Question as it Costs for
  100
• Easily Target Population of Interest
• Studies Show That More Answers Are Provided

• Disadvantages
• Lower Response Rates
• Need to Have Good Planning
• Need Complete Mailing List
• No Opportunity for Follow-up
• Need Questions that are Simple and Short
• Legible/Easy to Follow
• Correct Language
• Can Respondent Read?

• Design and Format Crucial
• Future--CSAQ

4
Mail SurveyExample 1998 MECS

• Three Different Forms, Forms A,B, and C
• Form A Collects Basic Data
• Form B Petroleum Refining
• Form C Establishments Who Ship Energy Offsite
• 18,000 Establishments
• Average Burden per Response--8.24 Hours
• U.S. Census Bureau--Contractor
• Two Mail Follow-Ups
• Additional Telephone Follow-Up for the Largest
  Establishments Not Responding

5
Personal Interview(Face-to-Face)

• Advantages
• High Response Rates Are Possible
• Survey Questions Can be More Complex
• Opportunity for Follow-Up Questions
• Can Assist Respondents Who are Disabled, Aged or
  Other Problem

• Disadvantages
• Costly
• Need a Large Staff or Contract Personnel
• Natural Disaster
• Lose Portion of Sample
• May Not Get Person Most Capable of Answering the
  Questions
• Sample May Be in Restricted area

6
Personal Interview (Face-to-Face)

• Interviewer Training (Classroom/Self-study/Both)
• Introductory Letter Before Initial Contact
• Purpose of the Survey
• Why Participation is Important
• Confidential Responses
• Exhibit Cards
• Multicontacts
• Callbacks at Different Times of the Day
• Multiwave
• Reassigned Different Interviewers
• Use of Incentives

7
Personal Interview Example-1997 RECS

• Trainer Training
• Three Days
• Interviewer Training
• Three Days (2 Sessions)/220 Interviewers
• Washington D.C.
• Topics Covered
• General Interviewing Techniques
• RECS Background
• Introduction to the CAPI system
• Practice With Mock Interviews
• Review of Administrative Requirements

All Training Sessions Were Monitored by EIA Staff
8
Personal Interview Example-1997 RECS

• Training Manual
• Locating Exercises
• Identifying Respondents and Getting In the Door
• Mock Exercises (one person is the
  interviewer..the other is the respondent)
• Instruction Manual
• Administrative Procedures
• Identifying RECS Households
• Conducting the Interview
• Question by Question Discussion
• Training Used Computer (CAPI)
• Letter Sent Before Initial Contact

9
Personal Interview Example-1997 RECS

• Data Collection Results
• Average Time 29 Minutes
• Interviewers Conducted an Average of 27
  Interviews
• Median Number of Visits--3
• 20 of the Interviews Were Were Verified by Mail
  or Telephone
• Used Multicontact Approach
• Different Time of Day
• Used Multiwave Approach
• Changed Interviewer
• Mail Follow-Ups for Nonresponsive Households
• Short Self-Administered Questionnaire
• Sent Pen as an Incentive

10
Personal Interview Example-1997 RECS

• Data Collection Response
• 8,319 Listed Units--7,285 Eligible
• Ineligible
• Out of Scope, e.g. Business, Not Habitable
• Vacant or Seasonal
• 5,902 Completed (81 Response Rate)
• 5 of the Personal Interviews Completed After the
  Second Wave (299)
• 3 of the Interviews Completed After the Mail
  Follow-up
• 1,421 Households--Returned 10 (181)

11
Personal Interview Example-1997 RECS
Response Rates and Household Characteristics

• High Response Rates
• South (81.1)
• Midwest (80.2)
• Rural Areas (85)
• Single Family and Mobile Homes (80.5)

• Low Response Rates
• Northeast (73.1)
• Urban/Suburbs (76.4)
• Large Apartment Buildings (72.1)

12
Telephone Interview(CATI)
Most Telephone Surveys are CATI
(Computer-Assisted Telephone Interviews)

• Advantages
• Increase Quality
• Speed
• Programmed to Correctly Skip Automatically
• Edit Replies
• Produce Statistics Quickly
• Eliminates the Keying of the Data

• Disadvantages
• Costly
• Savings When the Survey is Large and/or Frequent
  and Questions Remain the Same
• Lower Response Rates
• Answering Machines
• No Telephone
• Up Front Costs
• Programming

13
Other Modes of Data Collection
BLS Current Employment Statistics

• Touchtone Data Entry (TDE)
• Alternative to Mail Collection
• Questionnaire is on a Computer
• Respondent Answers Prerecorded Questions Using
  the Touchtone Buttons
• Useful for a Few Numbers
• Eliminates Key Entry
• Voice Recognition (VR)
• Recognizes Numbers 0 to 9, Yes and No
• Useful for A Few Numbers
• Accent Recognition Problem

Banking
14
Other Modes of Data Collection

• Facsimile Transmission (FAX)
• Eliminates Mail Delays
• Electronic Data Interchange (EDI) Collect Large
  Volumes of Data
• Electronic Transmission is in a Standard Format
• Errors are Reduced
• Data are Instantly Available
• Email Transmission
• Paperless Collection
• Eliminates Mail Delays
• Errors/Costs Are Reduced
• Confidentiality Issues

EIAs Financial Reporting System
15
Multiple Modes of Data Collection

• Survey Requires the Collection of Data from
  Different Respondent
• Increase Survey Response Rates
• Respondents May Favor One Mode Over Another
• Certify Data Collected Are Correct
• Reduce Costs

16
Multiple Modes of Data CollectionExample 1993
RECS

• Personal Interview
• Household/Housing Unit Data
• Authorization Form
• Mail Survey
• Household Mail Survey
• Obtain Data for Households Who Refused to
  Participate in the Personal Interview
• Energy Supplier Survey
• Natural Gas, Electricity, LPG, and Fuel Oil
• Obtain Energy Consumption and Expenditure Data
  from the Households Energy Suppliers

17
Multiple Modes of Data CollectionExample 1993
RECS

• Telephone Survey
• Rental Agent Survey
• Verify Information Received From Certain
  Households Who Did not Pay for All Their Energy
  or Paid A Third Party and Rented or Owned Living
  Quarters in A Multi-Unit Building

18
Survey Supplement Using A Subsample

• Obtain Detailed Data
• Use if Too Costly for the Full Sample
• Use if Response Burden is a Concern
• Use Data to Develop a Model Based on Data
  Available From the Full Survey

19
Survey Supplement Using A SubsampleExample 1993
RECS Lighting Supplement

• Purpose of the Supplement
• Collect Data to Develop a Model to Predict
  Electricity Consumption for Lighting
• Proportional Sample of All U.S. Households
• 7 of the Completed Interviews
• Lighting Model Used in Electricity Regression
  Equation Along Other End-Use Models, e.g. Space
  Heating.

20
Nonsampling ErrorUnit Nonresponse Error
Unit Nonresponse Error Unit nonresponse error
occurs when a sampling unit does not participate
in the survey , e.g. household, building or
establishment.

• Unit Nonresponse
• Refusal/Failure to Return Questionnaire
• Sensitive Information
• Incorrect Address
• Not Have Resources or Time To Reply
• Not Interested
• Not Like the Sponsor
• Could Not Be Located
• Natural Disaster
• Recently Vacant/Destroyed
• Telephone Disconnected

21
Nonsampling Error Unit Nonresponse Control of
Error

• Sampling Units Record Carefully Labeled and
  Transmitted
• Prior Notice
• Follow-Up
• Incentives
• Questionnaire Design
• Confidentially Ensured
• Reweighting the Sample

22
Nonsampling ErrorItem Nonresponse Error
Item Nonresponse Error Item nonresponse error
occurs when a particular item of interest is
missing from a questionnaire.

• Sensitive Question
• Confidentiality Issue
• Recall
• Aged or Disabled
• Fatigue/Burden
• Question Long and Complex
• Open Question
• Records Not Available
• No Advance Notice
• Interviewer Error
• Programming Problems

23
Nonsampling ErrorItem NonresponseControl of
Error

• Interviewer Training
• Confidentiality Reminder
• Use Memory Aids and Cues
• Question Shorts and Simple
• Questionnaire-Reasonable Length
• Personal Interview, e.g. 60 Minutes
• Understandable Skip Patterns
• Limit Open Questions
• Imputation

24
Nonsampling Error Response Error
Response Error Response error occurs when the
item response is answered incorrectly.

• Satisficing Answer
• Doesnt Care/Time Burden
• Inability to Answer--A Guess
• Deliberate Incorrect Response
• Sensitive Question
• Respondent Knows the Correct Answer
• Invalid/Out of Date Records
• Interviewer Incorrectly Records Response
• Interviewer Bias

25
Nonsampling ErrorResponse ErrorControl of Error

• Importance of the Survey and Accurate Responses
• Use Memory Aids/Examples
• Ensure Confidentiality of Responses
• Advance Notice
• Interviewer Training
• Locate the Respondent With the Proper Knowledge
• Quality Control of the Interviewing Process
• Allow Respondents to Ask Questions