Title: A Tale of Two Methods: Comparing mail and RDD data collection for the Minnesota Adult Tobacco Survey
1A Tale of Two Methods Comparing mail and RDD
data collection for the Minnesota Adult Tobacco
Survey III
- Wendy Hicks and David Cantor
- Westat
- Ann St. Claire,
- ClearWay Minnesotasm
- Rebecca Fee,
- Blue Cross and Blue Shield of Minnesota
- Peter Rhode,
- Minnesota Department of Health
2Overview of Presentation
- Report results of a feasibility study comparing
mail and telephone data collection methods - Seeming declines in RDD response rates (Curtin et
al, 2005) - Changes in coverage of the telephone population
(Blumberg et al, 2006, 2007) - Increasing costs associated with RDD surveys
- Findings mostly replicate those reported by Link
et al. (2006) - Some suggestion that an all adult selection
method in a mail survey results in biased
estimates for young adults (18-24 year olds)
3Methodological Approach
- Mail survey has small sample and large confidence
intervals - Contrast results found in a parallel mail vs.
telephone study for which mail study selected
individuals rather than addresses - Bring in similar findings from another pilot
study (Health Information National Trends Survey,
HINTS)
4MATS Overview
- Minnesota Adult Tobacco Survey (MATS)
- ClearWay Minnesotasm
- Blue Cross and Blue Shield of Minnesota
- Minnesota Department of Health
- Major objective of MATS collect a diverse set of
public health data about adult pop in MN,
focusing on tobacco and cigarette use - MATS samples from two frames
- RDD
- Blue Cross member list frame
5Overview of MATS 2007 samples by mode and frame
6USPS Mail Data Collection
- All adult respondent selection mechanism
- Mailing package included 4 identical
questionnaires and return envelopes - Each adult living at this address should
complete one of the enclosed questionnaires.
Please give one of the enclosed questionnaires
and an envelope to each person 18 years old or
older living at this address. - Some differences from the CATI instrument to make
it more suitable to paper, self-administration
7Response rates by frame and mode
8Nonresponse and coverage error by mode
- 9.3 weighted estimate of cell-phone only
households in the USPS survey - Generally, the demographic distributions for mail
and telephone respondents parallel one another - Overall, more missing data in mail than telephone
- USPS vs. RDD-only respondents (differences of 3
plus) - USPS has fewer HS grads, larger underestimate
relative to CPS - USPS has fewer 65, smaller overestimate relative
to CPS - USPS has fewer 1-adult households and more 4
adult households, closer distribution to CPS
9Nonresponse and coverage error by mode
- Similar story for the BC member list, generally
the distributions for mail and telephone parallel - BC-M had fewer high school grads and some
college - BC-M had more married than phone respondents
10Comparing survey estimates by mode
- Socially sensitive items (risk behaviors)
- Smoking prevalence,
- Binge drinking
- Factual items
- Exposure to media messages regarding tobacco use
- Workplace smoking policies
- Items assessing beliefs
- Harm in smoking an occasional cigarette
- Smoking increases comfort in social situations
11Estimates of risk behaviors
Hypothesis mail gt reporting of risk behaviors
12Percent current smokers, by age, mode General pop
p gt0.03
13Percent binge drinkers, by age, mode General pop.
14Percent current smokers, by age, mode Member
frame
15Percent binge drinkers, by age, mode Member frame
16More suggestive evidence . . .
- Health Information National Trend Survey (HINTS)
- Small pilot
- USPS frame
- National sample
- all adults selection method
- Replicated the smoking result Estimates for
18-24 year old mail respondents significantly
lower
17What about other types of items?
- Factual items
- Reported exposure to media messages
- Work place policy regarding smoking
- No differences by mode, for either frame
- No differences by mode for 18-24 years old
- Items assessing beliefs and attitudes
- Some support for age effect for one of the two
items
18Any harm in smoking an occasional cigarette? Gen.
pop.
19Any harm in smoking an occasional cigarette?
Member list
20Summary and Discussion
- Response rate roughly comparable
- Mail mode includes coverage of cell-only
population - Demographic comparison by mode-
- USPS (all adult) to RDD (next birthday) very
similar distributions, some differences in educ,
adults in HH, - Member list also very similar, differences as
anticipated - Some suggestion that 18-24 year olds not
representative for USPS all adult method - Lower reports of risk behaviors in mail mode
21Household type for 18-24 year olds, by mode
- In USPS sample
- Almost 80 live in family-type households
- About 6 live alone
- About 3 live in un-related households
- Only 13 of 18-24 year olds were cell-only
households - In the RDD sample,
- Almost 65 live in family-type households
- About 4 live alone
- Almost 25 live in un-related households
22Next steps
- Incorporate methods to broaden type of young
adult respondents - Advance letter
- Incentive
- Fed-ex the second package
- Explore other respondent selection mechanism
- Next birthday method
- Other methods