Surveying Cell Phone-Only Canadians: Looking at the Possibilities

1
Surveying Cell Phone-Only Canadians Looking at
the Possibilities

• Presentation to the
• Marketing Research and Intelligence Association
• Ottawa Chapter
• February 19, 2009

2
Why the research was conducted- presence of
cell-only households- higher incidence among
younger people, men and single- person
households- effect on representativeness of RDD
samples- part of wider study on the use the new
technologies in communications
3
Who conducted the researchThe sampling frame
development ASDE Survey SamplerThe sampling
and interviewing Ekos Research AssociatesThe
analysis Arcturus Research SolutionsThe
client The Public Opinion Research Directorate
of Public Works and Government Services Canada
4
The current situation- 14 of households in
the US and 5 in Canada are cell-only- The
trend in Canada and especially the US is up
(see citation below)- cell-only are
disproportionately - one-person unattached -
found in large urban areas - 18 to 34 yrs -
below-average incomeSource Statistics Canadas
Residential Telephone Survey and Ambrose, Don
Gray, Derrick and Halpenny, Gary Follow-up on
cell-only households a growing concern for
telephone surveys. VUE, January 2008 pages
16-18
5
The current situation (continued) RDD Surveys
that exclude cell-only households could encounter
the following- increasingly incomplete
sampling frame - non response bias (young
adults, single-person households)If present
trends continue, the problem is likely to get
worse
6
Comparison of Coverage of the PopulationTelephon
e and the InternetTelephone RTSS Survey,
Statistics Canada 2006 Internet New
Technologies and Government of Canada
Communications, 2007
7
How the Cell Phone Only (CPO) sample was developed

• Randa Bell
• Vice President
• ASDE Survey Sampler
• February 19, 2009

8

• Discussions with clients at 2007 MRIA Conference
  in Collingwood
• Following growth closely in US
• 5 of Canadian Households are CPO
• (Stats Can December 2006 Report)?

9
Idea comes to life!
10
Some numbers

• Cell phones first appeared in Canada around 1983
• Today, upwards of 67 penetration
• 3,000 exchanges reserved for cell phone only
  (Telcordia)?

11
Some numbers (continued)?

• Each exchange can contain up to 10,000 numbers
  potential of 30 million
• Estimate each exchange to be populated at 50
  which means 15 million cell phones to be reached
  in Canada

12
The process

• Huge capacity of ASDEs IVR System
• Generated numbers at random in cell phone
  dedicated exchanges
• Proportionate to population of Canada by province

13
The process (continued)?

• Programmed and recorded survey with one question
  only
• Nature of cell phone we were reaching CPO
  household or in addition to landline
• Simplified wording

14
The questionnaire

• 20 seconds, National Study
• Only one question, nothing to sell
• Phone line which is not cell phone PRESS 1
• Cell phone only household PRESS 2
• Reimbursement of call PRESS 3
• Repeat question PRESS 9

15
The results

• Two very useful lists
• Any cell phone that connected
• CPO households (most valuable list)?
• PWGSC chose sample from each list

16
How to incorporate cell only samples into RDD

• Dual frame sampling method
• Portion from cell phones
• Portion from RDD
• Augment traditional sample with cell phone
  numbers (no overlap)

17
How to incorporate cell only samples into RDD

• Include proportion of CPO households (7) in
  sample
• Possibly assign quota on completed interviews vs.
  starting sample

18
Research Approach

• A text message was sent on January 9, 2008 to
  2,391 numbers provided by ASDE Survey sampler
• English message was The Government has hired
  EKOS to conduct a study on internet-based
  technologies and their use in Canada. You are
  invited to participate.
• Average length of interview 12.8 minutes
• Interviews conducted January 14 to 28, 2008

19
The Government of Canada New Technologies in
Communications Study the three modes
20
Early Termination of InterviewsQuestion To
begin, we would like to confirm that you are not
currently operating a motorized vehicle or
machinery of any kind such as a car, truck, van,
all terrain vehicle, snowmobile, etc? n293
21
Terminations owing to Presence of Landline

• 70 interviews terminated because they have
  landline in the household

22
Recall of Text MessageQuestion Do you recall
receiving a text message on your cell-phone or
wireless device from our company in the past five
to seven days notifying you that we would be
phoning you to complete an interview?Please
Note All survey respondents who wish will be
sent a gift certificate for 10.00 to either
Chapters-Indigo or Tim Hortons, their choice, as
a token of our appreciation. n203
23
Preferred IncentiveQuestion As we mentioned
at the beginning of the survey, we would like to
send you a 10 gift certificate to either
Chapters-Indigo or Tim Hortons for taking part in
this surveys, would you like to receive a 10
gift certificate? n203
24
The Call Disposition
25
The Cell-Phone Interview Situation Response to
question, While completing this interview, were
you alone or with other people? (n203)
26
Cell Phone Ownership Response to question, Are
you the sole owner of this cell phone or is its
shared with at least one other person 16 years of
age or older? (n203)
27
Location of the Cell-Phone Interview Response to
question, When we contacted you, were at work
or schoolat homeoutdoors? (n203)
28
The Sample by Gender(n203)
29
The Sample by Age(n203)
30
The Sample by Education(n203)
31
The Sample by Household Income (in 000)(n203)
32
The Sample by Number of Persons in Household
(n203)
33
Internet Use Cell-only versus Landline
Respondents, by AgeQuestion Do you personally
use the Internet? This includes using
e-mail.(cell-only, n203 landline, n1718)
34
Mean Hours per Week on the Internet Cell-only
versus Landline RespondentsQuestion ln a
typical week, how many hours do you spend on the
Internet, not including the time you spend
sending or receiving e-mails? Please include all
locations, such as home, work, school, etc
(cell-only, n172 landline, n1,314)
35
Reading or Contributing to Blogs Cell-only
versus Landline RespondentsQuestion How
frequently, if ever, do you read or contribute
to a blog? (cell-only, n168 landline, n1203)
36
Participating in Social Network Sites Cell-only
versus Landline RespondentsQuestion How
frequently, if ever, do youparticipate in a
social networking site like Facebook or
Myspace?(cell-only, n168 landline1,271)
37
Using Sites like YouTube Cell-only versus
Landline RespondentsQuestion How frequently,
if ever, do youview, post or download videos
from sites like YouTube?(cell-only, n168
landline, n1340)
38
Mean Ratings for Web 2.0 Reliability compared to
Traditional Media Thinking about the different
Internet-based applications and technologies that
we have been talking about, do you think that the
information available on these sources is more or
less reliable than information obtained through
traditional media, such as television, radio or
print? Please use a 5-point scale where 1 means
much less reliable, 5 means much more reliable
and the mid-point 3 means the same level of
reliability.(cell-only, n168 landline1,292
online, n1,216)
39
Significant differences between the Cell-Phone
and Landline Samples Demographics(significant
at the a.05 level)
40
Significant differences between the Cell-Phone
and Landline Samples Attitudes(significant at
the a.05 level)
41
Significant differences between the Cell-Phone
and Landline Samples Behaviour(significant at
the a.05 level)
42
Preference for Survey Mode Cell-Only
SampleQuestion If you were invited to take
part in a Government of Canada public opinion
survey, which method would you prefer completing
an online survey or taking part in cell-phone
interview, or you have no preference?
43
Likelihood of Participating in Another Cell-Phone
SurveyResponse to question, How likely are you
to agree in the future to participate in a
Government of Canada survey over the cell phone
very likely, somewhat likely, not very likely or
not at all likely?.
44
Reasons Given for Not Participating in Future
Cell-Phone Survey (number of respondents)(Among
persons who said not very likely or not at all
likely) Response to question, Why do you say
that? Number of responses (n26)
45
Costing the Final AnalysisTotal cost
20,803Cost per completed interview 102.48
46
Costing Hypothetical Survey of 1,000 AdultsCost
per completed interview 50 cell-only
102.48Cost per completed interview 950
landline 24.21
47
Conclusions

• Surveys of cell-phone only households feasible in
  Canada
• Significant differences between cell-only and
  landline respondents
• subject matter
• demographics
• attitudes
• behavioural

48
Conclusions (continued)

• Canadians will respond
• ethical/respondent burden
• keep the survey short (12 minutes or less)
• high cost relative to other modes
• keep in mind target population of survey

49
The complete report is also available

• For a copy of the complete report (English) or
  the executive summary (English or French), see
  the report A Survey of Cellular-Telephone-Only
  Households The New Technologies Web 2.0 and
  Government of Canada Communications Project at
• http//www.porr-rrop.gc.ca/index-e.html