Overview: Online Surveys

1
Overview Online Surveys
Handbook of Online Research Methods Colloquium
28-29 March 2007

• Vasja Vehovar
• University of Ljubljana, Slovenia
• http//WebSM.org

2
Introduction
Structure

• Computer-assisted survey information collection
• Key methodological issues
• Related issues
• Applications

3
Computer-assisted survey information collection
Technology and surveys

• 1930s applications of probability sampling
• 1960s expansion of telephone surveys
• 1970s computer technology appears in surveys
• 1980s computer-assisted surveying
• 1990s Internet mediated surveys

4
Computer-assisted survey information collection
Early CASIC interviewer assisted modes

• CATI Computer-assisted telephone interviewing
• CAPI Computer-assisted personal interviewing
• CASI Computer-assisted self-interviewing

5
Computer-assisted survey information collection
CASIC benefits

• Reduced time and costs for data input
• Elimination of errors during data transcription
• Implementation of advanced features
• automatic skips and branching
• randomization of questions and response options
• control of answers
• inclusion of multimedia elements

6
Computer-assisted survey information collection
Computerized self-administered questionnaires
(CSAQ)

• Different modes of CSAQ
• disk-by-mail
• touch-tone data entry (TDE)
• interactive voice response (IVR)
• e-mail surveys
• web surveys

7
Computer-assisted survey information collection
Online, Internet, Web, CASIC, CSAQ, CADAC
Internet surveys
Web surveys
8
Computer-assisted survey information collection
Online, Internet, Web, CASIC, CSAQ, CADAC
CSAQ
Internet surveys
Web surveys
9
Computer-assisted survey information collection
Online, Internet, Web, CASIC, CSAQ, CADAC
CSAQ
Internet surveys
Web surveys
CADAC
10
Computer-assisted survey information collection
Interviewer-less and paper-less surveys
11
Computer-assisted survey information collection
Interviewer-less and paper-less data collection
12
Computer-assisted survey information collection
CSAQ input-output technology
13
Computer-assisted survey information collection
CSAQ input-output technology
14
Computer-assisted survey information collection
Technological aspects

• Developments of the Internet and related
  technologies
• Importance of broadband Internet access
• Possibilities of distribution across various
  platforms and devices
• Fast and easy implementation using specialized
  software tools

15
Key methodological issues
1. Probability vs. non-probability sampling

• Two major problems of probability samples in
  Internet surveys
• non-coverage
• sampling frame problems
• Image of Internet surveys as inherently
  non-probability ones

16
Key methodological issues
The spurious link
17
Key methodological issues
2. Sampling and invitation
18
Key methodological issues
3. Non-response

• Generally low and variable response rates to web
  surveys
• Complexity of the response process
• Measures to increase response rates
• incentives (e.g. using PayPal)
• multiple follow-up contacts
• other motivations

19
Key methodological issues
Involvement process
20
Key methodological issues
Participation in web surveys
21
Key methodological issues
4. Questionnaire design

• Question types and visual elements
• Advanced features of computerized questionnaires
• Inclusion of multimedia
• Problems and considerations
• standardization
• impact on respondents answers
• technical problems

measurement error
22
Key methodological issues
5. Post-survey adjustments

• Commonly utilized for correcting
• deviations from probability selections
• non-coverage and non-response problems
• Importance for non-probability Internet surveys
• Promising approaches
• calibration methods
• propensity score weighting
• modelling and causal analysis

23
Related issues
1. Costs, errors and management

• The managerial process of survey implementation
• costs optimization
• data quality
• relations between data quality and costs
• Cost effectiveness of web surveys
• costs for increasing response rates
• evaluations of costs in relation to errors

24
Related issues
2. Survey mode

• Web surveys vs. other modes
• lower response rates (Lozar Manfreda et. al.,
  2007)
• comparable or higher level of validity and
  reliability (e.g. Fricker et al., 2005 Chang
  Krosnick, 2002)

25
Related issues
Mixing survey modes
26
Related issues
Selecting the right mixture
27
Related issues
Programing of the questionaire
28
Related issues
3. Mixing research methods

• Quantitative and qualitative research methods
• Flexible combinations of approaches (Morgan,
  1998)
• qualitative preliminary
• quantitative preliminary
• qualitative follow-up
• quantitative follow-up
• New possibilities with the Internet

29
Related issues
Flexible on-line mixed methods (FOMM)
30
Related issues
4. Ethics, guides and standards

• New ethical dilemmas in survey research
• solicitation process, spam and privacy
• data protection
• children and minors
• combining data
• Development of new standards (e.g. ESOMAR, AoIR,
  MRA)

31
Current and future applications
Applications

• Different applications according to
• type (internet access panels, on-line voting,
  intercept)
• organizations (official, academic, commercial)
• topics (marketing research, polling opinions,
  health)
• complexity (simple one-shot surveys vs. complex
  integrated data collection)

32
Current and future applications
Related areas

• psychological research
• online experiments
• paradata and online measurement
• specific research fields (usability research,
  HCI...)
• emerging areas (e-learning, e-government)

33
Current and future applications
Trends

• Expansion of CASIC modes, options and mixtures
• Integration of devices (TV, mobile,..)
• Speech recognition and touch-screens
• Multimedia and interactivity
• Integration with other CADAC (GPS, paradata,
  observations,...)
• Specialisation and professionalisation