Web Survey Design: Recruitment contd

1
Web Survey Design Recruitment (contd)

• Invitations to participate can be via phone, fax,
  mail, in person, as well as via Web
• Web options popup windows, banner ads, sidebars,
  body textbut never start and stop with your own
  site!
• When using third party target lists with e-mail
  addresses, cannot assume opt-in unless explicitly
  confirmed must contact initially through other
  means
• Possible outcomes of illegitimate research
  include restricted access to panelists,
  disenrollment, low response rates, formal
  complaints, etc.

2
Web Survey Design Recruitment (contd)

• Screening/terminate considerations use secondary
  or self-reported data, e.g. on prescribing
  behavior?
• If use online screening criteria, obviously
  letting respondents go back and change initial
  answers is counterproductive
• Can allow disqualified respondents, particularly
  in high incidence population, to answer some
  questions and pay full or reduced (with proper
  explanation) honorarium
• Manage expectations raised by invitation dont
  promise inclusion for all if not possible

3
Web Survey Design Recruitment (contd)

• Over-recruit as always, since data validation may
  exclude some cases
• Assign unique user IDs and passwords to ensure
  only one complete per respondent
• Anonymous surveys risk multiple completes per
  respondent
• Cultural differences must guide choice of
  incentives
• Cozint research to date shows US physicians
  consensus is show me the money

4
Web Survey Design Administrative Considerations

• Design must be Web-specific, not just
  self-administration specific
• If online reporting used, plan who has access to
  whatconsider involving sales reps and management
  as appropriate
• Manage brand team expectations about turnaround
  field in haste, repent at leisure
• Support expectations about security and privacy,
  whether on HCP or consumer side
• CASRO and other guidelines apply
• Respondent-level data must be used for modeling,
  never marketing

5
Web Survey Design Question Types

• All that apply/ever
• Best/worst/most/least (applicable/likely, etc.)
• Numeric/percentage
• Rankings, ratings or point allocation tasks
• Forced totals and choices (where appropriate
  dont precode away your ability to understand the
  market)
• Three most potent
• Single most
• Paired comparisons
• Clickable maps, diagrams, pictures

6
Web Survey Design Capabilities to Use

• Iterative logic remove alternatives as used,
  e.g. for most important and its successors
• Branching logic skips, inserted values, new
  language, item rotation
• Perceptions/comparisons of audiovisual stimuli as
  bandwidth and design permit
• Maps, line art, Web forms, banner ads, photos,
  video, logos, storyboards, posters, programs
• Thumbnails in scale side by side

7
Web Survey Design Content

• Issues benefiting from contemplation and/or
  distance algorithms, recent details, clinical
  experiences, subtle differences, rare side
  effects, new agent impact
• What percentage of decisions
• Primary vs. secondary prescribing factors
• When/why began using Xwhat did it replacewhat
  other schools of thought regarding X exist
• What have you heard about Y? How interesting or
  important is that? What are you waiting for
  before using it?

8
Web Survey Design Context Matters

• Incorporate ability to repeat read, start and
  stop, seek help, wordsmith answers, reformulate
  responses offline
• Make it clear whether you want first thought,
  most pressing thought, or mind dumps in open
  ends
• Understand that prescribing period described will
  be closer to final report or data receipt data
  than with longer field period
• Decide if your analysis should incorporate date
  of survey, minutes online, browser or connection
  speed
• Options for assistance should appear on every
  page
• If you dont know the respondents context, dont
  field a quantitative Web survey

9
Web Survey Design Structure

• Introduction must convey benefit introduced by
  invitation closing must reinforce it
• Wording must be tight and interesting, concealing
  transitions while conveying flow
• Redirects to other sites must be seamless
• As always, consider parallel structure, scale
  consistency and groupings, isolated rating
  questions, bipolar vs. intensity scales
• Instructions refer to page elements as seen by
  all browsers unlike on faxed study, answer lists
  lacking other specify must be complete

10
Web Survey Design What to Avoid

• Phone-speak thank you, next well talk
  about, doctor, now please
• Large, daunting grids which prompt
  trigger-clicking
• Research and ad-speak concept description,
  white space, body copy, product
  description, story board
• Insults to the respondents intelligence record
  below, type here, and, worst of all, click
  here similarly, do not specify choose one
  only if a likelihood or yes/no question
• Other specifies which will not be analyzed

11
Web Survey Design What to Avoid (contd)

• Double-barreled or nonsensical questions
  irrational forced choices do you prefer Web
  sites with lots of white space or text? (it
  depends)
• Questions which require the respondent to perform
  arithmetic calculation (thats the computers
  job)
• Leaving punctuation or specification of units,
  such as dollar signs, decimal points, percentage
  signs, to respondent
• Equally unwise specifying the wrong units give
  two choices if needed (e.g. lb or kg)
• But specify the units in the answer blank or
  list dont assume the respondent knows them

12
Web Survey Design Back to the Basics

• Start blinded product label at L or similarly
  innocuous point
• Use the power of the skip pattern!
• Its percentage, not percent of
• Scales are from 0 to 10, not an 11-point
  scale
• Keep the answers away from the question stem
• Ask about percentages, not numbers, except early
  on
• Reassure as to validity of best estimate,
  approximately, never best guess
• Use most, some or other(s) rather than
  you for sensitive issues

13
Web Survey Design Wording Matters

• Click vs. check (what happens, actually?)
• Donts refused, classification purposes,
  unnecessary or double negatives check only one
  choice rather than check one or select the
  single most important reason (let validation be
  the bad guy)
• Dont let the medium intrude into the completion
  process avoid read in favor of review, go
  to next screen (should be continue)
• Consider three-point vs. five-point scales
  never (not at all) vs. somewhat/ sometimes vs.
  mostly/often/ always is much easier to grasp on
  screen than half empty/half full scales, and
  often achieves the same goal

14
Web Survey Design Wording Matters (contd)

• Dont write an open end that challenges
  professional judgment Why do you not prescribe
  agent Y? that looks more negative on screen.
  Instead, consider
• What are the reasons for not prescribing agent X?
• You are not currently prescribing agent X. Why is
  that?
• Under what circumstances might you begin
  prescribing agent X?
• In what timeframe would you anticipate
  prescribing agent X? (where a choice is not in
  foreseeable future)

15
Web Survey Design Wording Matters (contd)

• Imposing your context on the respondent is much
  more likely to cause breakoffs than with phone
  e.g. do physicians really classify this disorder
  as mild, moderate or severe ?
• Using the Web, do you easily become
• Impatient?
• Frustrated?
• Annoyed?
• Bored?
• Confused?
• So do all the other humans.

16
Web Survey Design Design Matters

• Cannot refer to colors or type sizes, nor expect
  every browser to show your colors, fonts, sizes,
  or bullets
• Can refer to left or right, but not first or
  last, horizontally
• Vertically, let clicks be close but not too
  close facilitate reading left to right (except
  in Japan!)
• Avoid bright backgrounds use cool colors
• Minimize horizontal and vertical scrolling
• DONT capitalize (wasnt that distracting?)
• Instructions for the entire survey, including
  definitions, need their own page
  question-specific instructions go with the
  question

17
Web Survey Design Design Matters (contd)

• When using graphics, ask browser and connection
  speed, to understand variable results
• Dont break grids,question stems, or answer lists
  across screens
• Keep follow-on open end, why do you say that?
  on same screen with closed end if at all possible
• If not, reference open-end appropriately You
  prescribe Agent X only for patients with Blue
  Hair Disease. Why is that?

18
Web Survey Design Common Errors

• Over-outlining, setting the stage, introducing
• Time remaining thermometers misleading,
  distracting, scream the context, invite
  breakoffs, guesses of remaining time, fitting
  answers into it
• Be accurate about length in the invitation and
  intro!
• Redundant question stems and answer lists or
  grids (the more they have to read, the less time
  for completion)
• Placing demographic questions that are not
  terminate points or questionnaire version
  criteria at the beginning, thus risking qualified
  breakoffs
• Failure to use automation, e.g. forcing
  respondent to restate choices you may have seen
  these before (Why?)

19
Web Survey Design Common Errors (contd)

• Using decades for age (use 25-34 ranges, or,
  better yet, just ask the age!)
• Use similar ranges for income (not by tens)
• Do you really need ethnicity, education, income,
  gender and/or age?
• Numbering answer list items for no reason, e.g.
  regimens
• Failing to note forced totals or ranges, then
  irritating respondent with error message
• Unnecessarily characterizing open ends (in a few
  sentences
• Ordering answer lists by your views instead of
  logically or alphabetically

20
Web Survey Design Common Errors (contd)

• Preceding a specific question with a general
  intro, generating answers to the wrong question
• Listing scale points or choices from top down,
  e.g. starting with 10 or completely
  successful
• Placing performance ratings on same page/grid as
  attribute importance ratings
• Using abstract-speak Was the level of samples
  satisfactory? Was the message motivating? Was
  the ad believable?
• Giving the show away Are you aware that there
  are different methods for assessing X? (yestrue
  yes or Im not an idiot, noIm clueless or Im
  oppositional)

21
Web Survey Design Some To-Dos

• Set range, value and inter-question validation
  criteria, both on and off line distinguish
  between the two
• Recognize the possibility of A/V distractions
  which compete with your stimuli
• Beta test with colleagues pre-test with
  appropriate respondents unknown to you
• You are not an appropriate length tester!
• Use log files to determine problem areas not
  caught in pretest (there shouldnt be any)