Questionnaire Design

1
Questionnaire Design

• Ed Blair
• University of Houston

2
Broad issues in writing questions

• Have you asked the right question?
• Do respondents understand the question?
• Do they know the answer?
• Are they willing to answer accurately?

3
Have you asked the right question?

• In marketing research, we ask questions because
  the answers will help improve marketing
  decisions. A question can be technically
  perfect, yet still be a bad question because it
  doesn't properly serve our marketing purpose.
• Is this a good question?
• 1. Have you heard of any physician referral
  services that you can call if you need help in
  finding a doctor?
• Yes . . . . . . 1 (ASK Q1A)
• No . . . . . . 2 (GO TO Q2)
• (IF YES)
• 1A. Can you name any of these services or tell
  me their telephone numbers? (PROBE Any
  others?)

4
Making sure its the right question

• Ask yourself
• Is this question consistent with my theory about
  how the market works?
• Do I know, specifically, how I plan to use the
  answers to this question to draw conclusions
  about the market?

5
Do respondents understand the question?

• Do respondents understand the question?
• Q Does your home have power conditioning
  equipment to protect your critical equipment
  against power fluctuations?
• Q (en Español?)
• Do all respondents interpret the question the
  same way?
• Q What is your income?
• Do respondents interpret the question the same
  way you do?
• Q What are all the reasons you bought your
  groceries at ValueLand rather than some other
  store? (PROBE Any others?)

6
Minimizing problems from misunderstanding

• Ask questions one at a time.
• BAD Do you consider a backup generator or a
  surge suppressor to be necessary equipment for
  your home?
• Use simple, specific language, and avoid words
  with more than one meaning.
• BAD? Do you exercise regularly?
• Specify who, what, when, where, how, and the
  desired form of response.
• Which of the following categories best describes
  your familys total income last year before
  taxes?
• Before doing the project, have someone read you
  the questions. Also consider pre-testing your
  questionnaire to see whether respondents have
  trouble understanding some of the questions, and
  how they interpret key questions.

7
Do respondents know the answer?

• Were they there?
• Q How satisfied were you with the service you
  received from the hospital's business office?
• Q Why did your company?
• Can they remember?
• Q In the past thirty days, about how many
  telephone calls did you make?
• Do they have an opinion?
• Q I have a list of hospitals in our area. As I
  read each one, please tell me how you would rate
  that hospital as a place to receive care
  excellent, good, fair, or poor. How about . . .
• Are intentions meaningful?
• Q If there was a local magazine like Consumer
  Reports that evaluated restaurants, auto repair
  shops, stores, plumbers, and other local
  businesses, would you subscribe?
• Q How many small batteries do you intend to buy
  during the next six months?

8
Minimizing knowledge problems (a)

• Consider qualifying respondents for knowledge or
  allowing for no opinion or not sure or dont
  know.
• Use the right time frame for your question.
• Event Reasonable time frame
• Hospital stay, car purchase One year
• Appliance purchase Three months
• Grocery purchase One week
• Food consumption One day
• Measure behavior over intentions/attitudes (e.g.,
  previous battery purchases over intent to
  purchase, actual referrals over intent to
  refer/satisfaction).
• Conjoint, anyone?

9
Minimizing knowledge problems (b)

• Anchor subjective measures with comparisons
  across time, objects, or people.
• 78 of patients say they were "very satisfied"
  overall with the food at your hospital. Is this
  good or bad?
• - What if the number was 84 last year?
• - What if the number is 84 at another
  hospital?
• - What if the number is 90 for maternity
  patients, 71 for oncology patients?
• Use previous questions wherever possible!
• Interpret and present self-report data as if they
  are approximate, not exact, and focus on patterns
  in the data rather than specific tabulations.

10
Are respondents willing to answer accurately?

• Is the behavior or attitude socially desirable or
  undesirable?
• Q During the past month, have you read any books
  other than for work or school?
• Q During the past month, since (DATE) , did you
  drink beer even once?
• (IF YES)
• A. About how often did you drink beer during the
  past month?
• Once . . . . . . . . . . . . . . .1
• 2-3 times . . . . . . . . . . . ....2
• About once a week . . . . . . .3
• 2-3 times a week . . . . . . . 4
• 4-6 times a week . . . . . . . . ..5
• Once a day or more . . . . . . . 6
• Do respondents think you care which answer they
  give?
• Q If there was a local magazine like Consumer
  Reports that evaluated restaurants, auto repair
  shops, stores, plumbers, and other local
  businesses, would your family subscribe to this
  magazine?

11
Minimizing problems with unwillingness (a)

• Professionalism is a must, to convince
  respondents that you care whether they answer but
  not how they answer. This means
• - Professional appearance for questionnaires
  that respondents will see.
• - A proper and professional introduction.
• - Formal language in the questionnaire (no
  slang, please)
• - Business clothing for interviewers if
  respondents will see them
• - Proper training so that interviewers seem to
  know what they are doing
• Use choice measures or previous behavior rather
  than intention measures.
• Check for loading. Don't ask subjective questions
  (attitudes, opinions, etc.) in in a "Yes-No"
  format, and avoid angel" or "devil" words (e.g.,
  reasonable, forbid), absolute terms (always,
  never), and appeals to the norm (most people
  think")

12
Minimizing problems with unwillingness (b)

• Don't reveal sponsor identity unless necessary to
  convince respondents that their answers will have
  a legitimate use.
• Consider less personal methods (mail, online) for
  sensitive topics.
• Use open questions for sensitive quantitative
  questions (except income)
• Use longer questions to reduce the social stigma
  attached to certain answers.
• Q Many people say that they just don't have time
  for reading anymore. In the past month, have you
  read any books other than for work or school?

13
Whats wrong with this questionnaire? (a)

• (Fan survey to be administered at an Astros game)
• 1. In what city do you live? ____________________
  
• (IF HOUSTON)
• A. In what part of Houston do you live?
• ? Inside the Loop ? South
• ? North ? Southwest
• ? Northeast ? West
• ? East ? Northwest
• ? Southeast
• 2. How many people are in your group at tonight's
  game?
• ? 1 ? 2-3 ? 4-5 ? 6 or more

14
Whats wrong with this questionnaire? (b)

• 3. How many of them are kids?
• ? 1 ? 2-3 ? 4-5 ? 6 or more
• 4. How many people in your group, including
  yourself, are Astros season ticket holders?
• ? 1 ? 2-3 ? 4-5 ? 6 or more
• 5. How many games have you, yourself attended
  this season?
• ? 1 ? 2-3 ? 4-5 ? 6 or more
• 6. What four radio stations do you listen to most
  often?
• __________________ _________________
• __________________ _________________

15
Whats wrong with this questionnaire? (c)

• 7. Why did you come to tonight's Astros game?
• __________________________________________
• __________________________________________
• __________________________________________
• We need your name and address for our prize
  drawing.
• NAME ________________________________________
• ADDRESS ________________________________________
  
• CITY, STATE, ZIP _______________________________
  ____

16
Revised questionnaire (a)

• 1. Are you attending tonight's game on a season
  ticket or a single game ticket?
• ? Season ticket ? Single game ticket
• 2. Counting tonight's game, how many Astros games
  have you attended this season?
• ? 1 ? 2-3 ? 4-5 ? 6 or more
• 3. Counting yourself, how many people are in your
  group at tonight's game?
• ? 1 ? 2-3 ? 4-5 ? 6 or more
• 4. How many of the people in your group are less
  than 13 years of age?
• ? 0 ? 1 ? 2-3
  ? 4-5 ? 6 or more

17
Revised questionnaire (b)

• 5. When did you decide to attend tonight's game?
  Was it today, before today but during the past
  week, or earlier than the past week?
• ? Today ? Past week ? Earlier
• 6. What promotions, if any, might influence you
  to attend an Astros game?
• (CHECK ALL THAT APPLY)
• ? 2 off tickets ? Prize drawings
• ? Family discounts ? Country music concerts
• ? Free parking ? Rock music concert
• ? Free Astros mug ? Tailgate parties
• ? Free kid's cap ? Home run derby

18
Revised questionnaire (c)

• 7. What is your favorite radio station?
• (FILL IN) _____________________
• We need your name and address for our prize
  drawing.
• NAME ________________________________________
• ADDRESS ________________________________________
  
• CITY, STATE, ZIP _______________________________
  _____