QUESTIONNAIRE DESIGN: TYPE OF QUESTIONS

1
QUESTIONNAIRE DESIGN TYPE OF QUESTIONS

• Lu Ann Aday, Ph.D.
• The University of Texas School of Public Health

2
MAJOR RESPONDENT TASKS DIFFERENT TYPES OF Qs
Major Tasks/ Types of Qs Comprehen- sion Retrieval Judgment Response
Health X
Demographics X X
Behavior-Nonthreatening X X
Behavior-Threatening X X
Knowledge X X
Attitudes X X
3
DESIGN ELEMENTS CRITERIA Health

• ELEMENTS
• Words

• CRITERIA
• Clearly define the concept
• Physical, mental, or social dimensions
• Positive vs. negative health
• Provider vs. patient judgments

4
DESIGN ELEMENTS CRITERIA Health

• ELEMENTS
• Responses

• CRITERIA
• Match scale type to design analyses
• Design precision sensitivity
• Analyses typology, Likert, utility, etc. scale
  types

5
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Household
• composition

• EXAMPLES
• Ask about relationships to reference person (who
  owns or rents the home) rather than head of
  household

6
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Age

• EXAMPLES
• What is your date of birth?
• In what year were you born?
• Which of the following categories best describes
  your age?

7
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Marital
• status

• EXAMPLES
• Are you currently married, not married but living
  with a sexual partner, separated, divorced,
  widowed, or never married?

8
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Ethnicity
• Race

• EXAMPLES
• Are any of the following groups your national
  origin or ancestry?
• Which group or groups BEST represents your race?

9
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Education

• EXAMPLES
• What is the highest grade or year of regular
  school that you completed?

10
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Employment status

• EXAMPLES
• Are you currently employed? IF YES, ASK, Are you
  self-employed Is that full-time or part-time?
  IF NO, ASK, Are you retired, disabled, a student,
  keeping house, temporarily unemployed, or not
  looking for paid employment?

11
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Occupation

• EXAMPLES
• What is your job title?
• What are your most important job activities or
  duties?
• What kind of business or industry is this?

12
DESIGN ELEMENTS CRITERIA Demographics

• VARIABLE
• Family Income

• EXAMPLES
• Which of the following categories best represents
  your total combined FAMILY income during the last
  12 months? Include money from jobs, social
  security, retirement income, unemployment
  payments, public assistance and so forth. Also
  include income from interest, dividends, net
  income from business, farm, or rent, and any
  other money income received.

13
DESIGN ELEMENTS CRITERIA BehaviorNonthreateni
ng

• ELEMENTS
• Administration
• Overreporting (telescoping salient events)
• Underreporting
• (omitting non-salient events)

• CRITERIA
• Prime R to remember
• Use bounded recall ask about events in previous
  then current time period
• Adjust recall period
• Ask in terms of usual behavior
• Use memory aids, e.g., aided recall, records,
  diaries

14
DESIGN ELEMENTS CRITERIA BehaviorThreatening

• ELEMENTS
• Words
• Phrases
• Sentences

• CRITERIA
• Use words that are familiar
• Judiciously consider loading questions, e.g.,
  others engage in behavior
• Make questions longer

15
DESIGN ELEMENTS CRITERIA BehaviorThreatening

• ELEMENTS
• Responses

• CRITERIA
• Consider open-ended rather and closed-end
  responses

16
DESIGN ELEMENTS CRITERIA BehaviorThreatening

• ELEMENTS
• Questionnaire

• CRITERIA
• Build in appropriate explanations transitions
• Ask whether they ever engaged in behavior
  before asking about current practices
• Ask questions at end or embed among less
  threatening items

17
DESIGN ELEMENTS CRITERIA BehaviorThreatening

• ELEMENTS
• Administration

• CRITERIA
• Consider a self-administered or more anonymous,
  e.g., computerized, format, to minimize threat

18
DESIGN ELEMENTS CRITERIAKnowledge

• ELEMENTS
• Questions

• CRITERIA
• Consider screening Q first to see if R knows
  anything about the topic
• Ask more than one question to find out about
  knowledge
• Phrase Qs more like Qs about opinions, e.g., In
  your opinion

19
DESIGN ELEMENTS CRITERIAKnowledge

• ELEMENTS
• Responses

• CRITERIA
• Use open-ended rather than closed-end response
  formats when necessary, to prevent guessing
• Provide dont know as an alternative

20
DESIGN ELEMENTS CRITERIAKnowledge

• ELEMENTS
• Administration

• CRITERIA
• Use self-administered quex, esp. in groups
• Consider inclusion of sleeper (fictional)
  options

21
DESIGN ELEMENTS CRITERIAAttitudes

• ELEMENTS
• Words
• Phrases

• CRITERIA
• Clarify the attitude object (or focus of attitude
  statement)
• Provide balanced alternatives, e.g., agree or
  disagree support or oppose
• Do not included double-barreled (more than one)
  referent

22
DESIGN ELEMENTS CRITERIAAttitudes

• ELEMENTS
• Sentences

• CRITERIA
• Use medium-length, followed by medium-length to
  long Q
• Limit the number of Qs included in a battery of
  similar Qs

23
DESIGN ELEMENTS CRITERIAAttitudes

• ELEMENTS
• Responses

• CRITERIA
• Minimize yea-saying
• Include positive negative items
• Put least socially desirable response first
• Choose best approach to measuring attitude
  strength
• Rating vs. ranking
• of points on scale
• Use of uncertain

24
DESIGN ELEMENTS CRITERIAAttitudes

• ELEMENTS
• Administration

• CRITERIA
• Ask more general attitudinal items before
  specific ones

25
SURVEY ERRORS Formulating Specific Types of
Questions
Systematic Errors Systematic Errors Systematic Errors Systematic Errors Variable Errors
Questionnaire effects under/ overreporting Respondent effects social desirability bias Respondent effects yea saying (acquiescent response set) Mode effects social desirability bias Questionnaire effects order context
Solutions to errors Employ cognitive question questionnaire design and evaluation procedures, and use bounded recall and memory aids, to improve the accuracy in answering nonthreatening questions about behavior. Employ cognitive question questionnaire design and evaluation procedures, to enhance the honesty in answering threatening questions about behavior. Reduce the length of a series of questions (or use a short-form of scales) measuring knowledge or attitudes. Consider using more anonymous modes of data collection. Conduct split-ballot experiments to evaluate how answers to questions vary when they appear in a different order in the questionnaire.
26
REFERENCES

• Dillman, Don A. (2000). Mail and Internet
  Surveys The Tailored Design Method. Second
  Edition. New York John Wiley Sons, Inc.
• Ware, J.E., Jr., Gandek, B., for the IQOLA
  Project (1998). Overview of the SF-36 Health
  Survey and the International Quality of Life
  Assessment (IQOLA) Project. J. Clinical
  Epidemiology, 51 (11), 903-912.