Questionnaires

1
Questionnaires
2
Judge a man by his questionsrather than his
answers.VoltaireHistorian and Philosopher
3
Questionnaire

• Definition
• A document containing questions and other types
  of response items designed to solicit
  information.
• Two Major Types of Questions
• Close-ended Respondent selects and answer from
  those provided.
• Open-ended Respondent is asked to provide his
  or her own answers (fill in the blank).
• Compared to Interviews
• Advantages Quick, cheap, standardized,
  anonymous.
• Disadvantages Inflexible, lower participation,
  trust.

4
Where are Questionnaires Used?
5
Reliability and Validity

• A questionnaire is a form of measurement, and, as
  such, it must meet the same standards of
  reliability and validity as other measures such
  as standardized tests.
• Reliability
• Consistency of measurement.
• Validity
• How meaningful, useful, and appropriate our
  conclusions are.

6
Factors that Influence Reliability

• 1. What you measure
• physical traits
• cognitive traits
• affective traits
• 2. The instrument you use
• 3. Your technique

7
1. What You Measure

• a. Physical Traits
• most reliable of these three
• e.g., height, weight, blood pressure
• b. Cognitive Traits
• less reliable than physical traits
• e.g., SAT questions
• c. Affective Traits
• least reliable of these three
• e.g., feelings, attitudes, opinions

8
2. The Instrument You Use
Skinfold
Bioelectrical Impedance
Dual Energy X-ray Absorptiometry
Height-Weight Table
9
3. Your Technique
10
Question Types

• Rating Scale
• Rank Order
• Semantic Differential
• Multiple Choice
• Open-Ended
• Likert Scale

11
Caution Use Balanced Scales

• Balanced
• Very Poor Poor Average Good Excellent
• Unbalanced
• Poor Average Good Very Good Excellent

12
Characteristics of Good Questions

1. One that all the people understand in a
   consistent way...and in a way consistent with
   what the researcher intended it to mean.
2. One that can be administered in a standard way.
3. One that communicates to all respondents the kind
   of answers that are desired/acceptable.
4. One that the respondents are capable of answering
   (unless measuring knowledge is the goal).
5. One that the respondents are willing to answer.

13
Four Rules for Writing Questions

• Be Brief
• Use the shortest way to ask a question without
  losing its intentits not just about using the
  fewest words.
• Be Objective
• Assuming your goal is to find the truth, pay
  attention to the neutrality of the words.
• Be Simple
• Use language that is simple in both words and
  phrases.
• Be Specific
• Ask precise questions. Avoid things that are
  too general, too complex, or undefined.

14
Effects of Question Wording Ex. 1
1. About how fast were the cars going when they
hit each other? smashed into each
other? 2. Did you see any broken glass?
15
Effects of Question Wording Ex. 1
...when they contacted each other? ...when they
hit each other? ...when they bumped into each
other? ...when they collided with each other?
...when they smashed into each other? One week
later Was there broken glass? Hit 14 said
yes Smashed 32 said yes
31.8 mph 34.0 mph 38.1 mph 39.3 mph 40.8 mph
16
Effects of Question Wording Ex. 2
Do you think the United States should allow
public speeches against democracy? Do you think
the United States should forbid public speeches
against democracy?
Schuman, H., Presser, S. (1977). Question
wording as an independent variable in
survey analysis. Sociological Methods Research,
6, 151-170.
17
Effects of Question Wording Ex. 2
Do you think the United States should allow
public speeches against democracy? Not Allow
44 Allow 56 Do you think the United States
should forbid public speeches against
democracy? Forbid 28 Not Forbid 72
Schuman, H., Presser, S. (1977). Question
wording as an independent variable in
survey analysis. Sociological Methods Research,
6, 151-170.
18
Allow-Forbid Question Formats
People respond differently to losses (e.g.,
prohibitions) than to gains (e.g.,
allowances). Allow-Forbid distinctions have
also been observed in surveys about X-rated
films, using salt to melt snow, and the legality
of abortion. What current issues could be asked
using this format? Possible final research
project?
19
Push-Polling Description

• A telemarketing technique in which telephone
• calls are used to canvas vast numbers of
• potential voters, feeding them false and
• damaging information about a candidate under
• the guise of taking a poll to see how this
• information affects voter preferences. In fact,
• the intent is to push the voters away from one
• candidate and toward the opposing candidate.
• The National Council on Public
  Polls

20
Push-Polling Example 1

• John McCain calls the campaign finance system
• corrupt, but as chairman of the Senate
• Commerce Committee, he raises money and
• travels on the private jets of corporations with
• legislative proposals before his committee. In
• view of this, are you much more likely to vote
  for
• him, somewhat more likely to vote for him,
• somewhat more likely to vote against him, or
• much more likely to vote against him?

21
Push-Polling Example 2

• The non-partisan National Journal has called
  Obama the country's most liberal candidate.
• - He's for gay adoptions and driver's licenses
  for illegal immigrants.
• - He voted against allowing people to have guns
  to protect themselves in their own homes.
• - He is weak on terrorism and has said he would
  personally negotiate with terrorist nations like
  Syria and Iran.
• Would this cause me to have very serious
  serious minor or no doubts about Barack Obama?

22
Steps In Using Questionnaires

• Define research objectives
• Define target population and select a sample
• Design the questionnaire
• Pretest the questionnaire (pilot study)!
• Pre-contact the sample
• Distribute the questionnaire
• Follow-up with non-respondents
• Analyze the questionnaire data

23
Maximizing Response Rates

• Request participation in advance
• Include background information in intro or letter
• State the purpose of the survey
• Give a reason for participation
• Give the terms of anonymity and confidentiality
• Allow enough time to complete survey (end date)
• Send reminders to those who have not completed it
• Design a survey that is easy to follow
• And one that the respondents are willing to
  answer
• Offer a gift or incentive for participation

24
Research on Response Rates
100
90
80
70
60
50
Percentage Returned
40
30
20
10
0
Initial Mailing
First Follow-up
Second Follow-up
Third Follow-up