Constructing Surveys and Questionnaires

1
Constructing Surveys and Questionnaires

• A Tutorial of Important Considerations
• PSY 217
• February 7, 2006

2
Important Considerations

• Constructing questionnaires and surveys is one of
  the most difficult tasks that a researcher can
  perform. To have a valid and reliable
  instrument, rigorous testing of the test is
  necessary. Following are some important factors
  to consider in writing and evaluating a
  questionnaire. You will have a chance to
  evaluate questions for a questionnaire in lab
  today.

3
The Order of the Questions

• The ordering of the questions in a survey or
  questionnaire has a direct and dramatic impact on
  the responsiveness of those answering the
  questions and the reliability of their responses.
  Ordering differs depending on the survey method
  mail, personal interview, telephone survey.
• Begin self-administered questionnaires (e.g.,
  mail surveys) with the most interesting questions
  to get attention. Demographic questions appear
  last.
• Begin interviews and phone surveys with
  demographic questions first to put the respondent
  at ease. More serious, sensitive, and important
  questions related to the research hypothesis
  should appear later.

4
The Vocabulary and Wording of the Questions

• Use straightforward, simple words most likely to
  be familiar to the respondents.
• Avoid unusual words or ambiguous words and
  phrases. Make wording clear and specific.
• Edit and test the questions on yourself and
  others before using them to collect data.

5
Special Types of Questionsto Use

• Funnel Questions
• Begin with a general question, funnel to a more
  specific and related question. Use when the need
  is to obtain responses specific to the order in
  which a series of questions is presented.
• Example of funnel
• How old were you when you began college?
• Example of specific question that follows
• What was your favorite college course?

• Filter Questions
• Their use is to eliminate or filter out
  respondents whose experiences do not apply to the
  question that follows.
• Example of filter
• Do you feel that pilots should be allowed to
  carry weapons in the cockpit? If answer is no,
  the respondent is told to skip the next Q.
• Example of the question that follows
• Do you feel that pilots should carry pistols or
  semi-automatic weapons?

6
Types of Questions to Avoid! 1

• Double-barreled questions that ask two things at
  once. These are difficult to answer because they
  may tap two potentially very different attitudes
• Example Should senior citizens be given more
  money for recreation centers and food assistance
  programs?

7
Types of Questions to Avoid! 2

• Double-negatives questions that contain two or
  more words that denote negatives, e.g., no,
  never, not. These are confusing and may
  produce responses that cant be interpreted.
• Example Would you not negate the information
  given in the reading if you could?
• Example Would you never not go to speak with the
  professor?

8
Types of Questions to Avoid! 3

• Leading questions that provide information that
  might bias the respondent. The leading part is
  in red!
• Example Most people favor the use of nuclear
  energy. What do you think?
• Example Do you favor eliminating the wasteful
  excesses in the public school budget?

9
Types of Questions to Avoid! 4

• Loaded questions contain emotionally loaded
  words, including those that are sexually,
  ethnically, racially, or gender-biased.
• Example How do you feel toward racist students?
• Example Would you prefer to have discussions of
  current events with men or with gals?

10
Types of Questions to Avoid! 5

• Long and unwieldy use questions that are as
  short as possible (20 words or less, according to
  our text!)
• A Good Example of a Really Bad Question Being a
  student at the University of Dayton in this day
  and age and having the benefits of a higher
  education, how do you feel about the present
  republican administrations stance on subsidized
  education for people of special circumstances,
  including the poor, immigrants, and veterans?
• Can you see other factors that are wrong with
  this question?

11
Question/Item Formats 1

• The following slides describe some of the formats
  for questions that can be used on surveys. Some
  are more or less popular than others for
  example, checklists may be more prone to error by
  the respondent and have fallen out of favor.
• The dichotomous format two alternatives for each
  question/item, e.g., yes or no, and true or false
• Example The thought of death seldom enters my
  mind. Circle one Yes No

12
Question/Item Formats 2

• Forced alternative format similar to the
  dichotomous format, this type of item tells the
  respondent to make a choice between two
  alternative statements
• Example Circle the letter of the statement that
  best describes your feeling
• There are institutions in our society that have
  considerable control over me.
• Little in this world controls me, I usually do
  what I decide to do.

13
Question/Item Formats 3

• The polytomous format each item/question has
  more than two alternatives and sometimes includes
  distracters (incorrect choices), e.g., the
  typical multiple-choice question
• Example Compared to the average student (select
  one of the following)
• A. I give much more effort because Im motivated
  more.
• B. I give an average amount of effort.
• C. I give less effort because of my heavy
  schedule.

14
Question/Item Formats 4

• The Likert format items that allow the
  respondent to indicate the degree of agreement
  with a particular attitudinal question
• Example I enjoy social gatherings just to be
  with people.
• 1Not at all
• 2Somewhat
• 3Average
• 4Pretty much
• 5Very much

15
Question/Item Formats 5

• Checklists commonly used in personality
  measurement is the adjective checklist. A long
  list of adjectives or other items is given and
  the respondent indicates whether each one is
  characteristic of him- or herself.
• Example Checkmark or circle those of the
  following that apply to you
• Adventurous
• Alert
• Curious
• Quiet
• Imaginative
• Fair-minded

16
Question/Item Formats 6

• Open-ended format these questions/items allow
  the respondent the opportunity to respond freely
  and to whatever length desired
• Example How would you summarize your chief
  problems in your own words?

17
Whats next?

• We have a lab worksheet with examples of
  questions on a questionnaire. I want you to
  identify the problems with the construction of
  these questions.
• Thank you!