Survey Research and Asking Questions

1
Survey Research and Asking Questions

• MIS 611B
• February 19, 2002

2
Definition

• A survey is an inquiry which collects
  information, by whatever method (usually a
  sample) of different cases and arranges the
  information about these cases as variables.
  Usually the independent and dependent variables
  are records, observations, or responses.

3
What is survey research

• Survey research has three distinct
  characteristics
• Collection of information by asking people for
  information in some structured format
• It is usually a quantitative method
• Information is gathered via a sample in order to
  generalize to the population

4
Question Content

• In questionnaire-based research, the goal may be
  to learn what respondents
• know (facts)
• think, expect, feel, or prefer (beliefs and
  attitudes) or have done (behaviors).

5
Making sense of the question asked

• The key issue is whether the respondents
  understanding of the question matches what the
  researcher had in mind
• Question comprehension involves
• Comprehending the literal meaning of a sentence
• Inferences about the questioners intention to
  determine the pragmatic meaning of the question

6
The cooperative principal (Grice)

• Four maxims
• Maxim of relation - question should be relevant
  to the aims of the conversation
• Maxim of quantity - question should be as
  informative as required
• Maxim of manner - question should be clear rather
  than obscure, ambiguous or wordy (an
  interpretability presumption)
• Maxim of quality - question should be truthful

7
Questions Aimed at Facts

• Facts include respondents age, education,
  religion, income, marital status, etc.
• What kinds of problems (errors) occur in
  reporting facts?
• Not all respondents have the same understanding
  of what is to be reported
• Knowing the answer and remembering the event
• The form of the answer

8
Reducing response distortion

• Respondents want to make themselves look good and
  to avoid looking bad
• Sensitive questions are risky to answer
  truthfully
• Impression management
• People vary in what they consider to be sensitive
  (e.g., having a library card is over-reported)

9
Questions to Measure Subjective States

• The largest number of survey questions ask these
• These are the hardest type to write. Why?
• Respondents may not have an attitude because they
  never thought about the issue.
• Attitudes are complex and multidimensional.
• Attitudes have a dimension of intensity.
• The result is that expressed attitudes are
  dependent on details of question wording,
  question sequence, and interviewer effects

10
Three components

• What is to be rated
• The dimension or continuum the rated objects are
  to be placed on
• Continuum
• Agree-disagree
• Rank order
• Narrative or open-ended
• The characteristics of the continuum that are
  offered to the respondent

11
Measuring Knowledge

• Asking people to self-report what they know
• True-False
• Multiple choice
• Open-ended short answers

12
The Relativity of Answers

• Question wording
• The wording of response alternatives
• Order of alternatives
• Mode of data collection
• Telephone, Interview, Self-Administered, Computer
• Context Effects
• How dont knows are handled

13
Questions Aimed at Behavior

• The most important guideline is that the question
  be specific.
• The length of time elapsed since the behavior
  will influence the accuracy of the response.

14
Utility Of Self-Report Methods

• Interviews
• The researcher asks questions directly of the
  respondent.
• Interviews typically take place face-to-face or
  on the telephone, but some occur via
  videoconference.
• Questionnaires
• The respondent reads and answers questions
  themselves.
• Self-Recorded Diary
• Respondents keep a record at the time an event
  occurs.

15
What are the advantages and disadvantages each?

• Interviews
• Questionnaires
• Self-Recorded Diary

16
Practical Guidelines For Developing Research
Questionnaires

• 1. Explore context
• 2. Pilot test your questions
• are items worded properly?
• do items lead the respondent?
• do items produce a biased answer?
• 3. Control socially desirable responses.
• randomized response technique to assure
  confidentiality
• measure lying or faking
• 4. Control yea-saying

17
Elements of Questions

• Questions
• Written or oral
• Response Recording
• Interviewer transcription
• Open-ended or verbatim coding
• Getting the Questions