Qualitative and Quantitative Methods of Survey Design

1
Qualitative and Quantitative Methods of Survey
Design

• What to do from start to finish

2
ATopic

• The first step is to identify a specific research
  question and then become an expert in providing
  the answer.
• I keep a working list of everything that I want
  to know---every single question.
• Example Are there differences among men and
  women? Old and young? By Social class.

3
How to Go About Being an Expert

• A point of agreement a researcher must be
  fully informed about a phenomenon in order to
  measure it in a way that produces rich
  quantitative data !
• I have found it extremely critical to learn how
  people experience, think, and talk about a topic
  BEFORE formulating (quantitative) questionnaire
  items to answer all of my research questions.

4
The most productive qualitative methods

• Personal interviewsone on one.
• Without an interview schedule.
• With an interview schedule.
• Group interviews/focus groups
• With an interview schedule

5
How to use the interview

• Identify the population of interest. It helps to
  identify or find typical cases.
• Begin without a list of questions.
• Take the subject(s) on a mental tour.
• A mental tour involves placing subjects in their
  setting, and recalling the time and place where
  they experience the phenomenon one wishes to
  study.

6
Qualitative Methods

• Yes, one should tape record OR have someone
  manually record a subjects words.
• Yes, one must have Human Subjects approval
• The task is to listen and probe
• Example Can you think of other instances when
  _________. How is this different than
  _____________.
• Comparison questions are ALWAYS helpful.

7
Qualitative Methods

• After the interview, I read the transcript, and
  ask myself these types of questions
• Who else experiences this in the same way? Who
  might experience it differently?
• What else is involved in this experience?
• What is the context? What is the role of
  context?

8
James Spradleys Matrix

• Space
• Object
• Act
• Activity
• Event
• Time
• Actor
• Goal
• Feeling

9
Qualitative Methods

• The ideal number of interviews is.
• The ideal is to continue doing these until one
  no longer learns new information.

10
Qualitative Methods

• Focus groups are more efficient, but more
  difficult to organize.
• Focus groups are small (6-10), structured, group
  interviews where subjects talk to each other
  rather than to the researcher.
• Focus groups benefit from a trained moderator.

11
How many focus groups?

• As many as one can afford, OR
• One continue until no new information emerges.

12
An aside

• Personal interviewees and focus group members
  identify people with a potential to become the
  researchers consultants!
• They can pretest surveys
• They can assist in providing qualitative
  interpretations on your quantitative data

13
What next?

• Qualitative data provides the many
  dimensions---and the ways of talking about all
  aspects of a research question.
• I use the information gleaned from a close
  reading of the transcripts to reformulate my
  research questions and to formulate my
  questionnaire items.

14
Five kinds of measures

• What kind of information might I measure?
• What people say that they do (behaviors)
• What people think is true (beliefs)
• What people are (attributes)
• What people say they want (their attitudes)
• Dillman, Don A. (1978). Mail and Telephone
  Survyes The Total Design Method. NYC John
  Wiley. P. 80

15
Examples

• Behaviors
• On an average weekday, how much times do you
  think about your breast cancer?
• Attitudes
• In your opinion, how do you feel about the
  legalization of marijuana?
• Strongly oppose/Somewhat oppose/neither oppose or
  favor/favor somewhat/strongly favor
• Beliefs
• In your opinion, does chemotherapy always cause
  hair loss?
• Always Yes/Usually Yes/Sometimes/Seldom/Almost
  Never/Never

16
Types of Qs, cont.

• Attributes/demographic/Profile Questions
• What is your marital status?
• 1 Never married 2 Married
• 3 Separated 4 Divorced
• 5 Widowed
• How many years of education have you completed?
  Please tell us the actual number of years
• __ __ years

17
Content Types of Qs

• Open-ended questions subject write their answers
  in a few words
• Describe some of the things that you do around
  the house_________________
• Closed-ended questions coded categories are
  providedthese can be analyzed more easily
• Please circle all of the things the things that
  you have done in your household during the past
  seven days
• 1 Bought groceries 2 Packed lunches
• 3 Cleaned 4 Ironed
• 5 Cooked 5 Made bed

18
Open-ended Questions

• Should be used sparingly.
• Are more appropriate for an exploratory study.
• Data must be encoded so that it can be analyzed
• This is time-consuming
• Reliable codes is an issue

19
Closed-ended Questions

• The responses to closed-ended questions can be
  entered directly into a statistical program or
  spread sheet.
• Closed-ended questions require categories that
  are mutually exclusive (only one response
  category per question) and exhaustive (a category
  for each person).

20
Content Writing the Qs

• Language Most people agree that questionnaires
  should be written in the language of the
  population of interest.
• If qualitative data are the basis of the
  questionnaire items, this should be relatively
  straightforward.
• When it comes to forming items for the
  questionnaire, here are a few things I check,
  recheck, and check again
• That language is neutral, not emotional
• That the subject is not led to socially desirable
  answers.
• That the language is unbiased
• That words are short---not too many letters or
  syllables
• That words are precise and specific rather than
  vague so that everyone understands the items the
  way one intends it.
• That language is common rather than haughty or
  academic

21
What should I ask?

• How to decide what to ask
• Keep a focus on the research question.
• Keep that list of the related questions.
• Make dummy tables!!!!!

22
Format

• Questionnaires should be formatted to promote
  visual appeal.
• PAY someone well to do this!!!!!
• Consider the flow of items
• Use transitions or Headings
• Include codes that can be entered directly into
  your statistical software. This increases the
  reliability of data entry.

23
Closing Thoughts

• Research is creative---thats the allure.
• Everyone has to find their own style
• Everyone has to find what works for them.
• What I have shared today is what works well for
  me.
• Research is time-consuming---thats the greatest
  difficulty.
• Collaborate to reduce the amount of labor and to
  increase the fun.
• Ask for help Teri Peterson, Statistical
  Consultant Ann Hunter, Sociology.