Quasi-experimental Designs

1
Quasi-experimental Designs SurveyResearch
2
Quasi-experiments

• Hedrick, Bickman Rog (1993) a
  quasi-experimental design is not the method of
  choice, but rather a fallback strategy
• Cannot infer cause and effect, BUT well designed
  quasi-experiments enable you to demonstrate that
  rival interpretations are rendered unlikely

3
Non-equivalent Group Designs

• Posttest only nonequivalent control group design
  (aka static group comparison)
• X O (treatment grp)
• O (nonequivalent control grp)
• Xtreatment Omeasurement/observation
• Because there is no random assignment to groups,
  confounding variables may explain any difference
  observed

4
Pre-test/Post-test Non-equivalent Control Group

• O X O treatment group
• O O non-equivalent control group
• Pre Post

5
Example Research Methods and Reasoning Ability

• Intervention critical thinking seminar
• Research Methods students receive the
  intervention (i.e., participate in the critical
  thinking seminar)
• Developmental Psychology students are used as a
  nonequivalent control group
• (i.e., do not attend the seminar)

6
Interrupted Time Series Designs

• Extension of the simple one group pre and post
  design
• Participants are pre-tested a number of times and
  then post-tested a number of times after being
  exposed to the treatment intervention
• O1 O2 O3 O4 X O5 O6 O7 O8

7
Interrupted Time Series Designs

• Useful when
• cannot randomize participants
• it is possible to obtain a series of assessments
  of the DV before and after treatment

8
Example Interrupted Time Series Designs

• Intervention Course to change students study
  habits, implemented during the summer (after
  semester 4).
• DV semester GPA

Post-tx
Baseline
X
9
How Many Measurements Are Needed for a Time
Series Design?

• Depends on
• amount of random fluctuation (noise) that may
  occur in the outcome being measured
• how much of an impact the intervention is
  expected to have

10
Multiple Time Series Designs

• Add a comparison group to the simple interrupted
  time series design
• O1 O2 O3 O4 X O5 O6 O7 O8
• ------------------------------------------------
  ------------------
• O1 O2 O3 O4 O5 O6 O7 O8
• Campbell Stanley consider this an excellent
  quasi-experimental design

11
Survey Research
12
Surveys

• Items are composed of 2 parts stem response
  options
• The university should not raise tuition under any
  circumstances (stem)
• Agree Disagree (response options)

13
Question Wording

• Important consideration when writing questions
• Simplicity
• Avoid jargon and technical terms
• Avoid abbreviations
• Be specific
• Avoid double-barreled questions -

14
Responses to Questions

• Open-ended
• Closed-ended

15
Open vs. Closed Questions
closed questions should be used where
alternative replies are known, are limited in
number, and are clear cut. Open ended questions
are used where the issue is complex, where
relevant dimensions are not known, and where
process is being explored. Stacey (1969)
16
Rating Scales

• Likert Scales
• Semantic Differential
• Graphic Rating Scale
• Non-verbal scale

17
Question Placement

• Very important
• Confusing questionnaire demoralizing to
  respondents
• Can affect peoples responses to them

18
Layout of Questionnaire

• Use closed-ended questions
• Use a consistent item format
• Dont crowd questions together
• Dont split questions or response options between
  pages
• It is most likely to be completed if easy on the
  eye, relevant, logical and as short as possible.

19
Response Biases

• Social desirability
• Acquiescence response - tendency either to
  agree/disagree with statements
• Yea-Saying
• Nay-Saying

20
Checklist for Survey Use

1. Do the items provide a representative sample of
   the behaviour under study?
2. Do the items ask for irrelevant information?
3. Does the question have the same meaning for
   researcher and respondent?
4. Is the question worded clearly? Are the words too
   difficult?
5. Does the question concern a sensitive issue?

21
Checklist for Survey Use

• Does the question suggest a socially desirable
  answer?
• Open vs closed questions?
• How many questions are necessary to obtain the
  required information?
• How might the questions be arranged to encourage
  the respondents participation?