Quasiexperimental Designs, Experimental Designs, and SingleSubject Designs

1
Quasi-experimental Designs, Experimental Designs,
and Single-Subject Designs

• OT 667
• July 2, 2003

2
Experimental Design

• Control Group
• Random Assignment
• Manipulate a variable
• (treat one group but not the other, OR use 2
  different treatments)

3
Random Assignment

• Subjects have an equal change of being assigned
  to the control or treatment group
• Assures group equivalence

4
Manipulation of Variables

• Active variable one that is manipulated by the
  researcher
• Attribute variable traits of subjects, such as
  age, diagnosis, gender, etc cannot be
  manipulated by the researcher
• Can manipulate one or more active variables at
  the same time
• Can also assign people with various attribute
  levels to receive the active variable

5
Blinding in Research
The purpose of blinding is to minimize bias in
the process of recording or collecting
information about the outcome of the treatment.
6
So who gets blinded?

• Subjects
• Persons administering the treatment
• Persons collecting the data
• Persons analyzing the data

7
Quasi-experimental Research
8
Quasi-experimental designs are those studies
which do not use - random assignment - a
control group
9
Rationale for Using Quasi-Experimental Design

• Clinical limitations space, time, money,
  equipment/personnel needed to provide
  treatments.
• Use of quasi-experimental design can be a step to
  collect data to apply for funding for
  experimental design
• Ethical issues random assignment and use of a
  control group are not always acceptable to the
  population or sample under study

10
Kinds of Quasi-Experimental Designs

• One group pretest-posttest design
• Time Series design
• Nonequivalent pretest-posttest design
• Nonequivalent posttest only design

11
Comparison of Nonequivalent Pretest-Posttest
Control Group Designs and Nonequivalent
Pretest-Posttest Designs

• Nonrandom selection
• Control group
• Baseline measures and post-intervention measures
  collected

• Nonrandom selection
• No control group
• Baseline measures and post-interventions measures
  collected

12
Nonequivalent Posttest-Only Control Group Design

• Two groups involved which are selected, not
  randomly assigned
• Intervention administered
• Posttest measures taken on the variable under
  study

13
The definition of time-series designs in your
text is the same as single subject, classifying
this design as quasi-experimental. Not everyone
agrees with this classification.
14
Does the experimental treatment really cause the
observed change in the dependent variable?This
question addresses the internal validity of a
study.
15
Threats to Internal Validity

• History
• Maturation
• Attrition
• Testing
• Instrumentation
• Statistical Regression
• Selection
• Ambiguity about Direction of Causal Influence
• Treatment Diffusion
• Demoralization of respondents receiving treatment
  assumed to be less desirable

16
Threats to Internal Validity

• History - events occurring after administration
  of the independent variable that could affect or
  cause outcomes
• Maturation - changes in behavior/skills occurring
  over time after administration of the independent
  variable
• Attrition - subjects who drop out of the study,
  causing groups to be asymmetrical

17
Threats to Internal Validity

• Testing - the effects of having repeated tests on
  behaviors under study OR the effect of certain
  kinds of assessment on subjects behavior
• Instrumentation - the effect of instrumentation
  with poor reliability on measures
• Statistical regression - another reliability
  concern extreme scores on a given variable can
  change to approach the mean over time (regression
  to the mean) a real issue when subjects are
  chosen because of extreme scores on a variable

18
Threats to Internal Validity

• Selection - When subjects are selected for groups
  rather than randomly assigned, the groups end up
  being different. Matching can help but does not
  balance out the selection factor completely
• Diffusion - occurs when one group becomes aware
  of the treatment used for the other group and
  integrates some aspect of the treatment into
  their own intervention
• Compensatory Equalization - when persons
  delivering one intervention believe the treatment
  the other group receives is more desirable and
  works harder, ultimately delivering compensatory
  services

19
Threats to Internal Validity

• Compensatory rivalry and resentful demoralization
  - this is the situation when the participants in
  one group believe the other groups treatment is
  desirable and work extra hard to compensate,
  thereby affecting the outcomes. Resentful
  demoralization occurs when one group becomes
  angry with the treatment they receive and dont
  work as hard as they should
• Ambiguity about causal influence - when the
  cause-effect sequence is unclear - does the
  independent variable cause the dependent variable
  or vice versa?

20
Threats to Internal Validity

• Selection and history - when the history of
  non-randomized subjects interacts with selection
  to influence the outcomes
• Subjects and maturation - when selected subjects
  in experimental and control groups mature at
  different rates, affecting the outcomes
• Subjects and instrumentation - when the
  instrumentation results in different measures for
  each group involved in the study

21
WHEW!
22
Can the results of the study be generalized to
the population of whom the sample is
representative in a given study? This question
asks about the external validity of a study.
23
Threats to External Validity

• Interaction of Treatment and Selection
• Interaction of Treatment and Setting
• Interaction of Treatment and History

24
Your challenge in all future empirical readings
(experimental and quasi-experimental) is to
consider threats to internal validity and discard
threats if at all possible.