FDN 5000 Research in Education Dr. George H. Olson, Instructor

1
FDN 5000 Research in Education Dr. George
H. Olson, Instructor

• Research Designs
• Descriptive Research Designs
• Correlation Research Designs
• Causal-comparative Designs
• Experimental Designs
• Non-experimental Designs
• Single-subject designs

2
Purposes of Research Designs

• Provide guidance in conducting research
• Provide guidance in interpreting research
• Guidance in recognizing types of research
• Guidance in recognizing potential threats to the
  validity of research conclusions
• Help guard against threats to internal and
  external validity

3
Descriptive research designs

• Survey research
• Longitudinal research designs
• Cohort designs
• Quasi-cohort designs
• Cross-sectional research
• Case studies
• Ethnographic studies

4
Correlational research designs

• Objective study the relationship between
  variables
• Examine scatter plots (e.g., pages 223 and 225 in
  our text)
• Compute correlation coefficients
• range from -1 through 0 to 1
• do not imply causation
• lack of correlation does NOT imply no causation

5
Interpreting correlations

• Rules of thumb
• Correlation coefficient Strength of
  relationship
• .00 to .20 negligible
• .24 to .40 low
• .40 to .60 moderate
• .60 to .80 substantial
• .80 to 1.00 high, very high

6
Causal comparative research designs

• A.k.a. ex-post-facto designs
• Aimed at discovering cause and effect
  relationships
• Defined groups are compared after they have been
  formed
• Theory plays an important role
• In arguing for the cause-effect relationship
• In eliminating rival explanations

7
Causal comparative designs An example

• Of students receiving the This proportion
  is known to
• following grades. drive a car to school
• A 8
• B 23
• C 43
• D 77
• F 96
• What should the superintendent conclude?

8
Diagramming Experimental Research Designs

• Symbols used
• T treatment intervention
• C control or comparison condition (often simply
  no treatment)
• O observation (often some test score)
• R designates random assignment
• M designates matching

9
Examples of Research Design Diagrams

• (1) T O
• (2) O1 T O2
• O1 T O2
• (3) ----------------------- (No Random
  Assignment)
• O1 C O2

10
Three pre-experimental designs

• Three designs frequently used in education
  research that
• are not sufficient for permitting strong tests of
  causal hypotheses
• often due suggest new ideas
• One-group posttest-only design
• One-group pretest-posttest design
• Comparison-group posttest-only design

11
One-group, posttest-only research design

• T O
• A treatment followed by an observation
• Should not be confused with the one-shot case
  study
• Threats to internal validity
• ALL (except regression and mortality)
• Threats to external validity
• ALL

12
One-group pretest-posttest research design

• Opre T Opost
• One of the most frequently used research designs
  in education
• Threats to internal validity
• extraneous events (history and maturation)
• statistical regression
• testing
• experimenter and subject effects
• Threats to external validity
• selection and settings interactions with treatment

13
Comparison-group, posttest-only design

• T O
• --------------
• C O
• --------------
• C2 O
• --------------
• C3 O

14
Comparison-group, posttest-only design

• Threats to internal validity
• Since the comparison groups are non-equivalent,
  the major threat is selection
• Other threats include mortality, and subject and
  experimenter reactive effects
• Threats to external validity
• Selection and settings by treatment interaction

15
True experiment research designs

• Randomized experiments
• Result in probabilistic equivalence
• Not a panacea that rules out all threats to
  internal validity
• Does not control for experimenter and subject
  reactive effects.
• Does not guarantee group equivalency (especially
  in small samples).

16
Randomized posttest comparison group design

• T O
• R ---------------
• C O
• Note R means RANDOMIZATION

17
Randomized pretest-posttest control group design

• Opre T Opost
• R ---------------------------
• Opre C Opost
• R ---------------------------
• Opre C2 Opost

18
Randomized matched-group design

• T O
• M R ---------------
• C O

19
Randomized factorial designs

• TA1,B1 O
• ---------------
• TA1,B2 O
• R ---------------
• TA2,B1 O
• ---------------
• TA2,B2 O

20
Factorial Design Example

• Method (B)____
• Word Type (A) Computer Handwriting
• B1 B2
• Easy A1 20 26
• Hard A2 16 20
• ____________________________

21
Quasi-experiments Time series designs

• O1 O2 O3 O4 T O5 O6 O7 O8
• Pre-observations to establish a baseline
• A treatment intervention
• Post-observations to establish new baseline

22
Quasi-experiments Non-equivalent control groups

• In these designs, randomization is either not
  possible or not feasible.
• Characterized by ...
• using intact groups for treatment and comparison
• manipulated independent variable
• Often, the best we can expect from education
  research

23
Non-equivalent, control group, pretest-posttest
design

• Opre T Opost
• -------------------------
• Opre C Opost
• Except for reactive effects, most threats to
  internal validity are controlled
• Again settings and selection by treatment
  interactions pose threats to external validity

24
Matched comparison group, posttest design

• T O
• M ---------
• C O
• Validity depends upon how well matching is
  achieved
• Potential threats to internal validity are same
  as those for posttest-only designs

25
Single-subject designs

• Similar to time-series designs, only with a
  single individual
• Repeated measurements over time (baselines)
• Subjects serve as their own controls
• Involve a manipulated independent variable (the
  intervention)

26
Basic single-subject designs

• Reversal A - B - A
• Double reversal A - B - A - B
• Multiple baseline
• A - B - A
• ------------------------
• A - B - A
• ------------------------
• A - B - A
• ------------------------
• A - B - A
• A is a period of no treatment
• B is a period of treatment

27
Example of a stable baseline
28
Example of an increasing baseline
29
Non-experimental research designs

• Characterized by the lack of manipulation of an
  independent variable
• Three types of non-experimental research designs
• Causal comparative research designs
• Correlational research designs
• Descriptive research designs