Research Design

1
Research Design

• Session 12

2
Research Design

• Well, lets define it . . .
• Research design is the scheme that guides the
  process of collecting, analyzing, and
  interpreting data. It is a logical model of
  proof that allows the making of valid causal
  inferences.
• Less formally stated . . . it is the plan for
  your research that best establishes causal
  relationships, and provides the best way to
  answer the questions that you have.

3
Elements of Research Design

• Which . . . ?
• variables which ones best measure the phenomena
  that interest me?
• analytical strategy is this a question for an
  experimental design, a statistical analysis?
• theory do I think there is reciprocal causality
  here, multi-stage causality, causality over time
  . . ?
• hypotheses can I actually test the questions I
  have raised here with the design, variables, etc.
  that I have chosen?

4
Experimental Research Designs

• Experimental Design includes
• Experimental group
• Control group
• Stimulus or treatment (terms can be used
  interchangeably)
• Test

5
Variations in Experimental Design

• Classic Experimental Design
• Solomon Two-Control-Group Design
• Solomon Three-Control-Group Design
• Assigning Cases to Groups
• Precision Matching
• Frequency Distribution Control
• Randomization

6
Field Experiments and Nonexperimental Design

• Field Experiments
• Real world setting
• Careful selection of subjects and random
  assignment of subjects to the test and control
  groups to maximize control over background
  characteristics that might influence results
• Monitoring of subjects circumstances throughout
  the experiment to prevent alternative causality

7

• Nonexperimental Designs
• Researcher cannot control the assignment of
  subjects to experimental groups nor the
  occurrence of the independent variable
• Cannot obtain pretest scores on the dependent
  variable
• May use after-only design a single observation
  of the presumed causal event

8
Quasi-Experimental Designs

• Quasi-Experimental Designs
• Researcher does not control exposure to the
  independent variable or accompanying conditions
• Researcher creates conditions of an experiment
  though collection of additional data

9
Quasi, Continued

• Ex Post Facto Experiment
• Researcher makes a single observation and
  collects data about the independent and dependent
  variables
• Time Series
• Researcher makes several observations both before
  and after the introduction of some causal
  phenomenon and compares values on the dependent
  variable before and after
• Researcher observes the trend line

10
Quasi, Continued

• Controlled Time Series
• Researcher gathers data on a group similar to the
  original group except in exposure to the
  treatment to use as a control group

11
V is for . . . No, Not Victory

• Internal Validity the degree to which the
  design provides a sound logical basis for
  inferring that the independent variable does or
  does not cause a change in the dependent variable
• External Validity the generalizability of our
  results

12
Factors that Threaten Validity

• Internal History, Maturation, Instability,
  Testing, Instrumentation, Regression Artifacts,
  Selection, Experimental Mortality
• External Testing, Treatment, Irrelevant
  Response, Replicability