Experimental Research

1
Experimental Research
2
Experimental Research

• What is experimental research?
• Research investigation in which conditions are
  controlled so that hypotheses can be tested and
  alternative explanations can be ruled out
• Research used to make cause-and-effect
  statements (X causes Y)
• X is the independent (or manipulated or causal)
  variable
• Y is the dependent variable

3
Experimental Research

• Evidence of causality (i.e., X causes Y)
• Evidence of concomitant variation (X and Y
  co-vary)
• The more of X, the better chance that we will get
  Y
• Time order sequence of variables
• If X causes Y, X must occur before Y
• Elimination of other possible explanations of why
  Y occurred
• Key is to keep all experimental conditions equal

4
Experimental Research

• Must try to eliminate all possible extraneous
  influences
• Possible extraneous influences
• History - event occurring during the course of an
  experiment (but not really part of the
  experimental manipulation) that influences the
  results
• Maturation - changes which occur in the
  experimental unit which occur during the course
  of an experiment that are due to the passage of
  time

5
Experimental Research

• Possible extraneous influences (continued)
• Testing Effects - changes in experimental unit
  due to the experiment itself (but not related to
  the key experimental manipulation
• Measurement effect -- prior measurements effect
  the measure of the dependent variable
• Interaction effects -- subject pays greater
  attention to stimuli than they normally would
  because its and experiment

6
Experimental Research

• Possible extraneous influences (continued)
• Instrument variation -- changes in the measuring
  instrument cause changes in Y (i.e, interviewer
  changes the way in which they ask questions)
• Selection Bias -- if two groups are compared,
  this notion suggests that the groups may not have
  been equivalent to begin with i.e., a
  manipulation did not cause Y group
  dissimilarities did

7
Experimental Research

• Possible extraneous influences (continued)
• Experimental mortality -- respondents in
  experimental groups were lost during the duration
  of the experiment
• Statistical Regression -- Extreme responses move
  closer to the midpoint during the course of an
  experiment subjects after questionning do not
  want to appear to be extreme

8
Experimental Research

• Experimental Symbols
• X Experimental Treatment (e.g., ad exposure)
• O Observation
• R Random Assignment of Subjects
• Pre - Experimental Designs
• One-Shot Case Study
• X O
• Problems -- Selection Bias, Control Group,
  Mortality, History, etc.

9
Experimental Research

• Pre-Experimental Designs (Contd)
• One Group Pre-Test/Post-Test Design
• O1 X O2
• Analysis -- O2 - O1
• Problems -- History, Maturation, interactive
  Testing Effect, etc.
• Static Comparison
• X O1
• O2
• Analysis -- O2 - O1
• Problems -- Selection Bias, Mortality, etc.

10
Experimental Research

• True Experimental Designs
• Before/After with Control Group
• (R) O1 X O2
• (R) O3 O4
• Analysis (O4 O3) (O2 O1)
• Lessens History and Maturation Problems
• However, interactive testing effect still an
  issue
• O2 and O4 may a function of O1 and O3

11
Experimental Research

• Experimental Designs (True Experimental Designs)
• Six Group - Four Study Design
• (R) O1 X O2
• (R) O3 O4
• (R) X O5
• (R) O6
• O2 O1 compared to O4 O3 tells if X worked
  (manipulation check) If yes, go on
• If O5 - O6 gt O2 O4 all uncontrolled sources of
  variation are accounted for we have true effects

12
Experimental Research

• True experiments are very complex
• Experimental designs that are used are often
  flawed (or not True)
• Random assignment helps

13
Experimental Research

• Analyzing Data
• Typically uses Analysis of Variance (ANOVA)
• Tests (determines if at least one treatment group
  mean is different from the others)
• Ho ?1 M2 M3 Mk
• Ha M1 ? M2 ? M3 ? ? Mk
• Why not use t-tests?
• Probability
• One comparison at 95 confidence 95 of
  rejecting null when it should be rejected
• Two comparisons -- .95 x .95 90.25
• Three comparisons -- .95 x .95 x .95 86
• Like regression
• Dependent Variable (interval or ratio scaled)
• Independent Variable (nominal or ordinal scaled)
• Often group membership

14
Experimental Research (Responses on 1 Bad 7
Outstanding)
Treat/Resp Ad 1 Ad 2 Ad 3
1 1 2 3
2 5 2 6
3 5 5 6
4 6 2 6
5 3 2 7
6 6 2 5
Mean 4.33 2.50 5.50
15
Experimental Research

• One-Way ANOVA
• Attempts to partition variance
• Total Variance (Within Group Between Group)
• Within treatment groups (Observation Group
  Mean)
• SSw S S (x ij Meani)2
• MSw SSw / df w
• Between treatment groups (Group Mean Grand
  Mean)
• SSb S nj (Meanj Mean) 2
• MSb SSb / dfb
• F-statistic
• MSb / MSw
• With dfb, df w
• Follow-up test
• Tukey (most common in MR)

16
Experimental Research

• Two-Way ANOVA
• One dependent variable
• Two or more independent variables
• MANOVA
• Multiple dependent variables
• ANCOVA
• Investigate effects after controlling for
  another variable (that is interval or ratio
  scaled)