Experimental Design

1
Experimental Design Analysis

• Analysis of Covariance Within-Subject Designs
• March 13, 2007

2
Outline

• Blocking vs. analysis of covariance
• ANCOVA
• Within-subject designs
• Greenwald

3
Randomized Block Design

• Completely randomized designs based on assumption
  that random assignment diminishes systematic bias
• Blocks are experimental counterparts of
  covariates
• Identify nuisance variable
• Randomly assign within blocks

4
Randomized Block Design

• Example Experiment about effect of 4 types of
  training programs on learning
• Completely randomized design N60, a4 levels,
  randomly assign 15 subjects to each of 4 levels
• Randomized block design assume intelligence
  could confound results, categorize subjects
  according to IQ or GPA (high, medium, low),
  assign 20 to each block and randomly assign
  within each block to one of 4 conditions

5
Randomized Block Design

• Code block as a factor
• Do not want treatment x block interaction
• More blocks require more subjects

6
Analysis of Covariance

• The main purpose of ANCOVA is statistical control
  of variability when experimental control can not
  be used
• Statistical method for increasing power of ANOVA
  by reducing mean square error (within-condition
  error)
• ANCOVA can be used to correct for extraneous
  variables and rule out rival explanations

7
Analysis of Covariance

• ANCOVA is like ANOVA on the residuals of  the
  values of the dependent variable, after removing
  the influence of the covariate, rather than on
  the original values themselves
• In so far as the measures of the covariate are
  taken in advance of the experiment and they
  correlate with the measures of the dependent
  variable they can be used to reduce experimental
  error
• Based on partial correlation analysis

8
Analysis of Covariance

• Examples of covariates
• If we have a theory about two different methods
  of learning and our dependent variable is memory,
  level of education, intelligence, IQ, experience
  and age may all be covariates
• If we have a theory about the effect of
  management style on firm profitability, some
  covariates may be firm size, competition, length
  of CEO tenure
• If we have a theory about the relationship
  between number of outstanding shares and share
  price, we may consider market capitalization,
  earnings per share and IPO year as covariates
• If we have a theory about the interactive effects
  of advertising and sales force training, some
  covariates to consider may be brand equity of
  product, size of sales force and past advertising

9
Analysis of Covariance

• One way of understanding ANCOVA is in terms of
  deviations from a common regression line
• If the regression lines of the dependent variable
  vs. covariate have same slopes but different
  intercepts, the effect of the treatment is
  consistent
• If the regression lines of the dependent variable
  vs. covariate have different slopes, the effect
  of the treatment depends on the value of the
  covariate

10
Analysis of Covariance

• ANCOVA is based on the sums of squares and sums
  of products
• It is used to test the same hypothesis as
  standard ANOVA
• The only difference is that each of the sums of
  squares are adjusted on the basis of the
  covariate variable
• The covariate reduces the amount of experimental
  error
• In calculations, reduce degree of freedom by 1
  for each covariate

11
ANCOVA Assumptions

• Dependent variable continuous
• Covariate may be continuous (like regression) or
  discrete (like ANOVA)
• Factor has discrete levels
• Variables are normally distributed
• Relationship is linear

12
ANCOVA

• Multiple covariates possible
• Should be theoretically motivated
• How much variance does covariate account for?

13
Within-Subject Designs

• Advantages of within-subject designs
• Economy
• Power
• In fully crossed designs, random assignment
  permits the assumption of equivalence (subject
  comparability). In within-subject designs,
  subjects are the same, thus removing source of
  unwanted variability and reducing the error term
• Usefulness
• Allows study of behavioral/attitudinal change and
  learning

14
Within-Subjects Designs

• Within-subjects designs involve applying all
  treatments to the same individuals
• Think about within-subjects designs as a 2-way
  ANOVA where the columns are the treatments and
  the rows the individuals, with one observation
  per cell
• This design introduces the idea of individuals as
  a random factor that crosses or intersects other
  factors in the design
• Within-subjects, or repeated measures design, has
  advantages and disadvantages
• Advantage is that each individual serves as his
  or her own control, thus a source on experimental
  error resulting from individual difference is
  controlled for
• Disadvantage is that it introduces dependencies
  across the treatments, known as carryover effects

15
Within-Subject Designs Economy

• For a 2x2 fully-crossed factorial
• N40 (different subjects in each cell)

• For a 2x2 within-subject design
• N10 (same 10 subjects in each cell)

16
Within-Subject Designs Power

• Consider example in which 1 factor (A) is
  manipulated
• Because factor A is completely crossed with
  subjects factor S, we denote as AxS
• Analogous to two-factor design AxB in which
  variability is SStotal SSA SSB SSAB
  SSerror
• Examine claim of greater power
• SSerror SStotal SSeffects
• In one-factor example SSerror SStotal SSA -
  SSS

17
Within-Subject Designs

• Consider complete within-subjects 2x2 design
  (each subject sees a1b1, a1b2, a2b1, a2b2
  conditions)
• Sources of variability A, B, S, AxB, AxS, BxS,
  AxBxS
• To test effects compare mean square of effects
  (A, B, AxB) with mean square for effects with
  subjects (MSAxS, MSBxS, MSAxBxS)

18
Within-Subjects Designs

• Example
• Lets say that we are interested in the effect of
  different types of exercise on memory
• We use two treatments, aerobic exercise and
  anaerobic exercise
• In the aerobic condition, participants run in
  place for five minutes, after which they take a
  memory test
• In the anaerobic condition they lift weights for
  five minutes, after which they take a different
  memory test of equivalent difficulty
• In a within-subjects design all participants
  begin by running in place and taking the test,
  after which the same group of people lift weights
  and then take the test
• We compare the memory test scores in order to
  answer the question as to what type of exercise
  most aids memory