Statistics for Psychology

1
(No Transcript)
2
Some issues in the design and planning of
experiments

• J Connolly
• Department of Statistics

3
Experimentation

• A growth in knowledge - rarely through a single
  experiment. Modify the hypothesis in the light of
  data.
• Hypothesis
• ?
• Experiment
• ? ?
• Data ?
• ? ?
• Modify hypothesis
• ?
• Conclusion

4
Valid and efficient comparative experiments

• Validity - the first requirement
• Independent experimental units - replication
• Randomisation of units to treatments
• Efficiency - minimise replication
• Exploiting structure in the experimental material
  Blocking to reduce variation
• Exploiting structure in experimental treatments
  Factorial design

5
Experimental unit
The experimental unit is the basic independent
unit to which a treatment is applied at random.
It is the basic unit against whose variation the
treatment differences are judged. It is the
independent member of the populations to be
compared.
Replication and pseudoreplication Example 1 Two
groups of three rats fluke infected vs.
non-fluke infected. Three Peyers patches (an
aggregate of lymphoid cells in specific areas of
the gut of the rat) to be tested per rat (post
mortem) for each of the three rats. Is this 9
replicates?
6
Example 2 Three drugs to be tested. Three drugs
assigned at random to three Peyers patches from
a rat (post mortem) and this process repeated for
six rats. Is this 6 replicates or is it
pseudoreplication?
7
Efficient Resource Use.

• Sample size calculations for experiment
• Experiment design to improve efficiency and
  reduce numbers

8
Sample size calculations for experiments Size
of study

• Design stage - how many replicates per treatment
  should I use?
• Pilot study - how does this help to determine the
  answer?

9
Importance?

• Expense (animal numbers) of a badly designed
  investigation if resources too great or too
  small.
• If resources too great - reduce or add treatments
• If resources too small use fewer treatments

10
Issues in determining sample size

• Variability of experimental material (CV)
• Size of effect expected or of interest to detect
  (d)
• Type I error rate
• Type II error rate
• The treatment structure

11
Variability of experimental material (SD or CV)

• SD standard error of a measurement Typical
  deviation of a response from the population mean,
  
• CV 100 x SD/Mean where mean is a typical mean
  for the response. Estimate CV from literature or
  experience. Typical values
• Chemical analyses 2 - 5
• Physiological responses ?
• Animal behavioural responses ?

12
Size of effect expected or of interest to detect
(d)

• From an understanding of the biological action of
  the treatments
• The size of effect likely to be of
  economic/practical value
• An international or professional standard that
  must be complied with and which, if exceeded,
  will imply that the process is seriously defective

13
Type I and II error rates

• In conducting tests of significance we can make
  two kinds of mistake
• Error type I - concluding that there is a real
  difference when there is not.
• Error type II - concluding that there is not a
  real difference when there is.
• Danger with error type II negative results may
  be interpreted as implying that no effect exists.

14
Type I error rate

• t Statistic (difference between two means/SED)
• The first error rate, the significance level, is
  chosen by the experimenter and is conventionally
  one of 5, 1 or 0.1 or (P0.05) (P0.01) or
  (P0.001). Each error rate gives a threshold
  value that must be exceeded for significance.
• Interpretation in 5 of experiments in which
  there is no real treatment effect the t statistic
  will exceed the threshold value
• Choice of Type I error rate. The smaller the rate
  chosen the stronger the evidence that there is an
  effect of treatment.

15
Type II error rate (missing a real effect)

• High error rate II means that a real difference
  (d) between treatments is unlikely to be
  detected.
• This is wasteful of resources. A rate of 0.5
  means that 50 of experiments will not detect
  effect.
• Power 1 - Type II error rate probability that
  the experiment will detect a real difference.
• Type II error rate depends on
• d (Size of effect)
• CV
• Replication
• Type I error rate

16
Criterion for Design

• Sufficient replication to ensure that a
  difference of size d will have a high probability
  (power 100-Type II error rate) of being
  detected as significant.

17
Formula for replication

• formula for the replication required per
  treatment (r)
• CV Coefficient of Variation
• d difference of importance.
• Chance of concluding that Treatments
  differ when they don't. (Error Type
  I) 5
• Chance of concluding that Treatments do
  not differ when they differ by d
  (Error Type II) 20
• r replication per treatment
• r 16 (CV/d)2

18
Example

• CV 15
• d 10
• Error Type I (significance level) 5
• Error Type II 20 (or Power 80)
• r 16 (15/10)2 36
• Note the square in the formula which accentuates
  the effect of variability.

19
Comments

• For more detail see Cochran and Cox, Experimental
  Designs. John Wiley or Snedecor and Cochran,
  Statistical Methods. Iowa State University Press.
  
• The multiplier 16 depends on the Type I and Type
  II error rates. Reducing either of these error
  rates increases the multiplier and hence the
  replication required.
• Continuous responses assumed - for yes/no
  responses see Cochran and Cox.

20
Estimating the SD and CV from the literature
SD is estimated by SE of mean SD is
estimated by SED SD is estimated by CV
100 SD/Mean
21
Example Cadogan et al. (1999) American
Physiological Society 122, 1616-1627

Variable - Change in PAP
22
Calculations
23
Example OConnor et al., (1992) Brain
Research 573209-216

24
Calculate


25
Comments on literature derivation

• Is standard deviation constant?
• Across treatments
• Does standard deviation change with mean?
• Transform data prior to analysis
• Can also use the length of error bars to get an
  approximate idea of SEM

26
Design for efficiency

• Minimise
• Design to reduce noise (?2) - block designs that
  group similar units and apply treatments within
  groups.
• Design to widen scientific scope and reduce
  replication required (r) - Factorial design

27
Block design - randomised block

• Example Three drugs to be tested. Three drugs
  assigned at random to three Peyers patches from
  a rat (post mortem). The process is repeated for
  six rats. Treatments compared within a rat, less
  variable than between rats

28
Factorial experiments

• Scientifically desirable - allows consideration
  of several factors affecting a process within one
  experiment.
• Statistical - powerful methodology
• Single factor experiments - May give clear
  results but they may be of limited generality.
  Information on response to the factor under
  different conditions is unavailable.

29
Factorial Designs - Summary

• Scientific advantages
• Broader basis for inference if no interaction
• Information on interaction if there is one
• Statistical efficiency advantage
• If there is interaction it can be estimated.
• If there is not interaction the experiment uses
  the same experimental units for estimation of a
  number of different factors hidden replication.

30
Example

• Studies on artificial insemination with 64
  heifers for each of four weeks. 4 factors each at
  2 levels. Two choices-
• A) 4 experiments, 1 per week
• B) 2 x 2 x 2 x 2 factorial over the 4 weeks.

31

• A) 32 animals assigned to each treatment.
• SED ?(2 S2/32) S/4
• where S2 is the estimate of noise.
• B) 16 treatment combinations tested on 4 animals
  in each week.
• To compare the levels of any factor
• SED ?(2 S2/128) S/8.

32
Summary - The factorial design

• - Halves the SED and quarters the number of
  animals required for a given level of precision,
• - Allows more general interpretation of the
  factor effects since they are tested over a wide
  range of levels of the other factors
• - Allows a test of whether the factors interact.
  

33
Thanks for listening,The End.
34
Automated alternative approach using
EXCELONeill and Thompson Sample size
calculations for comparison of two treatments
35

• Compute the minimum detectable difference
  between two treatments for specified
  levels of
• Replication
• Coefficient of Variation (CV)
• Level of significance
• Power (specified in SOLVER)
• Graph for various levels of replication or CV or
  Significance level or power

36
A range of values of Replication CV 10, Type I
0.05 and Power 0.8
37
A range of values of CV r 10, Type I 0.05
and Power 0.8
38
A range of values of Replication CV 10, Type I
0.05 and Power 0.8 or 0.9