Power and Sample Size

1
Power and Sample Size
I HAVE THE POWER!!!

• Boulder 2004
• Benjamin Neale
• Shaun Purcell

2
To Be Accomplished

• Introduce Concept of Power via Correlation
  Coefficient (?) Example
• Identify Relevant Factors Contributing to Power
• Practical
• Power Analysis for Univariate Twin Model
• How to use Mx for Power

3
Simple example

• Investigate the linear relationship (r)
• between two random variables X and Y r0 vs. r?0
  (correlation coefficient).
• draw a sample, measure X,Y
• calculate the measure of association r (Pearson
  product moment corr. coeff.)
• test whether r ? 0.

4
How to Test r ? 0

• assumed the data are normally distributed
• defined a null-hypothesis (r 0)
• chosen a level (usually .05)
• utilized the (null) distribution of the test
  statistic associated with r0
• tr ? (N-2)/(1-r2)

5
How to Test r ? 0

• Sample N40
• r.303, t1.867, df38, p.06 a.05
• As p gt a, we fail to reject r 0
• have we drawn the correct conclusion?

6
type I error rate probability of deciding r ?
0(while in truth r0) a is often chosen to
equal .05...why?
DOGMA
7
N40, r0, nrep1000 central t(38), a0.05
(critical value 2.04)
8
Observed non-null distribution (r.2) and null
distribution
9
In 23 of tests of r0, tgt2.024 (a0.05), and
thus draw the correct conclusion that of
rejecting r 0. The probability of rejecting
the null-hypothesis (r0) correctly is 1-b, or
the power, when a true effect exists
10
Hypothesis Testing

• Correlation Coefficient hypotheses
• ho (null hypothesis) is ?0
• ha (alternative hypothesis) is ? ? 0
• Two-sided test, where ? gt 0 or ? lt 0 are
  one-sided
• Null hypothesis usually assumes no effect
• Alternative hypothesis is the idea being tested

11
Summary of Possible Results

• H-0 true H-0 false
• accept H-0 1-a b
• reject H-0 a 1-b
• atype 1 error rate
• btype 2 error rate
• 1-bstatistical power

12
STATISTICS
Rejection of H0
Non-rejection of H0
H0 true
R E A L I T Y
HA true
13
Power

• The probability of rejection of a false
  null-hypothesis depends on
• the significance criterion (?)
• the sample size (N)
• the effect size (NCP)

The probability of detecting a given effect size
in a population from a sample of size N, using
significance criterion ?
14
Standard Case
Sampling distribution if HA were true
Sampling distribution if H0 were true
P(T)
alpha 0.05
POWER 1 - ?
?
?
T
Effect Size (NCP)
15
Impact of Less Cons. alpha
Sampling distribution if HA were true
Sampling distribution if H0 were true
P(T)
alpha 0.1
POWER 1 - ? ?
?
T
?
16
Impact of More Cons. alpha
Sampling distribution if HA were true
Sampling distribution if H0 were true
P(T)
alpha 0.01
POWER 1 - ??
?
T
?
17
Increased Sample Size
Sampling distribution if HA were true
Sampling distribution if H0 were true
P(T)
alpha 0.05
POWER 1 - ??
?
T
?
18
Increase in Effect Size
Sampling distribution if HA were true
Sampling distribution if H0 were true
P(T)
alpha 0.05
POWER 1 - ??
?
?
T
Effect Size (NCP)?
19
Effects on Power Recap

• Larger Effect Size
• Larger Sample Size
• Alpha Level shifts ltBeware the False Positive!!!gt
• Type of Data
• Binary, Ordinal, Continuous

20
When To Do Power Calcs?

• Generally study planning stages of study
• Occasionally with negative result
• No need if significance is achieved
• Computed to determine chances of success

21
Power Calculations Empirical

• Attempt to Grasp the NCP from Null
• Simulate Data under theorized model
• Calculate Statistics and Perform Test
• Given a, how many tests p lt a
• Power (hits)/(tests)

22
Practical Empirical Power 1

• We will Simulate Data under a model online
• We will run an ACE model, and test for C
• We will then submit our data and Shaun will
  collate it for us
• While hes collating, well talk about
  theoretical power calculations

23
Practical Empirical Power 2

• First get F\ben\2004\ace.mx and put it into your
  directory
• We will paste our simulated data into this
  script, so open it now in preparation, and note
  both places where we must paste in the data
• Note that you will have to fit the ACE model and
  then fit the AE submodel

24
Practical Empirical Power 3

• Simulation Conditions
• 30 A2 20 C2 50 E2
• Input
• A 0.5477 C of 0.4472 E of 0.7071
• 350 MZ 350 DZ
• Simulate and Space Delimited at
• http//statgen.iop.kcl.ac.uk/workshop/unisim.html
  or click here in slide show mode
• Click submit after filling in the fields and you
  will get a page of data

25
Practical Empirical Power 4

• With the data page, use control-a to select the
  data, control-c to copy, and in Mx control-v to
  paste in both the MZ and DZ groups.
• Run the ace.mx script with the data pasted in and
  modify it to run the AE model.
• Report the A, C, and E estimates of the first
  model, and the A and E estimates of the second
  model as well as both the
• -2log-likelihoods on the webpage
  http//statgen.iop.kcl.ac.uk/workshop/ or click
  here in slide show mode

26
Practical Empirical Power 5

• Once all of you have submitted your results we
  will take a look at the theoretical power
  calculation, using Mx.
• Once we have finished with the theory Shaun will
  show us the empirical distribution that we
  generated today

27
Theoretical Power Calculations

• Based on Stats, rather than Simulations
• Can be calculated by hand sometimes, but Mx does
  it for us
• Note that sample size and alpha-level are the
  only things we can change, but can assume
  different effect sizes
• Mx gives us the relative power levels at the
  alpha specified for different sample sizes

28
Theoretical Power Calculations

• We will use the power.mx script to look at the
  sample size necessary for different power levels
• In Mx, power calculations can be computed in 2
  ways
• Using Covariance Matrices (We Do This One)
• Requiring an initial dataset to generate a
  likelihood so that we can use a chi-square test

29
Power.mx 1

• ! Simulate the data
• ! 30 additive genetic
• ! 20 common environment
• ! 50 nonshared environment
• NGroups 3
• G1 model parameters
• Calculation
• Begin Matrices
• X lower 1 1 fixed
• Y lower 1 1 fixed
• Z lower 1 1 fixed
• End Matrices
• Matrix X 0.5477
• Matrix Y 0.4472
• Matrix Z 0.7071

30
Power.mx 2

• G2 MZ twin pairs
• Calculation
• Matrices Group 1
• Covariances ACE AC _
• AC ACE /
• Options MXEmzsim.cov
• End
• G3 DZ twin pairs
• Calculation
• Matrices Group 1
• H Full 1 1
• Covariances ACE H_at_AC _
• H_at_AC ACE /
• Matrix H 0.5
• Options MXEdzsim.cov
• End

31
Power.mx 3

• ! Second part of script
• ! Fit the wrong model to the simulated data
• ! to calculate power
• NGroups 3
• G1 model parameters
• Calculation
• Begin Matrices
• X lower 1 1 free
• Y lower 1 1 fixed
• Z lower 1 1 free
• End Matrices
• Begin Algebra
• A XX'
• C YY'
• E ZZ'
• End Algebra
• End

32
Power.mx 4

• G2 MZ twins
• Data NInput_vars2 NObservations350
• CMatrix Full Filemzsim.cov
• Matrices Group 1
• Covariances ACE AC _
• AC ACE /
• Option RSiduals
• End
• G3 DZ twins
• Data NInput_vars2 NObservations350
• CMatrix Full Filedzsim.cov
• Matrices Group 1
• H Full 1 1
• Covariances ACE H_at_AC _
• H_at_AC ACE /
• Matix H 0.5
• Option RSiduals