Chapter 27: More tests for averages:

1
Chapter 27 More tests for averages
1. The standard error for a difference Thus far
we have been comparing the average of a single
sample to an expected value defined by the null
hypothesis Now we will be comparing the sample
averages of 2 separate samples This requires
that we obtain the SE for the difference between
the two sample averages
B
A
For example
average 90 SD 40
average 110 SD 60
400 draws at random with replacement
100 draws at random with replacement
2
Chapter 27 More tests for averages
B
A
For example
average 90 SD 40
average 110 SD 60
400 draws at random with replacement
100 draws at random with replacement
A average minus B average the average
difference between A and B 110 - 90 20
To test whether this difference of 20 is real or
due to chance we need 1. The expected value for
the difference between the 2 sample averages 2.
The SE for the difference between the 2 sample
averages
3
Chapter 27 More tests for averages
B
A
For example
average 90 SD 40
average 110 SD 60
400 draws at random with replacement
100 draws at random with replacement
Step 1 compute the expected value and SE for
each box separately A expected value for the
average of 400 draws 110 SE /- 3 B
expected value for the average of 100 draws
90 SE /- 4 expected value for the
difference between the 2 sample averages 110 -
90 20
4
Chapter 27 More tests for averages
B
A - B
A
average 20 SE ?
average 90 SD 40 SE 4
average 110 SD 60 SE 3
Step 2 how to combine the 2 SEs together (110
/- 3) minus (90 /- 4) equals (20 /-
???) Option 1add the 2 SEs 3 4
7 ignoring the possibility of 2 chance errors
canceling each other Option 2 use the square
root law to find the correct SE the SE for the
difference between 2 independent quantities is
a the SE for the first quantity b the
SE for the second quantity
5
Chapter 27 More tests for averages
B
A - B
A
average 20 SE ?
average 90 SD 40 SE 4
average 110 SD 60 SE 3
In our example the draws from the 2 boxes were
done independently so the 2 averages are
independent of one another and the square root
law applies Thus the SE of the average difference
here
6
Chapter 27 More tests for averages
B
A
Example
1 2
3 4
100 draws at random with replacement
100 draws at random with replacement
Find the expected value and the SE for the
difference between the of 1s drawn from box A
and the of 4s drawn from box B Step
1 expected values and SEs for each box
separately
B
A
average 50 SD .50 SE 5
average 50 SD .50 SE 5
7
Chapter 27 More tests for averages
B
A
Example
1 2
3 4
100 draws at random with replacement
100 draws at random with replacement
Find the expected value and the SE for the
difference between the of 1s drawn from box A
and the of 4s drawn from box B Step
2 expected values and SE for the difference
between the 2 samples
B
A-B
A
average 50 SD .50 SE 5
average 0 SE
average 50 SD .50 SE 5
8
Chapter 27 More tests for averages
Example
100 draws at random with replacement
1 2 3 4 5
Expected value for the of 1s drawn 100(.20)
20 SD for the box .4 SE for
the sum of the draws 4 In addition, in the same
100 draws the expected value for the of
5s drawn 100(.20) 20 SD
for the box .4 SE for the sum of the draws
4 So the SE for the expected difference between
the of 1s and the of 5s
No - we do not have 2 independent draws here if
one is large, the other will tend to be small
Exercise Set A 1, 2, 3, 4
9
Chapter 27 More tests for averages
2. Comparing 2 sample averages Example
examining school performance in which tests on
several subjects is given to a nationwide sample
of 17 year old students math test score average
in 1978 300.4 math test score average in
1992 306.7 difference
6.3 Is this due to chance or have test scores
really improved? We will use a
2 sample z-test to answer this question
Box Model suppose that in each of the
years the tests were given to a nationwide random
sample of 17 year old students model is as
follows
1992
1978
all 17yr olds test scores
all 17yr olds test scores
1,000 draws
1,000 draws
10
Chapter 27 More tests for averages
Box Model null hypothesis on average
the differences between the sample averages is
expected to be 0 the observed difference is
simply due to chance variability
1992
1978
all 17yr olds test scores
all 17yr olds test scores
Sample average 306.7
Sample average 300.4
alternative hypothesis the 1992 average is
larger than the 1978 average the math scores
did really improve which is why the two sample
averages are different
The 2 sample z-test will help us choose between
the 2 hypotheses
11
Chapter 27 More tests for averages
The 2 sample z-test will help us choose between
the 2 hypotheses
Step 1 the numerator observed difference
306.7 - 300.4 6.3 expected difference
defined by the null hypothesis 0 (the
difference is simply due to chance and does not
reflect any real improvement in math scores)
Step 2 denominator SE for the
difference between the sample averages 1978
sample SD 34.9 SE for the sum of
draws 1103.6 SE for the average
1.1 1992 sample SD 30.1 SE
for the sum of draws 952 SE for the average
about 1.0 The draws are independent
so the square root law applies
12
Chapter 27 More tests for averages
The 2 sample z-test will help us choose between
the 2 hypotheses
4.2
Thus the observed difference between the 1992 and
1978 math test score averages was about 4.2 SEs
above the value expected under the null
hypothesis the observed P value is extremely
small providing very strong evidence that we can
reject the null hypothesis This leaves us with
the alternative hypothesis that the observed
increase reflects real differences (improvements)
in the math test scores Note we have not
proved the alternative hypothesis we have not
even tested the alternative hypothesis all we
have done is show that we can reject the null
hypothesis
13
Chapter 27 More tests for averages
The 2 sample z-test
With P
Assumes 2 independent random samples And is
computed from the sample sizes the sample
averages the sample SDs
14
Chapter 27 Another example
A large college takes 1. a
simple random sample of 200 males of which 107
used a PC (53.5) 2. a simple random sample of
300 females of which 132 used a PC (44) Is this
difference between the s real or due to
chance?
females
males
1 use PC 0 not
1 use PC 0 not
300 draws
200 draws
null hypothesis the of 1s in the two boxes
are the same alternative hypothesis the of 1s
in the male box gt female box
15
Chapter 27 More tests for averages
Step 1 the numerator observed difference
53.5 - 44 9.5 expected difference
defined by the null hypothesis 0 (the
difference is simply due to chance and does not
reflect the fact that a greater proportion of
males use PCs)
Step 2 denominator SE for the
difference between the sample s SE for of
males using PCs SE for
the (7/200)x100 3.5 SE for of females
using PCs SE for the
(8.6/300)x100 2.9 The draws are independent so
the square root law applies
16
Chapter 27 More tests for averages
The 2 sample z-test will help us choose between
the 2 hypotheses
Thus the observed difference between the of
males and females using PCs was about 2.1 SEs
above the value expected under the null
hypothesis the observed P value is about 2
which is smaller than the convention of 5 so we
can reject the null hypothesis This leaves us
with the alternative hypothesis that the observed
difference reflects real differences in the
proportions of males and females using
PCs we have not proved the
alternative hypothesis we have not even tested
the alternative hypothesis all we have done is
show that we can reject the null hypothesis
17
Chapter 27 Experiments
What about experiments where people are randomly
assigned to a treatment and control group the
groups only differ in terms of the
tx Exampleevaluating the effects of vitamin C on
colds sample of 200 half are randomly
assigned to receive 2,000mg of Vc remaining
half receive 2,000mg of a placebo over the
period of the study treatment group
averaged 2.3 colds SD 3.1 control group
averaged 2.6 colds, SD 2.9 Is the difference
in averages significant or simply due to
chance? Does taking 2,000mg of vitamin C result
in fewer colds? Null hypothesis difference in
the average of colds is due to chance (the
vitamin C does not help reduce the of
colds) Alternative hypothesis the reduced of
colds reflects a real difference between the 2
groups which can only be attributed to vitamin C
18
Chapter 27 Experiments
sample of 200 half are randomly assigned
to treatment group over the period of the
study treatment group averaged 2.3 colds
SD 3.1 control group averaged 2.6 colds, SD
2.9 Is the difference in averages significant
or simply due to chance? Does taking
2,000mg of vitamin C result in fewer
colds? Difference between the 2 averages
-.3 treatment group SE for the sum
Se for average 31/100
0.31 control group SE for the sum
SE for the average 0.29
19
Chapter 27 Experiments
The 2 sample z-test will help us choose between
the 2 hypotheses
Thus the observed difference between the of
colds experienced by the tx and control groups
was about -0.7 of a SE below the value expected
under the null hypothesis the observed P value
is about 52 which is larger than the convention
of 5 so we FAIL to reject the null
hypothesis Note again we have not
proved the alternative hypothesis we have
not even tested the alternative hypothesis
all we have done is show that we can reject the
null hypothesis
20
Chapter 27 Experiments
Notein this example we have assumed that the two
groups were sampled independently and at random
with replacement from two boxes
control
Tx group
100 draws
100 draws
In reality the experiment was not done like this
we had a total of 200 people of whom 100 were
chosen at random WITHOUT replacement to receive
the treatment the remaining 100 received the
placebo the draws are made without replacement
and are dependent Why does the SE still work out
correctly if we have violated the underlying
assumptions? The reasoning here depends on the
box model as follows
21
Chapter 27 Experiments
Box model 200 people randomly chosen some get
the treatment (A), others get the placebo
(B) Each person has a ticket with 2 s one
shows the response to the tx A, the 2nd the
response to tx B. Only 1 of the 2 s can be
observed
Some tickets are drawn at random without
replacement and the response to the treatment (A)
is observed
More tickets are drawn at random without
replacement and the response to the placebo (B)
is observed (those left over after tx group is
selected)
tx group
control group
A B
A B
A B
A B
A B
A B
100 draws
100 draws
B
B
B
A
A
A
22
Chapter 27 Experiments
Box model 200 people randomly chosen some get
the treatment (A), others get the placebo
(B) Each person has a ticket with 2 s one
shows the response to the tx A, the 2nd the
response to tx B. Only 1 of the 2 s can be
observed
Null hypothesis the response is the same for
both txs A and B tested by average
response in A - average response in B Recall SE
for the average difference requires replacement,
independence
tx group
control group
A B
A B
A B
A B
A B
A B
100 draws
100 draws
B
B
B
A
A
A
23
When the of draws is small relative to the of
tickets in the box, neither of these errors is
serious both violations have little
influence. If the of draws are large relative
to the box (as is usual), the impact of each
violation can be serious if half the subjects
are assigned to each group, the correction factor
is less than 1
We are not applying the correction factor - if we
did we would tend to produce smaller SEs for A
0.71(.31) .22 for B .71(.29) .21 We do
not apply this correction factor and this
compensates for the dependence and this method of
computing the SE for the difference between 2
averages (when applied to randomized experiments)
tends to be conservative (in other words,
overestimating the SE by some amount) Even though
we have dependence and no replacement, this
method works Exercise Set C 1, 2, 5
24
Chapter 27 More on Experiments
The method for testing average differences
applies to qualitative data recall that we
reclassify the boxes and count occurrences Study
evaluating the effect of how information about 2
cancer therapies are presented and the type of
therapy recommended 167 doctors
randomly assigned to death rate group
info on the death rates for each therapy
(80) survival rate info on survival rates for
each therapy (87) both contained same info 10
of 100 die (so 90 of 100 survive) Results death
rate group 50 favored surgery, 50
radiation survival rate group 87 favored
surgery, 14 radiation Some argue for a rational
decision making process based on the facts. Here
the facts in both groups were the same - only the
manner of their presentation differed. One might
propose that any difference in the selection s
is simply due to chance as too many Dr.s favoring
surgery were selected by chance for the survival
rates group
25
Chapter 27 More on Experiments
167 doctors randomly assigned to death
rate group info on the death rates for each
therapy (80) survival rate info on survival
rates for each therapy (87) both contained same
info 10 of 100 die (so 90 of 100
survive) Results death rate group 50 favored
surgery, 50 radiation survival rate group 84
favored surgery, 16 radiation Requires a test of
significance which needs a box model
Death rate group
Survival rate group
D S
D S
D S
Remaining 87
S
D S
D S
D S
80
D
1 0
1 0
1 0
1 0
1 0
1 0
Surgery 1 Radiation 0
26
Chapter 27 More on Experiments
Results death rate group 50 favored surgery,
50 radiation survival rate group 84 favored
surgery, 16 radiation death rate 50 for surgery
of 1s in this batch of draws survival rate
groups 84 for surgery of 1s in the
remaining batch
Death rate group
Survival rate group
D S
D S
D S
Remaining 87
S
D S
D S
D S
80
D
1 0
1 0
1 0
1 0
1 0
1 0
Surgery 1 Radiation 0
27
Chapter 27 More on Experiments
Results death rate group 50 favored surgery,
50 radiation survival rate group 84 favored
surgery, 16 radiation death rate 50
for surgery of 1s in this batch of
draws survival rate groups 84 for surgery
of 1s in the remaining batch Death rate SE for
favoring surgery Death rate SE for the
favoring surgery (4.5/80)x100 5.6 Survival
rate SE for favoring surgery Survival rate SE
for the favoring surgery (3.42/87)x100 3.93
28
Chapter 27 Experiments
When does the z-test apply The square root law
for the SE of a difference between 2 averages
applies for 2 independent and simple random
samples and for randomized controlled
experiments The formulas do not apply when we
have 2 correlated responses for each subject
i.e. we have dependent samples
29
Hypothesis Tests
Thus far we have done 1. Single sample z-tests
and single sample t-tests
2. Two sample z-tests t-tests
30
Hypothesis Testing
an inferential procedure that uses sample data to
evaluate the credibility of a hypothesis about a
population Logic 1. State the hypothesis about
the population 2. Obtain a random sample from
the population 3. Compare the sample data with
the hypothesis Example administer a treatment to
a population with a known mean what is its
effect? What happens to the mean? Assumption
about the effect of the treatment -
simply adds/subtracts a constant to each
individuals score - does NOT
change the shape of the distribution or standard
deviation
treatment
31
Hypothesis Testing
Next step set the criteria for the decision
determine the critical values based upon the
level of significance selected level of
significance the probability value that defines
the very unlikely sample values when the null
hypothesis is true called the alpha level by
convention .05
Reject HO
Reject HO
Extreme values (probability ltalpha) if the HO
is true these are possible but very unlikely
outcomes the critical region
32
Hypothesis Testing
Problem in hypothesis testing deciding whether
or not the sample data are consistant with the
null hypothesis if observed P gt alpha, we have
no evidence vs null hypothesis if observed P lt
alpha, we have evidence vs the null hypothesis In
either case, we could be wrong!!!!!!
REALITY No effect Effect Exists (HO
True) (HO True) Reject HO
Type I error Correct DECISION
Fail to reject HO Correct Type II error