Test Of Significance

1
Test Of Significance
2
Evidence Against A Claim

• Claim I make 80 of my basketball free throws
• To test this claim, I shoot 20 free throws
• The result I make 8 of the 20, only 40 success
• If the claim is true, I would be making on
  average 16 of the 20 free throws. So making 8 of
  20 may be evidence against my claim!
• How strong or significant is the evidence against
  the claim?

3

• Assume my claim of shooting 80 is true.
• If there is a good chance I make 8 of 20, it
  (making 8 of 20) is not strongly against my
  claim.
• If its almost unlikely that I make 8 of 20, then
  the fact I did make 8 of 20 says my claim may not
  be substantiated.
• In other words, under the assumption that my
  claim is valid, we want to know the probability
  of what actually happens.
• If P( making 8 of 20) is big enough, you accept
  my claim, if P( making 8 of 20) is too small, you
  reject my claim.
• When my claim is rejected, then you accept I
  cant make 80 free throws (alternative claim).

4
Hypothesis Testing Free Throws

• A claim about population (null hypothesis H0)
• What actually happens (evidence based on real
  data).
• An alternative claim (alternative hypothesis Ha)

• Question How significant is the evidence
  (provided by data) against the claim?
• If the evidence against the claim is strong, the
  claim will be rejected, the alternative claim
  will be accepted If the evidence against the
  claim is weak, the claim will be accepted.
• If H0 is assumed to be true and the probability
  of what actually happens is big, we accept H0 If
  H0 is assumed to be true but the probability of
  what actually happens is too small, we reject H0
  and instead accept Ha

5
Hypothesis Testing Free Throws

• H0 make 80 free throws
• Ha make less than 80 free throws
• Compute the probability that if H0 is true, I
  make 8 or less free throws of 20.
• If this probability is too small, say less than
  0.05, we reject H0, accept Ha if this
  probability is bigger than 0.05, we accept H0.

The probability of what happens or more extreme
against the null hypothesis, in favor of the
alternative hypothesis if H0 is true is called
P-value. P-value is compared to a number, called
level of significance, . If P-value is as
small or smaller than , we say the data are
statistically significant at level (against
the claim) ------- Test of significance against a
claim.
6
Hypothesis Testing Free Throws
Lets say the P-value is 0.0001 The chance of
hitting 8 or less of 20 is 0.0001 if my claim I
hit 80 free throws is true.
Interpretation The chance of hitting 8 of 20 is
so small, smaller than the preset significance
level that it should rarely happen. But it
(hitting 8 of 20) happened, the data provides a
strong evidence against the claim. So H0 will
be rejected. Accept Ha.
Interpretation The chance of hitting 8 of 20
is small. But the chance of its happening is
bigger than the preset significance level. It
happens more probable than we required. So the
fact of its happening (8 of 20) doesnt provide a
strong evidence against the claim. H0 will be
accepted.
7
Example Sweetening Colas

• Diet colas use artificial sweeteners to avoid
  sugar. These sweeteners sometimes lose sweetness
  over time. So manufacturers test new colas for
  loss of sweetness before marketing them
• Each trained taster score the cola on a
  sweetness score of 1 to 10 before and after a
  lengthy storage (this is matched pairs
  experiment subjects?). Then they measure the
  differences of sweetness (before-after)
• Suppose the sweetness loss scores vary from
  taster to taster according to a normal
  distribution with s 1
• Here are the loss of sweetness scores by 10
  tasters

8
Example Sweetening ColasLoss of Sweetness Over
Time?

• . Let the preset significance
  level a0.001
• State Hypotheses
• (Use m to represent the mean loss of
  sweetness)
• H0 There is no loss of sweetness, i.e., m
  0
• Ha There is a loss of sweetness, i.e, m gt
  0
• Find the P-value If H0 is true, whats the
  probability that the sample mean is bigger or
  equal to 1.02?

9
Hypothesis Testing Sweetening Colas

• Whats the distribution of ?

Whats the z-score?
P-value
10
Hypothesis Testing Sweetening Colas

• The probability (that the sample mean is bigger
  or equal to 1.02) is smaller than a 0.001. So
  the data given is statistically significant at
  level a 0.001. The fact that it happens less
  likely than preset level a 0.001 but it
  actually happened is evidence against the null
  hypothesis.
• Reject the null hypothesis. Accept the
  alternative hypothesis.
• Lets say a is not 0.001. Based on the data and
  following the test of significance procedure, we
  accept the null hypothesis. What can you say
  about a ?
• a should be at most .0006 i.e.

11
Three Notes
Note 1 The z-score computed from the sample mean

is called the z-statistic. Its
used in the test of significance, so its a test
statistic.
Note 2 Whether you reject (or accept) the null
hypothesis, it just means youve made a decision
based on the evidence (data). Did you make the
right decision? The decision you made is a right
decision only with probability (in many
decisions you make, the chance of right
decisions). Even the probability of getting a
right decision is high, it doesnt mean you
indeed made the right decision.
Note 3 Rejecting H0 in favor of Ha means making
an expensive changeover from one type of product
packaging to another, you need strong evidence
that the new packaging will boost sales.
12
Examples Stating Hypotheses And Computing
P-values

• Exercise 14.1, p 342 Exercise 14.3, p 344
  Exercise 14.11, p 348 Exercise 14.14, p 349
• Exercise 14.2, p 343 Exercise 14.4, p 344
  Exercise 14.12, p 348 Exercise 14.15, p 349
• Exercise 14.5
• Alternative hypothesis in above examples are
  one-sided.

13
Two-sided alternative

• Example The diameter of a spindle in a small
  motor is 5 mm. If the spindle is too small or too
  large, the motor wont work properly. The
  manufacturer measures the diameter in a sample of
  motors to determine whether the mean diameter is
  different from 5 mm.
• Let m be the mean diameter of all spindles.

In Ha, there are two possibilities m gt 5 mm and
m lt 5 mm
More Examples Example 14.3, p 344 Example
14.7, p 351 Example 14.10, p 354
14
Two-sided alternative P-value

• Example 14.7, p 351 The systolic blood pressure
  for males 3544 has mean 128 and standard
  deviation 15. But an exam of 72 executives in a
  large company reveals their average is 126.07.
  (Suppose the executives blood pressure follows a
  normal distribution with standard deviation 15.)
• Is this evidence the executives have a different
  mean blood pressure from the general population?

Lets consider the mean of an SRS of 72 men 3544
and see if there is significant difference
between this SRS and the 72 executives.
15

• m is the unknown mean of executives population

State Hypotheses
Compute the P-value the probability of what
happened or more extreme against the Null
hypothesis, more in favor of the alternative
hypothesis
Equally distant from 128 as 126.07
16
The z-score for 126.07 and 129.93 (the
z-statistics) are
So the P-value P(z lt -1.09) P(z gt 1.09) 2P(z
gt 1.09) 2(1-0.8621) 0.2758
Interpretation More than 27 of the time, an SRS
of size 72 from the general male population 3544
would have a mean blood pressure at least as far
from 128 as that of the executive sample.
Conclusion not good evidence that executives
differ from others.
If the alternative hypothesis is two-sided,
P-value has two parts.
17
Tests for a Population Meanz-procedure

• Hypotheses are claims about the population mean.
• Four steps in carrying out a significance test

• State the hypotheses
• Calculate the test statistic z
• Find the P-value
• State the interpretation and conclusion. If
  level of significance a is given, state whether
  the data is significant at level a against the
  null hypothesis.

18
More Examples
Exercise 14.19, page 353
Interpretation ? Conclusion ?
Exercise 14.18, page 353
Interpretation ? Conclusion ?
19
(No Transcript)
20
Exercise 14.21, page 356
What should the z-statistic of this point be so
that the P-value (the area to the right of this
point) is less than .005?
Inverse lookup in table A for 1-.005.995 yields
2.58. So the z-statistic should be bigger than
2.58
21
Exercise Same as in Exercise 14.21. But Ha now
is m lt 0.
What should the z-statistic of this point be so
that the P-value (the area to the left of this
point) is less than .005?
Inverse lookup in table A for 0.005 yields -2.58.
So the z-statistic should be smaller than -2.58
22
Exercise 14.20, page 356
What should the z-statistics of this point A and
this point B be so that the P-value (the area to
the right of A and to the left of B) is less than
.005?
Inverse lookup in table A for 1-.0025.9975
yields 2.81. So the z-statistic should be bigger
than 2.81 or smaller than -2.81, i.e. z gt 2.81
23
Confidence Interval and Hypothesis Testing

• Repeated measurements of the length of an object
  will yield different results. Assume the standard
  deviation of all the measurements is 6 mm. An SRS
  of 1000 measurements has an average 112.6 mm
• Whats the 95 confidence interval?
• Should we reject or accept H0 if a 0.05 and

24
95 confidence interval
If is between A and B, accept H0
otherwise reject H0
The z-score for A is the same as for m m
Similarly,
So the decision of whether to reject or accept H0
depends on if
Or in other words,
Answer No. Therefore reject H0
25
Consider the following level a test of
significance question
Answer Reject H0, if m0 lies outside the
confidence interval
Accept H0, if m0 lies in the confidence interval.
Exercises 14.24 and 14.25, page 358.
26
Type I and II Errors