More About Significance Tests

1
More About Significance Tests
Presentation 11
2
Inference Techniques

• Confidence Intervals
• 1. 1-Proportion
• 2. 1-Mean
• 3. Difference between 2-Proportions
• 4. Difference between 2-Means

• Hypothesis Tests
• 1-Proportion
• 1-Mean
• Difference between 2-Means
• Difference between 2-Proportions
• Difference between Paired Means.
• Chi-Square (Relationship between 2 Categorical
  Variables)
• Regression (Relationship between 2 Quantitative
  Variables)
• ANOVA (Difference between 3 or More Means)
• Median Tests

3
General Steps for Hypothesis Tests

• Step 1 Determine the H0 and Ha.
• The alternative hypothesis, Ha, is the claim
  regarding the population parameter that we want
  to test. There are three possible Ha's - the
  parameter is not equal to a null value
  (two-sided), less than a null value (one-sided)
  or greater than a null value (one-sided). The
  null hypothesis, H0, claims that nothing is
  happening. H0 can be the opposite of Ha or just
  that the parameter is equal to the null value.
• Step 2 Verify necessary data conditions, if met
  summarize the data into an appropriate test
  statistic.
• The conditions to be verified are the same
  conditions we have seen in Chapter 12 for
  creating C.I's.
• The test statistic is a standardized
  statistic, i.e. is of the form
• Sample statistic - Null value
• Null s.e (sample statistic)
• Under H0 the test statistic follows either a
  normal distribution (proportion cases) or a
  t-distribution with some d.f (mean cases).

4
General Steps for Hypothesis Tests

• Step 3 Find the p-value.
• The p-value is the probability that the test
  statistic is as large or larger in the
  direction(s) specified from Ha assuming that H0
  is true. This, to get the p-value you need the
  form of Ha, the value of the test statistic, and
  the distribution of the test statistic under the
  H0.
• Step 4 Decide whether or not the result is
  statistically significant.
• The results are statistically significant if
  the p-value is less than alpha, where alpha is
  the significance level (usually 0.05). Based on
  the p-value we have two possible conclusions
• If p-value lt a we reject the null and we claim
  the alternative is true.
• If p-value gt a we fail to reject the null, we
  claim that there isn't enough evidence to report
  Ha is true. In this case, we do not claim that
  the null is true!
• Step 5 Report the conclusion in the context of
  the situation.
• Finally, write one or two sentences
  explaining what is the conclusion in terms of the
  problem.

5
One sample t-test (one mean or paired data)

• In this case we consider the following
  hypotheses
• 
  Ha µ ? µ0
• H0 µ µ0 vs one of the following
  Ha µ gt µ0
• 
  Ha µ lt µ0
• We have the following t-test statistic
• Assuming that H0 is true, the test
  statistic has a t-distribution with (n-1) degrees
  of freedom, if one of the following conditions is
  true
• the random variable of interest is bell-shaped
  (in practice, for small samples the data should
  show no extreme skewness or outliers).
• the random variable is not bell-shaped, but a
  large random sample is measured, n 30.

6
One sample t-test (one mean or paired data)

• If we denote with T a random variable with
  t-distribution with (n-1) d.f, and t is the value
  of out test statistic, then

Ha p-value
µ lt µ0 P( T lt t )
µ gt µ0 P( T gt t )
µ ? µ0 2P( T gt t )

• We can find the p-value using the t-table (table
  A3), and draw a conclusion about the hypotheses
  in the usual manner.

7
Two-Sample t-test (difference between two means).

• There are two populations (Population 1 and 2) of
  interest having unknown means µ1 and µ2
  respectively. In this case we consider the
  following hypotheses
• 
  Ha µ1 - µ2 lt
  0
• H0 µ1 - µ2 0 (i.e. no difference) vs one
  of the following Ha µ1 - µ2 gt 0
• 
  Ha µ1 - µ2 ?
  0
• We have the following t-test statistic
• Assuming H0 is true, the test statistic has a
  t-distribution with approximate d.f. equal to the
  minimum of n1-1 and n2-1 if the following
  conditions are true
• The two samples are independent.
• Each sample is either coming from a bell shaped
  population or the sample size is 30.
• Using the appropriate d.f, we can get the p-value
  from table A.3 in the same way as in the one-mean
  case.

8
Difference between two proportions

• Now we will consider two populations having some
  unknown proportions of interest, p1 and p2
  respectively. In this case we consider the
  following hypotheses
• 
  Ha p1 - p2
  lt 0
• H0 p1 - p2 0 (i.e. no difference) vs one
  of the following Ha p1 - p2 gt 0
• 
  Ha p1 - p2 ?
  0
• The sample statistic we will use is .
  To get the null standard error of the statistic
  we assume that H0 is true, so let p1 p2 p.
  Thus, instead of using and in the s.e
  formula, we will substitute them with an estimate
  of the common population proportion. That is, we
  use the proportion in all available data
• and we have the null s.e.

9
Difference between two proportions

• Thus, we have the following t-test statistic
• Assuming H0 is true, the test statistic has
  a standard normal distribution if the following
  conditions are true
• The two samples are independent.
• All the quantities
  are at least 5 and preferably at
  least 10.
• We can obtain the p-value using the standard
  normal table A1. If we denote with Z a random
  variable with standard normal distribution, and z
  is the value of out test statistic, then

Ha p-value
p1 - p2 lt 0 P( Z lt z )
p1 - p2 gt 0 P( Z gt z )
p1 - p2 ? 0 2P( Z gt z )
10
Table For Hypothesis Testing
Type Parameter Statistic Null Std. Error Test Statistic
One Mean (or Paired mean) µ or µd or or t df n-1
Difference Between Means µ1- µ2 t dfmin(n1-1,n2-1)
One Proportion p z
Difference Between Proportions p1-p2 z

.


This is the overall proportion-like if we had one
big sample!
11
Detailed Example 1

• A box of corn flakes is advertised as containing
  16 oz. of cereal. John works for consumer
  reports and is interesting in verifying the
  companies claim. He suspects that the average
  amount of cereal is less than 16oz. He takes a
  random sample of 100 boxes of cereal and records
  the weight of each box. The sample mean is
  15.7oz and the sample standard deviation is
  1.2oz.

12
Example 1 Overview

• Before you carry out the hypothesis test it is a
  good idea to label the information you have.
• Since there is 1 quantitative variable we are
  doing a hypothesis test of 1-mean.
• The parameter of interest is µ, the mean weight
  of cereal in all corn flakes boxes.
• The statistic of interest is 15.7oz, the
  sample mean weight.
• Other information The sample size n 100, and
  the sample standard deviation s 1.2oz

13
Example 1 Hypotheses and Conditions

• 1. Define the null and alternative hypotheses
• Ho µ 16oz
• Ha µ lt 16oz
• 2. Check conditions Either A or B
• A) The distribution of weights is normal.
• B) The sample size is 30.

14
Example 1 Test Statistic and P-Value

• 3. Calculate the test statistic.
• Null Value 16, Sample Statistic 15.7
• Null Std. Error 1.2/10 .12
• Test Statistic
• P-Value P(Tlt-2.5) Use Table A.3!
• degrees of freedom df n-1 100-1 99

15
Example 1 P-value Conclusion
Table A.3 gives the one-sided p-values for
t-tests. For 2-sided hypothesis tests make sure
you multiply the p-value by 2! Look up the
absolute value of the test statistic and df in
the table. If they do not have the exact test
statistic, then take your best guess OR use the
next smallest test statistic. Using the next
larger and next smallest value for t-statistic on
the table you can get an interval for the p-value.
In our case, df 99 and t-statistic -2.5.
Since 2.5 is not in the table we can use 2.33,
and 90 df (since 99 is not in the table). We get
that the p-value is less than .011. The p-value
is lt .05 so we REJECT the null hypothesis and we
conclude that the average weight IS less than
16oz.
16
Detailed Example 2

• A potential presidential candidate in 2002
  election wished to know if men and women had
  equal preferences for her candidacy versus this
  not being the case (she suspected that men were
  more likely to favor her). Her pollster polled a
  random sample of 400 men and 400 women, with the
  following results.
• Preference Would vote for her?
• No Yes Total
• Men 164 236 400
• Women 180 220 400
• Total 344 556 800

17
Example 2 Overview

• Here there are 2 categorical variables, both
  with 2 levels so we are interested in a test of
  2-proportions!
• Is the proportion of men who favor the candidate
  greater than the proportion of women who favor
  the candidate?
• The parameter of interest is pm-pw.
• The sample statistic of interest is
  .59-.55 .04

18
Example 2

• 1. Calculate the null and alternative hypotheses
• Ho pm pf or pm-pf 0
• Ha pm gt pf or pm-pf gt 0
• Lets denote with pm with p1 and pf with p2.
• 2. Check the conditions
• All quantities, and
  are greater than or equal to 10.
  

19
Example 2 Test Statistic and P-Value

• 3. Calculate the test statistic

20
Example 2 P-Value and Conclusion

• P-Value P(Zgt1.14) 1-P(Zlt1.14)
• 1-0.8729 0.127
• P-Value is gt.05 so we can NOT reject the null
  hypothesis. Therefore, we do NOT have enough
  evidence to conclude that men are more likely to
  favor the candidate.

21
MINITAB Output for Example 2

• Test and CI for Two Proportions
• Sample X N Sample p
• Men 236 400 0.590000
• Women 220 400 0.550000
• Estimate for p(1) - p(2) 0.04
• Test for p(1) - p(2) 0 (vs gt 0) Z 1.14
• P-Value 0.126

22
CIs and Two-sided Alternatives

• When testing the hypotheses
• H0 parameter null value vs.
  
• Ha parameter ? null value
• If the null value is covered by a (1 - ?) CI, the
  null hypothesis is not rejected and the test is
  not statistically significant at level ?.
• If the null value is not covered by a (1 - ?) CI,
  the null hypothesis is rejected and the test is
  statistically significant at level ?.
• For instance, for a 95 Confidence Interval, (1-
  ?) 0.95 95. So for 95 confidence, the
  significance level is ? 0.05, which is the
  significance level and confidence interval used
  most frequently.

23
Problem 13.38 in the Text

• Each of the following presents a 95 CI and the
  alternative hypothesis of a corresponding
  hypothesis test. In each case, state a
  conclusion for the test, including the level of
  significance you are using.
• A) CI for µ is (101 to 105), Ha µ ?1 00
• B) CI for p is (.12 to .28), Ha plt.10
• C) CI for µ1-µ2 is (3 to 15), Ha µ1-µ2 gt 0
• D) CI for p1-p2 is (-.15 to .07), Ha p1-p2 ? 0

24
Possible Errors, Power and Sample Size
Considerations

• When we are testing a hypothesis two out four
  possible decisions lead to an error.
• Type I error - We reject H0 when it is true
• Type II error- We fail to reject H0 when Ha is
  true.
• There is an inverse relationship between the
  probabilities of the two types of errors.
  Increase in the probability of type I error leads
  to decrease in the probability of type II error
  and vice versa.
• When the alternative hypothesis is true, the
  probability of making the correct decision is
  called power of the test.
• The power increases when the sample size
  increased. Can you see why?
• The power increases when the difference between
  the true population parameter value and the null
  value increases. Can you see why?
• The hypothesis test may have very low power
  because of small data set.
• We should also be careful in cases our
  conclusions are based on extremely large samples,
  since in cases like that even a weak relationship
  (or a small difference) can be statistically
  significant.