Chapter 6 Statistical Significance

1
Chapter 6Statistical Significance

• By Vanessa Cortes
• Darlene Villafranca

2
Significant means reliable not important

• In normal English, "significant" means important,
  while in Statistics "significant" means probably
  true (not due to chance).
• A research finding may be true without being
  important.
• When statisticians say a result is "highly
  significant" they mean it is very probably true.
• They do not (necessarily) mean it is highly
  important.

3
Statistics vs. Parameters

• Characterized aspects of data that constitute a
  sample Statistics
• Characterized features of data that constitute a
  populationParameters
• Researchers want to know if the sampling
  descriptive statistics accurately approximates
  population parameters

4
Three types of Significance

• Statistical Significance
• Practical Significance
• Clinical Significance
• All require calculations
• All yield numerical results

5
Statistical Significance Inferential Statistics

• Inferential Statistics are often the next step
  after you have collected summarized data
• Inferential statistics are used to make
  inferences from smaller group of data to possibly
  larger one
• Inferential statistics are used to infer
  something about the population based on the
  samples characteristics

6
Null Hypothesis

• Null is always a statement of equality
• Null can be true or false
• No difference or truly is an inequality
• Null applies ONLY to the population
• The researcher never really knows the Null
  hypothesis and if there is or is not a difference
  between groups
• WHY?

7
Hypothesis Testing

• You cannot make both Type I and Type II Error on
  a given hypothesis
• Once you reject, you could not have made a Type
  II error.
• Once you fail to reject, you could not have made
  Type I error.
• Hypotheses predicting zero differences or zero
  relationship

8
Accept or Reject the Null Hypothesis
9
Statistical Power

• The power of a statistical test is the
  probability that the test will reject a false
  null hypothesis (that it will not make a Type II
  error). As power increases, the chances of a Type
  II error decrease, and vice versa. The
  probability of a Type II error is referred to as
  ß. Therefore power is equal to 1 - ß.

10
One-Tailed and Two-Tailed Significance Tests

• One important concept in significance testing is
  whether you use a one-tailed or two-tailed test
  of significance. The answer is that it depends on
  your hypothesis. When your research hypothesis
  states the direction of the difference or
  relationship, then you use a one-tailed
  probability. For example, a one-tailed test would
  be used to test these null hypotheses Females
  will not score significantly higher than males on
  an IQ test. Blue collar workers are will not buy
  significantly more product than white collar
  workers. Superman is not significantly stronger
  than the average person. In each case, the null
  hypothesis (indirectly) predicts the direction of
  the difference.

11
Two-Tailed Significance Tests

• A two-tailed test would be used to test these
  null hypotheses There will be no significant
  difference in IQ scores between males and
  females. There will be no significant difference
  in the amount of product purchased between blue
  collar and white collar workers. There is no
  significant difference in strength between
  Superman and the average person. The one-tailed
  probability is exactly half the value of the
  two-tailed probability.

12
Properties of Sampling Distributions

• For all sampling distributions, the standard
  deviation of the sampling distribution gets
  smaller as sample size increases.
• The mean of the statistics in a sampling
  distribution for unbiased estimators will equal
  the population parameter being estimated

13
Standard Error/Sampling Error

• Standard deviation of the sampling
  distributionStandard error of the statistic or
  Standard error. (SE)

14
Inferentially

• Statistical significance test is to divide the
  statistic by its standard error.
• 3 synonymous names for the fundamental ratio
• 1. t statistic
• 2. critical ratio
• 3. Wald statistic

15
Test Statistics

• Test statisticsstandardized sampling
  distributions
• Test statistics (TS) distributions can be used in
  place of P values .
• The comparison of a given P calculated with a
  given PCRITICAL will always yield the same
  decision about the comparison of a given TS
  calculated with a given TSCRITICAL.

16
P calculated

• P calculated is calculating the estimated
  probability
• P values range from 0 to 1
• or 0 to 100

17
Procedure Used to Test for Significance

• Whenever we perform a significance test, it
  involves comparing a test value that we have
  calculated to some critical value for the
  statistic. It doesn't matter what type of
  statistic we are calculating (e.g., a
  t-statistic, a chi-square statistic, an
  F-statistic, etc.), the procedure to test for
  significance is the same.
• 1. Decide on the critical alpha level you will
  use (i.e., the error rate you are willing to
  accept).
• 2. Conduct the research.
• 3. Calculate the statistic.
• 4. Compare the statistic to a critical value
  obtained from a table.

18
If your statistic is higher than the critical
value from the table

• Your finding is significant.
• You reject the null hypothesis.
• The probability is small that the difference or
  relationship happened by chance, and p is less
  than the critical alpha level (p lt alpha ).

19
If your statistic is lower than the critical
value from the table

• Your finding is not significant.
• You fail to reject the null hypothesis.
• The probability is high that the difference or
  relationship happened by chance, and p is greater
  than the critical alpha level (p gt alpha ).