Hypothesis tests with related samples

1
Hypothesis tests with related samples

• Overview
• Previously we have discussed situation when data
  come from two separate samples called an
  independent-measures research design or a
  between-subjects design
• Now we will discuss a situation in which the
  comparison is made between two sets of data
  collected from the same sample called a repeated
  measures study

2
Hypothesis tests with related samples

• Overview
• Same participants used in all conditions
  generally requires fewer participants because the
  same participants are in both conditions also
  variability is reduced for this reason, and,
  hence, power is increased
• Matched-subjects study each individual in one
  sample is matched with a subject in the other
  sample e.g., age, memory performance score
• Note each subject in one treatment condition
  must be matched with a subject in a second
  treatment condition

3
Hypothesis tests with related samples

• Overview
• Related sample statistics are used in a repeated
  measures design and in a matched-subjects study

4
Hypothesis tests with related samples

• t statistic for related samples
• t statistic for related samples is the same as a
  single sample t statistic except that it is based
  on a difference score between performance in
  condition 1 and condition 2 for each participant

5
Hypothesis tests with related samples

• t statistic for related samples
• Example
• Does relaxation training reduce the number of
  doses of medicine asthma patients require for
  asthma attacks?
• Design within subjects design measure number
  of doses before and after training

6
Hypothesis tests with related samples
7
Hypothesis tests with related samples

• t statistic for difference scores has the same
  structure as the single sample t statistic.
• t (Dbar - ?D)/sDbar
• The t statistic for single sample t (M-?)/
  sM
• where sM v s2/n
• Dbar mean of Difference scores
• sDbar estimated standard error of the mean
  difference score
• sDbar vs2/n

8
Hypothesis tests with related samples
9
Hypothesis tests with related samples

• State hypothesis
• H0 ?D 0
• H1 ?D ? 0
• Set critical region a .05
• Locate critical region
• df n-1 4
• Critical region 2.776

10
Hypothesis tests with related samples

• Calculate t statistic
• t (Dbar - ?D)/sDbar
• The t statistic for single sample t (M-?)/
  sM
• where sM v s2/n
• Dbar mean of Difference scores SD/n -17/5
  -3.4
• sDbar estimated standard error of the mean
  difference score
• sDbar v s2/n, where s2 SS / n-1

11
Hypothesis tests with related samples
12
Hypothesis tests with related samples

• Calculate t statistic
• t (Dbar - ?D)/sDbar
• The t statistic for single sample t (M-?)/
  sM
• where sM v s2/n
• Dbar mean of Difference scores SD/n -17/5
  -3.4
• sDbar v s2/n, where s2 SS / n-1
• SS S (D Dbar) 2 9.2
• s2 SS / n-1 9.2/4 2.3
• sDbar v s2/n v2.3/5 v.46 .68

13
Hypothesis tests with related samples

• Calculate t statistic
• t (Dbar - ?D)/sDbar
• t (-3.4 0)/.68
• -5.0
• Make decision
• Observed value of t falls in critical region.
  Conclude that treatment had a significant effect.
• Report finding APA style
• Treatment reduced the number of doses required to
  control symptoms of asthma ( M 3.4, SD 1.52
  ). This reduction was significant t(4) 5.0, p lt
  .05, two tailed.
• Recall SD vSS/(n-1) v9.2/4 1.52

14
Hypothesis tests with related samples

• Checking the plausibility of the result two
  approaches
• 1. look at data subject by subject
• Each subject shows the same pattern of results
  fewer doses are needed after treatment than
  before
• 2. compare Dbar -3.4 and SD 1.52 to null
  hypothesis of 0. note that most of the data must
  be lt 0
• In each case these observations suggest that the
  reduction will be statistically significant

15
Hypothesis tests with related samples

• Advantages of repeated measures design
• 1. ensures that individual differences between
  different samples are not responsible for the
  observed differences
• 2. reduces variance, hence increases power of
  test

16
Hypothesis tests with related samples

• Disadvantages of repeated measures design
• This design can create problems if there are
  carryover effects. A carryover effect occurs when
  the effect of treatment one is observed in the
  second treatment that follows
• E.g., in drug treatment studies
• E.g., in learning/memory studies

17
Hypothesis tests with related samples

• Disadvantages of repeated measures design
• This design can create problems if there is
  progressive error or fatigue effects
• Participants get fatigued, lose motivation, etc.
  over time

18
Hypothesis tests with related samples

• Dealing with carryover/progressive error effects
• 1. counterbalance order of presentation. That
  means, randomly assign half participants to order
  T1 then T2 assign other participants to order T2
  then T1
• 2. if these effects are large, use a between
  subjects experimental design

19
Estimation

• Overview of estimation
• Purpose to estimate population parameters such
  as the population mean using a sample of data.
  For example, you may want to describe a typical
  Canadian family on a number of attributes.
• Approach take a sample, determine its mean, and
  then use that estimate and other information from
  the sample to determine the population mean

20
Estimation

• Overview
• process is based on sampling, hence there is
  sampling error associated with it this means
  that the sample estimate will differ from the
  population estimate, and that any given sample is
  part of a sampling distribution
• Two different types of estimates point estimate
  and interval estimate

21
Estimation

• Overview
• point estimate single number used to estimate an
  unknown parameter
• interval estimate range of values used to
  estimate an unknown quantity interval estimate
  accompanied by a level of confidence, it is
  called a confidence interval

22
Estimation

• Overview
• Estimation and hypothesis testing are similar in
  that they both use similar data to address
  related questions
• Difference is in the question addressed.
  Hypothesis testing determines whether or not
  treatment has an effect. Estimation determines
  magnitude of the effect

23
Estimation

• Overview
• When to use estimation
• Use estimation procedures when you want to
  determine the size of an effect
• E.g., when null hypothesis is rejected, when you
  are interested in knowing the size of an effect
  because you want to be able to describe an
  unknown population, when you want to gain a sense
  of the practical or clinical significance of a
  treatment

24
Estimation

• Overview
• Logic of estimation
• Both hypothesis testing and estimation are
  inferential statistics
• Both use samples to draw conclusions about
  populations
• Both attempt to take into consideration the
  discrepancy or error between a statistic and a
  population parameter

25
Estimation

• Point estimation with a z score
• Recall a z score is used when the population mean
  is unknown but the population SD is known

26
Estimation

• Point estimation with a z score
• Recall whenever, a sample of observations are
  obtained, one can think of the sample as
  belonging to a sampling distribution
• In the case of a z score distribution, the mean
  is 0 and the SD 1
• For a given sample with M X, the most probable
  outcome is that M ?. Why

27
Estimation

• Point estimation with a z score
• Recall Z (M - µ)/sM
• When z 0, this is the highest probability
  outcome. Z 0 when M µ
• Example. Evaluate a program designed to improve
  reading. A test is used to determine whether
  reading improved. The test designed to evaluate
  reading performance of the (n 25) students in
  the reading program has ? 80, ? 10.
  Performance on the test after being in the
  training program was M 88.
• Goal. Estimate ? after being in the reading
  program.

28
Estimation

• Point estimation with a z score
• Z (M - µ)/sM 0 (M - µ)/sM Thus, M µ

29
Estimation

• Interval estimation with a z score
• Want to determine a range of values as an
  estimate of an unknown quantity
• E.g., the high temperature today is 20-22 degrees
  centigrade
• E.g., it will take me 45 60 minutes to drive to
  Mississauga

30
Estimation

• Confidence interval with a z score
• In a confidence interval, a range of scores is
  presented with a confidence interval
• The length of time it takes to drive to
  Mississauga is 45 60 minutes with a 95
  confidence interval
• This means that mean time it takes to drive to
  Mississauga will be in this range of scores 95
  of the time

31
Estimation

• Confidence interval with a z score
• Determining a confidence interval from a sample
  of scores in which the ? is unknown but ? is
  known
• This sample is part of a z sampling distribution
• Recall Z (M - µ)/sM
• if we want to construct a 95 confidence
  interval, we can determine the z score associated
  with the top and bottom 2.5. That value is z
  1.96

32
Estimation

• Confidence interval with a z score
• Z (M - µ)/sM
• Now need to determine upper and lower values of µ
• First step, isolate µ
• ZsM (M - µ)
• µ M - zsM

33
Estimation

• Confidence interval with a z score
• The test designed to evaluate reading performance
  of the (n 25) students in the reading program
  has ? 80, ? 10. Performance on the test
  after being in the training program was M 88.
• sM ?/vn 10/5 2
• One extreme z 1.96
• µ M - zsM
• µ 88 1.96 (2) 84.08

34
Estimation

• Confidence interval with a z score
• other extreme z -1.96
• µ M - zsM
• µ 88 1.96 (2) 91.92
• Hence we conclude that mean performance after the
  reading program falls between 84.08 and 91.92
  with 95 confidence

35
Estimation

• Confidence interval with a z score (Summary)
• In order to calculate a 95 confidence interval
  in a situation in which the SD is known, take the
  sample mean, then add or subtract the sampling
  error
• µ M zsM
• Population mean sample mean error

36
Estimation

• Confidence interval with t statistics
• Same process as with z statistic
• Pop. Mean sample mean t (standard error)
• Mean diff mean diff t (standard error)
• Estimation procedure. Use sample data to compute
  sample mean (or mean difference). Use that
  estimate to compute the population mean. Why most
  likely value for t 0. Then as before determine
  for a given confidence level the t scores, and
  use the formulae shown in the next slide to
  calculate the range

37
Estimation

• Confidence interval with a z score (Summary)
• In order to calculate a 95 confidence interval
  in a situation in which the SD is known, take the
  sample mean, then add or subtract the sampling
  error
• µ M zsM
• Population mean sample mean error

38
Estimation
39
Estimation

• Confidence interval with single sample scores in
  which pop. SD is unknown
• e.g., Determine mean age of people who purchase
  jeans. Sample obtained of 30 jean purchasers.
  Mean age of sample M 30.5 SS 709.
  Calculate point estimate and 95 confidence
  interval

40
Estimation

• Confidence interval with single sample scores in
  which pop. SD is unknown
• ? M tsM
• M 30.5
• sM vs2 /n, where s2 SS/(n-1)
• S2 709/29 24.45
• sM vs2 /n v24.45/30 .90

41
Estimation

• Confidence interval with single sample scores in
  which pop. SD is unknown
• ? M tsM
• Point estimate of ? is M or 30.5. Why? Hint, what
  is the mean of t? see page 284
• 95 confidence interval for t (29) is 2.045
• ? M tsM 30.5 2.045 .90 30.5 1.84
• ? 28.66 to 32.34
• Conclusion population mean of jean consumers
  falls between 28.66 and 32.34 with 95 confidence

42
Estimation

• Confidence interval for independent measures
  studies
• ?1 - ?2 X1 X2 ts m1 m2
• the logic is the same, only here the goal is to
  find a point estimate of the difference of the
  population means and a confidence interval for
  the same difference

43
Estimation

• Confidence interval for independent measures
  studies
• Example. Comparison of two different diets. DV
  learning performance of 2 samples

44
Estimation

• Confidence interval for independent measures
  studies
• ?1 - ?2 X1 X2 ts m1 m2
• Point estimate X1 X2 33-25 8
• s m1 m2 v(sp2/n1 sp2/n2)
• Where sp2 (SS1 SS2)/(df1 df2)
• sp2 (250 140)/(9 4) 390/13 30

45
Estimation

• Confidence interval for independent measures
  studies
• s m1 m2 v(sp2/n1 sp2/n2)
• s m1 m2 v(30/10 30/5)
• v(36) 3
• ?1 - ?2 X1 X2 ts m1 m2
• ?1 - ?2 8 t 3

46
Estimation

• Confidence interval for independent measures
  studies
• ?1 - ?2 8 t 3
• Suppose you want to compute an 80 confidence
  interval, then look up in table for
• t with df 10 -1 5 -1 13
• t (13) 1.35
• ?1 - ?2 8 1.35 3 8 4.05
• Conclusion. 80 confidence interval for mean
  increase in performance is 8 4.05

47
Estimation

• Confidence interval for repeated measures studies
• Logic is the same. Want to determine confidence
  interval for D, the difference between two
  related (or repeated) measures
• e.g. Purpose to determine effectiveness of a
  reading program. Test n16 students on a test of
  reading comprehension administer reading
  program re-administer test of reading
  comprehension
• Results Dbar 21 SS for difference scores
  1215. Construct point estimate of ?D and a 90
  confidence interval
• ?D Dbar tsDbar

48
Estimation

• Confidence interval for repeated measures studies
• ?D Dbar tsDbar
• sDbar vs2/n, where s2 SS/ (n-1) 1215/15 81
• sDbar vs2/n v81/16 2.25
• ?D Dbar tsDbar 21 t (2.25)
• But t (15) 1.753
• ?D 21 1.75 (2.25)
• 21 3.94
• Conclusion 90 confidence interval for mean
  improvement in reading comprehension is 21 3.94

49
Estimation

• Confidence interval width
• Width is affected by the sample size larger the
  sample size the smaller the confidence interval
• Width is also affected by the magnitude of the
  confidence higher the confidence greater the
  width