Linear Regression and Correlation

1
Chapter 11

• Linear Regression and Correlation

2
Linear Regression and Correlation

• Explanatory and Response Variables are Numeric
• Relationship between the mean of the response
  variable and the level of the explanatory
  variable assumed to be approximately linear
  (straight line)
• Model

• b1 gt 0 ? Positive Association
• b1 lt 0 ? Negative Association
• b1 0 ? No Association

3
Least Squares Estimation of b0, b1

• b0 ? Mean response when x0 (y-intercept)
• b1 ? Change in mean response when x increases by
  1 unit (slope)
• b0, b1 are unknown parameters (like m)
• b0b1x ?? Mean response when explanatory
  variable takes on the value x
• Goal Choose values (estimates) that minimize the
  sum of squared errors (SSE) of observed values to
  the straight-line

4
Example - Pharmacodynamics of LSD

• Response (y) - Math score (mean among 5
  volunteers)
• Predictor (x) - LSD tissue concentration (mean
  of 5 volunteers)
• Raw Data and scatterplot of Score vs LSD
  concentration

Source Wagner, et al (1968)
5
Least Squares Computations
Parameter Estimates
Summary Calculations
6
Example - Pharmacodynamics of LSD
(Column totals given in bottom row of table)
7
SPSS Output and Plot of Equation
8
Inference Concerning the Slope (b1)

• Parameter Slope in the population model (b1)
• Estimator Least squares estimate
• Estimated standard error
• Methods of making inference regarding population
• Hypothesis tests (2-sided or 1-sided)
• Confidence Intervals

9
Hypothesis Test for b1

• 1-sided Test
• H0 b1 0
• HA b1 gt 0 or
• HA- b1 lt 0

• 2-Sided Test
• H0 b1 0
• HA b1 ? 0

10
(1-a)100 Confidence Interval for b1

• Conclude positive association if entire interval
  above 0
• Conclude negative association if entire interval
  below 0
• Cannot conclude an association if interval
  contains 0
• Conclusion based on interval is same as 2-sided
  hypothesis test

11
Example - Pharmacodynamics of LSD

• Testing H0 b1 0 vs HA b1 ? 0

• 95 Confidence Interval for b1

12
Confidence Interval for Mean When xx

• Mean Response at a specific level x is
• Estimated Mean response and standard error
  (replacing unknown b0 and b1 with estimates)
• Confidence Interval for Mean Response

13
Prediction Interval of Future Response _at_ xx

• Response at a specific level x is
• Estimated response and standard error (replacing
  unknown b0 and b1 with estimates)
• Prediction Interval for Future Response

14
Correlation Coefficient

• Measures the strength of the linear association
  between two variables
• Takes on the same sign as the slope estimate from
  the linear regression
• Not effected by linear transformations of y or x
• Does not distinguish between dependent and
  independent variable (e.g. height and weight)
• Population Parameter ryx
• Pearsons Correlation Coefficient

15
Correlation Coefficient

• Values close to 1 in absolute value ? strong
  linear association, positive or negative from
  sign
• Values close to 0 imply little or no association
• If data contain outliers (are non-normal),
  Spearmans coefficient of correlation can be
  computed based on the ranks of the x and y values
• Test of H0ryx 0 is equivalent to test of
  H0b10
• Coefficient of Determination (ryx2) - Proportion
  of variation in y explained by the regression
  on x

16
Example - Pharmacodynamics of LSD
Syy
SSE
17
Example - SPSS OutputPearsons and Spearmans
Measures
18
Hypothesis Test for ryx

• 1-sided Test
• H0 ryx 0
• HA ryx gt 0 or
• HA- ryx lt 0

• 2-Sided Test
• H0 ryx 0
• HA ryx ? 0

19
Analysis of Variance in Regression

• Goal Partition the total variation in y into
  variation explained by x and random variation

• These three sums of squares and degrees of
  freedom are
• Total (TSS) DFT n-1
• Error (SSE) DFE n-2
• Model (SSR) DFR 1

20
Analysis of Variance for Regression

• Analysis of Variance - F-test
• H0 b1 0 HA b1 ?? 0

21
Example - Pharmacodynamics of LSD

• Total Sum of squares

• Error Sum of squares

• Model Sum of Squares

22
Example - Pharmacodynamics of LSD

• Analysis of Variance - F-test
• H0 b1 0 HA b1 ?? 0

23
Example - SPSS Output