Chapter 5 Multiple Linear Regression

1
Chapter 5Multiple Linear Regression
2
Agenda

• Introduction
• Least Squares Estimation of the Parameters
• Matrix Approach
• Estimating ?2
• Properties of the least square estimators
• Hypothesis tests in multiple linear regression
• Confidence intervals

3
Introduction

• Many applications of regression analysis involve
  situations in which there are more than one
  regressor variable.
• A regression model that contains more than one
  regressor variable is called a multiple
  regression model.

4
Introduction

• For example, suppose that the effective life of
  a cutting tool depends on the cutting speed and
  the tool angle. A possible multiple regression
  model could be

where Y tool life x1 cutting speed x2 tool
angle
5
Introduction
(a) The regression plane for the model E(Y)
50 10x1 7x2. (b) The contour plot

6
Introduction
7
Introduction

• Three-dimensional plot of the regression model
  E(Y) 50 10x1 7x2 5x1x2.
• The contour plot

8
Introduction

• Three-dimensional plot of the regression model
  E(Y) 800 10x1 7x2 8.5x12 5x22 4x1x2.
• The contour plot

9
Least Squares Estimation of the Parameters
10
Least Squares Estimation of the Parameters

• The least squares function is given by

• The least squares estimates must satisfy

11
Least Squares Estimation of the Parameters

• The least squares normal Equations are

• The solution to the normal Equations are the
  least squares estimators of the regression
  coefficients.

12
Example 1
13
Example 1
14
Matrix of scatter plots for the wire bond pull
strength data
15
Example 1
16
Example 1
17
Example 1
18
Matrix Approach to Multiple Linear Regression
Suppose the model relating the regressors to the
response is
In matrix notation this model can be written as
19
Matrix Approach to Multiple Linear Regression
where
20
Matrix Approach to Multiple Linear Regression
We wish to find the vector of least squares
estimators that minimizes
The resulting least squares estimate is
21
Matrix Approach to Multiple Linear Regression
22
Example 2
23
Example 2
24
Example 2
25
Example 2
26
Example 2
27
Example 2
28
Minitab Practice

• Data file Example 5_2.xls
• Menu ? Stat ? Regression ? Regression
• response y
• predictors x1 x2
• ? Options Select variance inflation factors
• PRESS and predicted R-sq.
• Prediction intervals 8 275
• Select all options
• ? Results select In addition .

29
(No Transcript)
30
Estimating ?2
An unbiased estimator of ?2 is
31
Properties of the Least Squares Estimators
Unbiased estimators
Covariance Matrix
32
Properties of the Least Squares Estimators
Individual variances and covariances
In general,
33
Hypothesis Tests in Multiple Linear Regression
1. Test for Significance of Regression
The appropriate hypotheses are
The test statistic is
34
Hypothesis Tests in Multiple Linear Regression
1. Test for Significance of Regression
35
Example 3
36
Example 3
37
Example 3
38
Example 3
39
Hypothesis Tests in Multiple Linear Regression
R2 and Adjusted R2
The coefficient of multiple determination

• For the wire bond pull strength data, we find
  that R2 SSR/SST 5990.7712/6105.9447 0.9811.
• Thus, the model accounts for about 98 of the
  variability in the pull strength response.

40
Hypothesis Tests in Multiple Linear Regression
R2 and Adjusted R2
The adjusted R2 is

• The adjusted R2 statistic penalizes the analyst
  for adding terms to the model.
• It can help guard against overfitting
  (including regressors that are not really useful)

41
Hypothesis Tests in Multiple Linear Regression
2. Tests on Individual Regression Coefficients
and Subsets of Coefficients
The hypotheses for testing the significance of
any individual regression coefficient
42
Hypothesis Tests in Multiple Linear Regression
2. Tests on Individual Regression Coefficients
and Subsets of Coefficients
The test statistic is

• Reject H0 if t0 gt t?/2,n-p.
• This is called a partial or marginal test

43
Example 4
44
Example 4
45
Hypothesis Tests in Multiple Linear Regression
The general regression significance test or the
extra sum of squares method
We wish to test the hypotheses
46
Hypothesis Tests in Multiple Linear Regression
A general form of the model can be written
where X1 represents the columns of X associated
with ?1 and X2 represents the columns of X
associated with ?2
47
Hypothesis Tests in Multiple Linear Regression
For the full model
If H0 is true, the reduced model is
48
Hypothesis Tests in Multiple Linear Regression
The test statistic is
Reject H0 if f0 gt f?,r,n-p The test in Equation
(12-32) is often referred to as a partial F-test
49
Example 5
50
Example 5
51
Example 5
52
Confidence Intervals in Multiple Linear Regression
1. Confidence Intervals on Individual Regression
Coefficients
Definition
53
Example 6
54
Confidence Intervals in Multiple Linear Regression
2. Confidence Interval on the Mean Response
The mean response at a point x0 is estimated by
The variance of the estimated mean response is
55
Confidence Intervals in Multiple Linear Regression
2. Confidence Interval on the Mean Response
Definition
56
Example 7
57
Example 7