Multivariate Data Analysis Chapter 4

1
Multivariate Data AnalysisChapter 4 Multiple
Regression

• MIS 6093 Statistical Method
• Instructor Dr. Ahmad Syamil

2
Chapter 4What is Multiple Regression Analysis?

• An Example of Simple and Multiple Regression
• Setting a Baseline Prediction Without an
  Independent Variable
• Prediction Using A Single Independent Variable
  Simple Regression
• The Role of the Correlation Coefficient
• Specifying the Simple Regression Equation
• Establishing a Confidence Interval for the
  Prediction
• Assessing Prediction Accuracy

3
Chapter 4What is Multiple Regression Analysis?

• Prediction Using Several Independent Variables
  Multiple Regression
• The Impact of Multicollinearity
• The Multiple Regression Equation
• Adding a Third Independent Variable
• Summary

4
Chapter 4A Decision Process for Multiple
Regression Analysis

• Stage 1 Objectives of Multiple Regression
• Research Problems Appropriate for Multiple
  Regression
• Prediction with Multiple Regression
• Explanation with Multiple Regression
• Specifying a Statistical Relationship
• Selection of Dependent and Independent Variables

5
Chapter 4A Decision Process for Multiple
Regression Analysis Cont.

• Stage 2 Research Design of a Multiple Regression
  Analysis
• Sample Size
• Statistical Power and Sample Size
• Generalizability and Sample Size
• Fixed Versus Random Effects Predictors
• Creating Additional Variables
• Incorporating Nonmetric Data with Dummy Variables
• Representing Curvilinear Effects with Polynomials
• Representing Interaction or Moderator Effects
• Summary

6
Chapter 4A Decision Process for Multiple
Regression Analysis Cont.

• Stage 3 Assumptions in Multiple
• Regression Analysis
• Assessing Individual Variables Versus the Variate
• Linearity of the Phenomenon
• Constant Variance of the Error Term
• Independence of the Error Terms
• Normality of the Error Term Distribution
• Summary

7
Chapter 4A Decision Process for Multiple
Regression Analysis Cont.

• Stage 4 Estimating the Regression Model
• and Assessing Overall Fit
• General Approaches to Variables Selection
• Confirmatory Specification
• Sequential Search Methods
• Combinational Approach
• Overview of the Model Selection Approaches
• Testing the Regression Variate for Meeting the
  Regression Assumptions

8
Chapter 4A Decision Process for Multiple
Regression Analysis Cont.

• Stage 4 Estimating the Regression Model
• and Assessing Overall Fit
  (Cont.)
• Examining the Statistical Significance of Our
  Model
• Significance of the Overall Model The
  Coefficient of Determination
• Significance Tests of Regression Coefficients
• Identifying Influential Observations

9
Chapter 4A Decision Process for Multiple
Regression Analysis Cont.

• Stage 5 Interpreting the Regression
• Variate
• Using the Regression Coefficients
• Standardizing the Regression Coefficients Beta
  Coefficients
• Assessing Multicollinearity
• The Effect of Multicollinearity
• Identifying Multicollinearity
• Remedies for Multicollinearity

10
Chapter 4A Decision Process for Multiple
Regression Analysis Cont.

• Stage 6 Validation of the Results
• Additional or Split Samples
• Calculating the PRESS Statistics
• Comparing Regression Models
• Predicting with the Model

11
Chapter 4Illustration of a Regression Analysis

• Stage 1 Objectives of the Multiple
• Regression
• Stage 2 Research Design of the Multiple
• Regression Analysis
• Stage 3 Assumptions of the Multiple
• Regression Analysis

12
Chapter 4Illustration of a Regression Analysis
(Cont.)

• Stage 4 Estimating the Regression Model
• and Assessing Overall Model Fit
• Stepwise Estimation Selecting the First Variable
• Stepwise Estimation Adding X3
• Stepwise Estimation A Third Variable is Added
• ----
  X6
• Evaluating the Variate for the Assumptions of
  Regression Analysis
• Identifying Outliers as Influential Observations

13
Chapter 4Illustration of a Regression
Analysis(Cont.)

• Stage 5 Interpreting the Variate
• Measuring the Degree and Impact of
  Multicollinearity
• Stage 6 Validating the Results
• Evaluating Alternative Regression Models
• A Confirmatory Regression Models
• Including a Nonmetric Independent Variable
• A Managerial Overview of the Results

14
Chapter 4

• Summary
• Questions
• References
• ..to Chapter 4A

15
Chapter 4A

• Assessing Multicollinearity
• A Two-Part Process
• An Illustration of Assessing Multicollinearity

16
Chapter 4aIdentifying Influential Observations

• Step 1 Examining Residuals
• Analysis of Residuals
• Partial regression plots
• Step 2 Identifying Leverage Points from
• the Predictors
• Hat Matrix
• Mahalanobis distance (D2)

17
Chapter 4aIdentifying Influential Observations
(Cont.)

• Step 3 Single-Case Diagnostics
• Identifying Influential
  Observations
• Influences on individual coefficients
• Overall influence measures
• Step 4 Selecting and Accommodating
• Influential Observations

18
Chapter 4aIdentifying Influential Observations
(Cont.)

• Example from the HATCO Database
• Step 1 Examining the Residuals
• Step 2 Identifying Leverage Points
• Step 3 Single-Case Diagnostics
• Step 4 Selecting and Accommodating
• Influential Cases
• Overview

19
Chapter 4A

• Summary
• Questions
• References
• end