INTRODUCTION TO EMPIRICAL MODELS

1
Regression Analysis
2
CHAPTER OUTLINE

• INTRODUCTION TO EMPIRICAL MODELS
• LEAST SQUARES ESTIMATION OF THE PARAMETERS
• PROPERTIES OF THE LEAST SQUARES ESTIMATORS
• AND ESTIMATION OF s 2
• HYPOTHESIS TESTING IN LINEAR REGRESSION
• CONFIDENCE INTERVALS IN LINEAR REGRESSION
• PREDICTION OF NEW OBSERVATIONS
• ASSESSING THE ADEQUACY OF THE REGRESSION MODEL

3
Definitions

• Regress
• The act of reasoning backward
• Regression
• A functional relationship between two or more
  correlated variables that is often empirically
  determined from data and is used esp. to predict
  values of one variable when given values of the
  others.

4
Models

• Abstraction/simplification of the system used as
  a proxy for the system itself
• Can try wide-ranging ideas in the model
• Make your mistakes on the computer where they
  dont count, rather for real where they do count
• Issue of model validity
• Two types of models
• Physical (iconic)
• Logical/Mathematical -- quantitative and logical
  assumptions, approximations

5
What Do You Do with a Logical Model?

• If model is simple enough, use traditional
  mathematics (queueing theory, differential
  equations, linear programming) to get answers
• Nice in the sense that you get exact answers to
  the model
• But might involve many simplifying assumptions to
  make the model analytically tractable --
  validity??
• Many complex systems require complex models for
  validity simulation needed

6
INTRODUCTION TO EMPIRICAL MODELS

• models
• theoretical (mechanical) model
• empirical model
• scatter diagram

7
(No Transcript)
8
(No Transcript)
9

• linear model
• (equation)
• probabilistic linear
• model
• simple linear
• regression model
• regression
• coefficients

10

• multiple regression model
• multiple linear regression
• model
• intercept
• partial regression coefficients
• contour plot

11

• dependent variable or response y may be related
  to k
• independent or regressor variables
• interaction
• any regression model that is linear in
  parameters (the
• bs) is a linear regression model, regardless
  of the
• shape of the surface that it generates.

12
(No Transcript)
13
(No Transcript)
14
LEAST SQUARES ESTIMATION OF THE PARAMETERS
Simple Linear Regression
15

• method of least squares
• least squares normal equations
• fitted or estimated regression line
• residual

16
(No Transcript)
17
Example 10-1, pp. 436
18
(No Transcript)
19
(No Transcript)
20
Multiple Linear Regression
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
PROPERTIES OF THE LEAST SQUARES ESTIMATORS AND
ESTIMATION OF s2

• unbiased estimators
• covariance matrix
• estimated standard error
• residual mean square (or error mean square)

25
Hypothesis Testing on b0and b1, pp. 447
26
HYPOTHESIS TESTING IN LINEAR REGRESSION
27
(No Transcript)
28
k p - 1
29
(No Transcript)
30
Tests on Individual Regression Coefficients
31
Confidence Intervals on Individual Regression
Coefficients
32
(No Transcript)
33
Confidence Interval on the Mean Response
34
(No Transcript)
35
(No Transcript)
36
PREDICTION OF NEW OBSERVATIONS
37
(No Transcript)
38

• simple linear regression

39
(No Transcript)
40
ASSESSING THE ADEQUACY OF THE REGRESSION MODEL

• normal probability plot of residuals
• standardize
• outlier

41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
Coefficient of Multiple Determination
50
(No Transcript)
51
Influential Observations
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)