Introduction to Regression Analysis

1
Introduction to Regression Analysis
2
Dependent variable (response variable)

• Measures an outcome of a study
• Income
• GRE scores
• Dependent variable Mean (expected value)
  random error
• y E(y) e
• If y is normally distributed, know the mean and
  the standard deviation, we can make a probability
  statement

3
Probability statement

• Lets say the mean cholesterol level for graduate
  students 250
• Standard deviation 50 units
• What does this distribution look like?
• the probability that ____s cholesterol will
  fall within 2 standard deviations of the mean is
  .95

4
Independent variables (predictor variables)

• explains or causes changes in the response
  variables
• (The effect of the IV on the DV)
• (Predicting the DV based on the IV)
• What independent variables might help us predict
  cholesterol levels?

5
Examples

• The effect of a reading intervention program on
  student achievement in reading
• Predict state revenues
• Predict GPA based on SAT
• predict reaction time from blood alcohol level

6
Regression Analysis

• Build a model that can be used to predict one
  variable (y) based on other variables (x1, x2,
  x3, xk,)
• Model a prediction equation relating y to x1,
  x2, x3, xk,
• Predict with a small amount of error

7
Typical Strategy for Regression Analysis
8
Fitting the Model Least Squares Method

• Model an equation that describes the
  relationship between variables
• Lets look at the persistence example

9
Method of Least Squares

• Lets look at the persistence example

10
Finding the Least Squares Line

• Slope
• Intercept
• The line that makes the vertical distances of the
  data points from the line as small as possible
• The SE Sum of Errors (deviations from the line,
  residuals) equals 0
• The SSE (Sum of Squared Errors) is smaller than
  for any other straight-line model with SE0.

11
Regression Line

• Has the form y a bx
• b is the slope, the amount by which y changes
  when x increases by 1 unit
• a is the y-intercept, the value of y when x 0
  (or the point at which the line cuts through the
  x-axis)

12
Simplest of the probabilistic models
Straight-Line Regression Model

• First order linear model
• Equation y ß0 ß1x e
• Where
• y dependent variable
• x independent variable
• ß0 y-intercept
• ß1 slope of the line
• e random error component

13
Lets look at the relationship between two
variables and construct the line of best fit

• Minitab example Beers and BAC