Chapter 8: Regression Models for Quantitative and Qualitative Predictors

1
Chapter 8 Regression Models for Quantitative and
Qualitative Predictors
Ayona Chatterjee Spring 2008 Math 4813/5813
2
Polynomial Regression Models

• When the true curvilinear response function is
  indeed a polynomial.
• When the true curvilinear response function is
  unknown but the polynomial is a good
  approximation to the true function.

3
One Predictor Variable Second Order

• Let us consider a polynomial model with one
  variable raised to the first and second order.
• This polynomial is called a second-order model
  with one predictor.

4
One Predictor Variable Second Order

• Note the second order regression equation in one
  variable represents a parabola.

Here ß1 is called the linear effect coefficient
and ß11 is called the quadratic effect
coefficient.
5
One Predictor Variable Third Order

• The third-order model with one predictor variable
  is given as

6
Two Predictor Variables Second Order

• The regression model
• This is a second-order model with two predictor
  variables.
• The equation represents a conic section.

7
Example of a Quadratic Response Surface
8
Hierarchical Approach to Fitting

• The norm is to fit a second-order or a
  third-order polynomial and explore if a lower
  order model is adequate.
• For example if we have a third-order model in one
  variable, we may want to test of ß1110, or
  whether or not both ß11 and ß111 equal zero.
• We use extra sums of squares to do the test.

9
Extra Sums of Squares

• To test if ß1110 we would use SSR(x3x, x2). If
  we want to test if both ß11 and ß111 equal zero
  then we would use SSR(x2, x3x).
• Note SSR(x2, x3 x) SSR(x2x) SSR(x3x2,
  x).
• If a polynomial of a given order is retained,
  then all related terms of lower-order are also
  retained.

10
Regression Function in Terms of X

• To revert back to the original scale, and un do
  the centering of the predictor variables, we use
  the following transformations.

11
Example

• A researcher studied the effects of the charge
  rate and temperature on the life of a new type of
  power cell in a small-scale experiment. The
  charge rate (X1) was controlled at 3 level, and
  so was the ambient temperature (X2). The life of
  the power cell was the response (Y).
• The researcher decided to fit a second-order
  polynomial regression model.

12
Data Set - Power Cells Example

• Scale the units and fit the second order
  polynomial regression model.
• Obtain the correlation between the new variables
  x and original X. Has the transformation reduced
  collinearity?

13
Test of Fit

• An F test to test the goodness of fit of the
  model to the data.
• Define
• If F is greater than F table value, the model is
  not a good fit.

14
Partial F Test

• Suppose for the given data you want to test if a
  first-order model is sufficient.
• Here H0 ?11 ?22 ?120
• The F statistics

15
Interaction Regression Models

• A regression model with p-1 variables contains
  additive effects if the response function can be
  written as
• EY f1(X1)f2(X2) fp-1(Xp-1)
• Note all functions need not be simple.
• If a response function cannot be written as
  above, then the model is not additive and
  interaction terms are present.

16
Interpretation of Interaction Regression Models

• In presence of interaction term, the regression
  coefficients cannot be interpreted as before.
• For a first-order model with interaction term,
  the change in the mean response with a unit
  increase in X1 when X2 is held constant is ?1
  ?3X2 and not just ?1 .

17
Reinforcement Effects

• When the regression coefficients are positive, we
  say the interaction effect between the two
  quantitative variables is of a reinforcement or
  synergistic when the slope of the response
  function against one of the predictor variables
  increases for higher levels of the predictor
  variables. That is when ?3 is positive.

18
Interference Effects

• When the regression coefficients are positive, we
  say the interaction effect between the two
  quantitative variables is of an interference or
  antagonistic type when the slope of the response
  function against one of the predictor variables
  decreases for higher levels of the predictor
  variables. That is when ?3 is negative.

19
Implementing an Interaction Model

• There are two points to keep in mind
• High multicollinearity may exist between some
  predictors and hence centering the variables may
  help in reducing this problem.
• If there are larger number of predictors, then we
  have a large choice for possible interaction
  terms. Choose only the terms that you think will
  influence the response.

20
Qualitative Predictors

• Example Y is speed at which an insurance
  innovation is adopted, X1 is the size of the
  firm, and another predictor variable to identify
  type of firm.
• Here let the firm types be stock or mutual
  company. Thus we can define

21
Principle

• A qualitative variable with c classes will be
  represented by c-1 indicator variable, each
  taking on the values 0 and 1.
• We modify the previous example as

22
Qualitative Predictor with More than Two Classes

• Suppose the regression on tool wear (Y) on tool
  speed (X1) and tool model. Tool model is a
  qualitative variables with M1, M2, M3 and M4
  possible models.

23
Indicator Variables versus Allocated Codes

• An alternative to using indicator variables is to
  use allocated codes.
• Consider, for instance the predictor variable
  frequency of product use, which has three
  classes.
• Frequent user 3
• Occasional user 2
• Nonuser - 1
• Here we have Yi?0?1Xi1error.
• This coding implies that the mean response
  changes by the same amount when going from a
  nonuser to an occasional user as when going from
  occasional user to frequent user.

24
Why indicator variables?

• Indicator variables make no assumptions about the
  pacing of the classes.
• They reply on data to show the differential
  effects.
• Alternative model Yi?0?1Xi1?2Xi2error
• Here X1 1 for frequent user
• X2 1 for occasional user
• All other cases we have zero.

25
Quantitative to Qualitative

• Sometimes we may convert quantitative data to
  qualitative data, for example ages can be grouped
  and we can use indicator variables to denote the
  age groups.
• An alternative coding is to use 1 and 1 for the
  two levels of a qualitative factor.

26
Comparison of Two or More Regression
Functions-Example

• We can compare regression functions using
  hypothesis testing and see if two functions
  represent the same response function or now.
• Examples.

27
Comparison of Two or More Regression
Functions-Example

• A company operates two production lines for
  making soap bars. For each line, the relation
  between the speed of the line and the amount of
  the scrap for the day was studied. A scatter plot
  of the data for the two production lines suggest
  that the regression relation between production
  line speed and amount of scarp is linear but not
  the same for the two production lines. The slopes
  appear same but the heights of the regression
  lines differ. A formal test is desired to
  determine if the two regression lines are
  identical.

28
Soap Production line - Example

• First fit separate regression models for both
  production lines.
• Next combine all the data and using an indicator
  variable fit a first-order regression model with
  interaction.
• Identity of the regression functions for the two
  production lines is tested by considering the
  alternatives
• H0 ?2?30 and H0 ?30