Regression with two predictors

1
G89.2229 Multiple Regression Week 3 (Wednesday)

• Regression with two predictors
• Path models
• Mediation, adjustment, suppression
• Standard errors of B weights
• Partial and semi-partial correlations

2
The geometry of E(YX1 X2) b0 b1X1 b2X2

• E(YX1 X2) b0 b1X1 b2X2 describes a plane.
• For every change in either X1 or X2, E(YX1 X2)
  changes linearly
• The distribution of X1 and X2 does not affect the
  form of the plane itself, but it does affect the
  estimates of b0, b1, and b2.
• MOST IMPORTANTLY, The distribution of X1 and X2
  has a profound effect on how the semipartial
  effects, b0, b1, and b2 relate to bivariate
  correlations.

3
When X1 and X2 are uncorrelated

• In experiments, we manipulate X1 and X2 so that
  they are balanced, and uncorrelated
• X1 and X2 are said to be "orthogonal"
• When X1 and X2 are orthogonal the semipartial
  effects are identical to the bivariate effects of
  X1 or X2 on Y

X2
X1
4
When X1 and X2 are correlated

• In surveys we usually observe X1 and X2 without
  adjusting their relative frequency
• As an exception, sometimes we oversample certain
  rare values
• When X1 and X2 are correlated the semipartial
  effects are different from the bivariate effects
  of X1 or X2 on Y

5
Path models

• A useful way to think about regression models is
  through path diagrams.
• Y a bX e

e
b
X
Y
Y B0 B1X1 B2X2 e
6
Mediation Model

• Suppose we have an association between X and
  YThat we believe is mediated by a variable M

The indirect effect of X on Y through M is ab.
If c' is zero, complete mediation is suggested.
If c' is less than c, then partial mediation is
suggested.
7
Two equations for Mediation Model

• M a0 aX eM
• Y b0 bM c'X eY
• When M in the second is replaced with M in the
  first, we see where the multiplicative path comes
  from.
• Special path terms
• M and Y are endogenous
• X is exogenous

8
Example of Mediation

• Suppose that X is time spent studying for GRE
  verbal test.
• Suppose Y is the Verbal GRE score.
• Suppose M is the number of new vocabulary words
  learned.
• A pattern of complete mediation would follow from
  the following correlations
• r(XY) .35
• r(XM) .70
• r(MY) .50
• See Shrout and Bolger (2002) for other examples.

9
Algebraic thinking about semipartial coefficients

• The OLS estimates of B0, B1 and B2 are those
  numbers which make the sum of the squared E terms
  as small as possible.
• Some math yields the following

• The factor to the right is the standardized
  coefficient
• The numerator of that factor shows the adjustment
  for the second variable.

10
To Test B estimate with Wald statistic

• Compute standard error of Bi and compute Wald
  Statistic
• Divide Bi by the square root of this variance
• Under the null hypothesis, H0Bi0, this
  statistic has a central t distribution on (n-3)
  degrees of freedom
• The same standard error can be used to compute
  symmetric confidence bound
• Bj (sej)(t.5a)

11
Suppression

• Sometimes the semipartial effect for X1 (i.e. b1)
  inY b0 b1X1 b2X2 eis larger in absolute
  magnitude than the bivariate effect inY b0
  b1X1 e
• This has been called suppression
• Example
• X1 is stress
• Y is distress
• X2 is coping
• Classic pattern is when one of the three
  correlations is negative.

12
Spurious effect

• Consider a path model that resembles the
  mediation model.
• Suppose that there is a bivariate association
  between X and Y, but when W is considered, the
  semipartial effect b is zero. The original
  association is often said to be "spurious". It
  is explained by the common cause, W.