Introduction the General Linear Model (GLM)

1
Introduction the General Linear Model (GLM)

• what model, linear general mean
• bivariate, univariate multivariate GLModels
• kinds of variables
• some common models

2
General Linear Model

• Model means that we are usually interested in
  predicting or modeling the values of one
  variable (criteria) from the values of one or
  more others (the predictors)
• Linear means that the variables will be
  linearly transformed ( /) and linearly
  combined ( -) to produce the models
  estimates
• General means that the model intends to provide
  a way to model test RHs about any combination
  of criterion and predictor variables (i.e., any
  model), and to test RHs about comparisons among
  models

3
Regression vs. GLM

• The constant is often represented differently
  in GLM than in multiple regression
• Single predictor models
• ? single predictor regression y bx
  a
• ? single predictor GLM y b0
  b1x1
• Multiple predictor models
• multiple predictor regression y b1x1
  b2x2 a
• multiple predictor GLM y b0 b1x1
  b2x2

4
Common kinds of GLModels

• Bivariate ? one criterion one predictor
  ? simple regression
• y b0 b1x
• Univariate ? one criterion multiple predictors
  ? multiple regression in
  all its forms
• y b0 b1x1 b2x2 b3x3
• Multivariate ? multiple criterion multiple
  predictors ? canonical regression in
  all its forms
• b0 b1y1 b2y2 b0 b1x1 b2x2 b3x3

5
Common kinds of variables

• Quantitative variables
• Raw variable
• Centered variables X mean
• Mean ? 0 simplifies math of more complicated
  models
• Re-centered variables X ? a more meaningful
  value
• Change start or stop values
• E.g., aging intellectual decline
• Mathematical trick to get the desired
  model/weights
• selecting which group or value will be
  represented in models bs

6
Common kinds of variables

• Quadratic quantitative variables
• X2 actually represents combination of linear
  quadratic
• Xcen2 represents the pure quadratic term
• Model with X2 will have R2 as model with Xcen
  Xcen2
• A model with a quadratic term should always
  include the linear term for that variable

7
Common kinds of variables

• 2-group variables
• Unit coding (usually 1-2)
• Dummy Coding
• control or comparison group coded 0
• treatment or target group coded 1
• Effect Coding
• control or comparison group coded -1
• treatment or target group coded 1

8
Common kinds of variables

• k-group variables
• Raw coding (usually 1-2-3, etc.)
• Dummy Coding
• control or comparison group coded 0
• treatment or target groups coded 1 on one
  variable 0 on all others
• the full set of codes must be included in the
  model
• Effect Coding
• control or comparison group coded -1
• treatment or target group coded 1 on one
  variable 0 on all others
• the full set of codes must be included in the
  model

9
Common kinds of variables

• K-groups variables, cont.
• Comparison coding
• Combining simple and complex analytical
  comparison codes to represent specific,
  hypothesis driven, group comparisons
• E.g., Say you have 4 groups and RH that
• Group 1 has higher scores that the average scores
  of groups 2-4 the codes would be gp1 3 gp2
  -1 gp3 -1 gp4 -1
• Groups 2 3 have higher average scores than do 1
  4 the codes would be gp1 -1 gp2 1 gp3
  1 gp4 -1
• Group 2 has higher scores than the average scores
  of groups 3-4 the codes would be gp1 0 gp2
  2 gp3 -1 gp4 -1
• Usually havea set of k-1 codes

10
Common kinds of variables

• K-groups variables, cont.
• Polynomial coding
• If the groups represent a quantitative
  continuum, you use codes to represent different
  polynomial functions (linear, quadratic, cubic,
  etc.) to explore the shape of the relationship
  between that variable and the criterion
• E.g., for a 5-group variable, the polynomial
  codes are
• Linear -2 -1 0 1 2
• Quadratic 2 -1 -2 -1 2
• Cubic -1 2 0 -2 1
• Quartic 1 -4 6 -4 1
• the full set of codes must be included in the
  model

11
Common kinds of variables

• Ordered-category variables
• Sometimes you have a quantitative variable that
  you want to change into a set of ordered
  categories
• e.g. ? grade into A B C D F
• e.g. ? grade into Pass Fail
• e.g. ? aptitude test scores into remedial
  normal gifted
• Sometimes this is done to help with ill-behaved
  distributions
• e.g. ? frequency variable with mean1.1,
  std8.4, sk4.2
• e.g. ? frequency variable with 60 0 38
  1 max 118
• Important because ? skewed univariate
  distributions can create
  apparently nonlinear bivariate relationships

12
Common kinds of variables

• Ordered-category variables, cont.
• Once you form the ordered categories (using IF,
  RECODE or other transformations), you can
  enter those variables into the GLM in different
  ways
• Using the category values (e.g., 1, 2, 3, etc)
  
• Centering or re-centering the category values
• Dummy codes of the category values
• Effect codes of the category values
• Polynomial codes of the category values
• indicates approaches that make assumptions
  about the interval nature of the variable and/or
  its normal distribution, with which not everyone
  agrees!

13
Common kinds of variables

• Interactions
• Interactions represent the joint effect or
  non-additive combination of 2 or more
  predictors as they relate to a criterion (or set
  of criteria in the multivariate case).
• They are the moderation, it depends,
  sometimes, or maybe that makes our science
  and statistical analyses so interesting.
• Interactions can be formed from the combination
  of any 2 or more variables of the types just
  discussed.
• There are some guidelines about forming,
  including and interpreting interaction terms.

14
Common kinds of variables

• Interaction ? Guidelines
• When including a 2-way interaction, both related
  main effects must be included
• When including a 3-way interaction, all 3 main
  effects and all 3 2-way interactions must be
  included
• When including a non-linear interaction term, the
  related linear and nonlinear main effects, and
  linear interaction terms must be included
• The associated terms can not exceed the df of the
  variables involved (except for quantitative
  variables)

15
Common kinds of GLModels

• Linear Multiple regression models
• y b0 b1x1 b2x2 b3x3
• Can include any of the variable types
• Quantitative (raw, centered or re-centered)
• 2- or k-group (with dummy, effect, or comparison
  coding)
• Ordered category (coded)

16
Common kinds of GLModels

• Non-Linear Multiple regression models with
  quant variables
• y b0 b1x1 b2x12 b3x2 b4x22
• Can include any of the variable types
• Linear terms should be centered
• Non-linear terms should be centered then powered
• Non-linear terms above quadratic should be based
  on theory
• Include linear term for all non-linear terms, at
  least at first

17
Common kinds of GLModels

• 2-way Interaction Multiple regression models
• y b0 b1x b2z b3xz
• Can include any of the variable types
• Quantitative variables should be centered
• 2- or k-group variables should be coded
• Interaction terms formed as product of main
  effect terms
• Must included main effects terms for any
  interaction variable

18
Common kinds of GLModels

• 3-way Interaction Multiple regression models
• y b0 b1x b2z b3v b4xz b5xv b6zv
  b7xzv
• Can include any of the variable types
• Quantitative variables should be centered
• 2- or k-group variables should be coded
• Interaction terms formed as product of main
  effect terms
• Must included main effects terms for any
  interaction variable

19
Common kinds of GLModels

• 2-group ANOVA models
• y b0 b1x1
• X is a dummy or effect coded 2-group variable

20
Common kinds of GLModels

• 3-group ANOVA models
• y b0 b1x1 b2x2
• X1 X2 are a dummy or effect codes for a
  3-group variable

21
Common kinds of GLModels
4-group ANOVA models y b0 b1x1 b2x2
b3x3 X1, X2 X3 are a dummy or effect
codes for a 4-group
variable
22
Common kinds of GLModels

• 2x2 Factorial ANOVA model
• y b0 b1x b2z b3xz

X is a dummy or effect code of 1st 2-group
variable Z is a dummy or effect code of 2nd
2-group variable XZ represents the interaction
of X and Z
23
Common kinds of GLModels

• 2x3 Factorial ANOVA model
• y b0 b1x1 b2z1 b3z2 b4xz1 b5xz2

X1 is a dummy or effect code of 1st 2-group
variable Z1 Z2 are dummy or effect codes of
2nd k-group variable XZ1 ZX2 represent the
interaction of X and Z
24
Common kinds of GLModels

• 2-group ANCOVA models
• y b0 b1x b2z
• X is a dummy or effect coded 2-group variable
• Z is the covariate (dummy coded or quantitative)

25
Common kinds of GLModels

• 2-group ANCOVA models with covariate interaction
• y b0 b1x b2z b3xz
• X is a dummy or effect coded 2-group variable
• Z is the covariate (dummy coded or
  quantitative)
• XZ represents the interaction of X and
  Z

26
Common kinds of GLModels

• 3-group ANCOVA models
• y b0 b1x1 b2x2 b3z
• X1 X2 are a dummy or effect codes for a
  3-group variable
• Z is the covariate (dummy coded or quantitative)

27
Common kinds of GLModels

• 3-group ANCOVA model with covariate interaction
• y b0 b1x1 b2x2 b3z b4xz1 b5xz2
• X1 X2 are a dummy or effect codes for a
  3-group variable
• Z is the covariate (dummy coded or
  quantitative)
• XZ1 ZX2 represent the interaction of
  X and Z

28
Common kinds of GLModels

• 2x2 Factorial ANCOVA model
• y b0 b1x b2z b3xz b4v

X is a dummy or effect code of 1st 2-group
variable Z is a dummy or effect code of 2nd
2-group variable XZ represents the interaction
of X and Z V represents the covariate
29
Common kinds of GLModels

• 2x2 Factorial ANCOVA model with covariate
  interactions
• y b0 b1x b2z b3xz b4v b5xv b6zv
  b7xzv

X is a dummy or effect code of 1st 2-group
variable Z is a dummy or effect code of 2nd
2-group variable XZ represents the interaction
of X and Z V represents the covariate XV
represents the interaction of X and V ZV
represents the interaction of Z and V XZV
represents the interaction of X, Z and V