Multilevel Modeling

1
Multilevel Modeling

• 1. Overview
• 2. Application 1 Growth Modeling
• Break
• 3. Application 2 Individuals Nested
  Within Groups
• 4. Questions?

2
Overview

• What is multilevel modeling?
• Examples of multilevel data structures
• Brief history
• Current applications
• Why multilevel modeling?
• What types of studies use multilevel modeling?
• Computer Programs (HLM 6
• SAS Mixed
• Resources

3
Multilevel Question

• What effects do the following variables have on
  3rd grade reading achievement?
• School Size
• Classroom Climate
• Student Gender

4
What is Multilevel or Hierarchical Linear
Modeling?

• Nested Data Structures

5
Several Types of Nesting

• 1. Individuals Nested Within Groups

6
Individuals Undivided
Unit of Analysis Individuals
7
Individuals Nested Within Groups
Unit of Analysis Individuals Classes
8
and Further Nested
Unit of Analysis Individuals Classes Schools
9
Examples of Multilevel Data Structures

• Neighborhoods are nested within communities
• Families are nested within neighborhoods
• Children are nested within families

10
Examples of Multilevel Data Structures

• Schools are nested within districts
• Classes are nested within schools
• Students are nested within classes

11
Multilevel Data Structures

• Level 4 District (l)
• Level 3 School (k)
• Level 2 Class (j)
• Level 1 Student (i)

12
2nd Type of Nesting

• Repeated Measures Nested Within Individuals
• Focus Change or Growth

13
Time Points Nested Within Individuals
14
Repeated Measures Nested Within Individuals

• Carlos
• Day Energy Level
• Monday 0 98
• Tuesday 1 90
• Wednes. 2 85
• Thursday 3 72
• Friday 4 70

15
Repeated Measures Nested Within Individuals
16
Repeated Measures Nested Within Individuals
17
Changes for 5 Individuals
18
3rd Type of Nesting (similar to the 2nd)

• Repeated Measures Nested Within Individuals
• Focus is not on change
• Focus in on relationships between variables
  within an individual

19
Repeated Measures Nested Within Individuals

• Carlos
• Day Hours of Sleep Energy Level
• Monday 9 98
• Tuesday 8 90
• Wednesday 8 85
• Thursday 6 72
• Friday 7 70

20
Repeated Measures Nested Within Individuals (Not
Change)
21
Repeated Measures Nested Within Individuals (Not
Change)
22
Repeated Measures Nested Within Individuals
23
Repeated Measures Within Persons

• Level 2 Student (i)
• Level 1 Repeated Measures
• Over Time (t)

24
Nested Data

• Data nested within a group tend to be more alike
  than data from individuals selected at random.
• Nature of group dynamics will tend to exert an
  effect on individuals.

25
Nested Data

• Intraclass correlation (ICC) provides a measure
  of the clustering and dependence of the data
• 0 (very independent) to 1.0 (very dependent)
• Details discussed later

26
Brief Historyof Multilevel Modeling

• Robinson, W. S. (1950). Ecological correlations
  and the behavior of individuals. Sociological
  Review, 15, 351-357.
• Burstein, Leigh (1976). The use of data from
  groups for inferences about individuals in
  educational research. Doctoral Dissertation,
  Stanford University.

27
Table 1
Frequency of HLM application evidenced in
Scholarly Journals
Journal 1999 2000 2001 2002 2003 Total by journal
American Educational Research Journal 3 5 4 3 ? 15
Child Development 3 2 6 5 13 29
Cognition and Instruction 1 0 0 0 0 1
Contemporary Educational Psychology 0 0 0 0 0 0
Developmental Psychology 2 1 2 5 7 17
Educational Evaluation and Policy Analysis 2 1 5 2 2 12
Educational Technology, Research and Development 0 0 0 0 0 0
Journal of Applied Psychology 1 1 5 7 6 20
Journal of Counseling Psychology 0 2 1 0 0 3
Journal of Educational Computing Research 0 0 0 0 0 0
Journal of Educational Psychology 1 2 3 6 1 13
Journal of Educational Research 2 0 3 3 5 13
Journal of Experimental Child Psychology 0 0 0 0 0 0
Journal of Experimental Education 0 0 0 0 1 1
Journal of Personality and Social Psychology 4 4 6 5 13 32
Journal of Reading Behavior/Literacy Research 0 0 0 0 0 0
Journal of Research in Mathematics Education 0 0 0 0 0 0
Reading Research Quarterly 0 0 0 1 0 1
Sociology of Education 1 2 5 2 1 11
Total by Year 20 20 40 39 49 168
28
Multilevel Articles
29
Some Current Applications of Multilevel Modeling

• Growth Curve Analysis
• Value Added Modeling of Teacher and School
  Effects
• Meta-Analysis

30
Multilevel Modeling Seems New But.

• Extension of General Linear Modeling
• Simple Linear Regression
• Multiple Linear Regression
• ANOVA
• ANCOVA
• Repeated Measures ANOVA

31
Multilevel Modeling

• Our focus will be on observed variables (not
  Latent Variables as in Structural Equation
  Modeling)

32
Why Multilevel Modelingvs. Traditional
Approaches?

• Traditional Approaches 1-Level
• Individual level analysis (ignore group)
• Group level analysis (aggregate data and ignore
  individuals)

33
Problems withTraditional Approaches

• Individual level analysis (ignore group)
• Violation of independence of data assumption
  leading to misestimated standard errors (standard
  errors are smaller than they should be).

34
Problems withTraditional Approaches

• Group level analysis
• (aggregate data and ignore individuals)
• Aggregation bias the meaning of a variable at
  Level-1 (e.g., individual level SES) may not be
  the same as the meaning at Level-2 (e.g., school
  level SES)

35
Multilevel Approach

• 2 or more levels can be considered simultaneously
• Can analyze within- and between-group variability

36
What Types of Studies Use Multilevel Modeling?

• Quantitative
• Experimental
• Nonexperimental
• (Survey, Observational)

37
How Many Levels Are Usually Examined?

• 2 or 3 levels very common
• 15 students x 10 classes x 10 schools
• 1,500

38
Types of Outcomes

• Continuous Scale (Achievement, Attitudes)
• Binary (pass/fail)
• Categorical with 3 categories

39
Software to do Multilevel Modeling

• SPSS Users
• 2 SAV Files Level 1
• Level 2
• HLM 6 (Menu Driven)
• (Raudenbush, Bryk, Cheong, Congdon, 2004)

40
HLM 6
41
Software to do Multilevel Modeling

• SAS Users
• Proc Mixed

42
Resources (Samplesee handouts for more complete
list)

• Books
• Hierarchical Linear Models Applications and
  Data Analysis Methods, 2nd ed. Raudenbush
  Bryk, 2002.
• Introducing Multilevel Modeling.
• Kreft DeLeeum, 1998.
• Journals
• Educational and Psychological Measurement
• Journal of Educational and Behavioral Sciences
• Multilevel Modeling Newsletter

43
Resources (cont)(Samplesee handouts for more
complete list)

• Software
• HLM6
• SAS (NLMIXED and PROC MIXED)
• MLwiN
• Journal Articles
• See Handouts for various methodological and
  applied articles
• Data Sets
• NAEP Data
• NELS88 High School and Beyond

44
Self-Check 1

• A teacher with 1 classroom of 24 students used
  weekly curriculum-based measurements to monitor
  reading over a 14 week period. The teacher was
  interested in individual students rates of
  change and differences in change by male and
  female students.

45
Self-Check 1

• How would you classify this situation?
• (a) not multilevel
• (b) 2-level
• (c) 3-level

46
Self-Check 2

• A researcher randomly selected 50 elementary
  schools and randomly selected 30 teachers within
  each school. The researcher was interested in
  the relationships between 2 predictors (school
  size and teachers years experience at their
  current school) and teachers job satisfaction.

47
Self-Check 2

• How would you classify this situation?
• (a) not multilevel
• (b) 2-level
• (c) 3-level

48
Self-Check 3

• 60 undergraduates from the research participant
  pool volunteered for a study that used written
  vignettes to manipulate the interactional style
  (warm, not warm) of a professor interacting with
  a student.  30 randomly assigned students read
  the vignette depicting warmth and 30 randomly
  assigned students read the vignette depicting a
  lack of warmth.  After reading the vignette
  students used a questionnaire to rate the
  likeability of the professor.

49
Self-Check 3

• How would you classify this situation?
• (a) not multilevel
• (b) 2-level
• (c) 3-level

(Select ONLY one)
50
Growth Curve Modeling

• Studying the growth in reading achievement over a
  two year period
• Studying changes in student attitudes over the
  middle school years

51
Research Questions

• What is the form of change for an individual
  during the study?

52
Research Questions

• What is an individuals initial status on the
  outcome of interest?

53
Research Questions

• How much does an individual change during the
  course of the study?

Rise
Run
54
Research Questions

• What is the average initial status of the
  participants?

55
Research Questions

• What is the average change of the participants?

56
Research Questions

• To what extent do participants vary in their
  initial status?

57
Research Questions

• To what extent do participants vary in their
  growth?

58
Research Questions

• To what extent does initial status relate to
  growth?

59
Research Questions

• To what extent is initial status related to
  predictors of interest?

60
Research Questions

• To what extent is growth related to predictors of
  interest?

61
Design Issues

• How many waves a data collection are needed?
• gt2
• Depends on complexity of growth curve

62
Design Issues

• Can there be different numbers of observations
  for different participants?
• Examples
• Missing data
• Planned missingness

63
Design Issues

• Can the time between observations vary from
  participant to participant?
• Example Students observed
• 1, 3, 5, 7 months
• 1, 2, 4, 8 months
• 2, 4, 6, 8 months

64
Design Issues

• How many participants are needed?
• More is better
• Power analyses
• gt 30 rule of thumb

65
Design Issues

• How should participants be sampled?
• What you have learned about sampling still
  applies

66
Design Issues

• What is the value of random assignment?
• What you have leaned about random assignment
  still applies

67
Design Issues

• How should the outcome be measured?
• What you have learned about measurement still
  applies

68
Example

• Context description
• A researcher was interested in changes in verbal
  fluency of 4th grade students, and differences in
  the changes between boys and girls.

69

• ID    Gender         Time______
•            
•                   t0    t4    t7
• 1    0  20    30    30
• 2     0          40    44    49
• 3 0          45    40    60
• 4     0         50    55    59
• 5     0          42    48    53
• 6 1          45    52    61
• 7 1          39    55    63
• 8 1          46    58    68
• 9 1          44    49    59

70
Example

• Level-1 model specification

71
Example

• Level-2 model specification

72
Example

• Combined Model

73
Example

• SAS program
• proc mixed covtest
• class gender
• model score time gender timegender/s
• random intercept / substudent s

74
Example

• SAS output variance estimates

Covariance Parameter
Estimates  
Standard Z Cov Parm Subject Estimate
Error Value Pr Z   Intercept Student
62.5125 35.9682 1.74 0.0411 Residual
14.1173 4.9912 2.83 0.0023
75
Example

• SAS output fixed effects

Solution for Fixed Effects  
Standard Effect Gender Estimate
Error DF t Value Pr gt t   Intercept
39.8103 3.7975 7 10.48 lt.0001 time
1.5077 0.3295 16 4.58
0.0003 Gender F 5.7090 5.6962 16
1.00 0.3311 Gender M 0
. . . . timeGender F 1.0692
0.4943 16 2.16 0.0460 timeGender M
0 . . . .
76
Example

• Graph fixed effects

77
Example

• Conclusions
• Fourth grade girls verbal fluency is increasing
  at a faster rate than boys.

78
Persons Nested in Contexts

• Studying attitudes of teachers who are nested in
  schools
• Studying achievement for students who are nested
  in classrooms that are nested in schools

79
Research Questions

• How much variation occurs within and among
  groups?
• To what extent do teacher attitudes vary within
  schools?
• To what extent does the average teacher attitude
  vary among schools?

80
Research Questions

• What is the relationship among selected within
  group factors and an outcome?
• To what extent do teacher attitudes vary within
  schools as function of years experience?
• To what extent does student achievement vary
  within schools as a function of SES?

81
Research Questions

• What is the relationship among selected between
  group factors and an outcome?
• To what extent do teacher attitudes vary across
  schools as function of principal leadership
  style?
• To what extent does student math achievement vary
  across schools as a function of the school
  adopted curriculum?

82
Research Questions

• To what extent is the relationship among selected
  within group factors and an outcome moderated by
  a between group factor?
• To what extent does the within schools
  relationship between student achievement and SES
  depend on the school adopted curriculum?

83
Design Issues

• Consider a design where students are nested in
  schools
• How should schools should be sampled?
• How should students be sampled within schools?

84
Design Issues

• Consider a design where students are nested in
  schools
• How many schools should be sampled?
• How many students should be sampled per school?

85
Design Issues

• What kind of outcomes can be considered?
• Continuous
• Binary
• Count
• Ordinal

86
Design Issues

• How will level-1 variables be conceptualized and
  measured?
• SES
• How will level-2 variables be conceptualized and
  measured?
• SES

87
Terminology

• Individual growth trajectory individual growth
  curve model
• A model describing the change process for an
  individual
• Intercept
• Predicted value of an individuals status at some
  fixed point
• The intercept cold represent the status at the
  beginning of a study
• Slope
• The average amount of change in the outcome for
  every 1 unit change in time

88
intercept
89
(No Transcript)
90
HLM

• Hierarchical Linear Model
• The hierarchical or nested structure of the data
• For growth curve models, the repeated measures
  are nested within each individual

91
Levels in Multilevel Models

• Level 1 time-series data nested within an
  individual

92
Levels in Multilevel Models

• Level 2 model that attempts to explain the
  variation in the level 1 parameters

93
More terminology

• Fixed coefficient
• A regression coefficient that does not vary
  across individuals
• Random coefficient
• A regression coefficient that does vary across
  individuals

94
More terminology

• Balanced design
• Equal number of observations per unit
• Unbalanced design
• Unequal number of observation per unit
• Unconditional model
• Simplest level 2 model no predictors of the
  level 1 parameters (e.g., intercept and slope)
• Conditional model
• Level 2 model contains predictors of level 1
  parameters

95
Estimation Methods

• Empirical Bayes (EB) estimate
• optimal composite of an estimate based on the
  data from that individual and an estimate based
  on data from other similar individuals (Bryk,
  Raudenbush, Condon, 1994, p.4)

96
Estimation Methods

• Expectation-maximization (EM) algorithm
• An iterative numerical algorithm for producing
  maximum likelihood estimates of variance
  covariance components for unbalanced data.