A Latent Variable Model for Classroom Observations

1
A Latent Variable Model for Classroom Observations
preliminary set of
s
V

• Lee Branum-Martin, Christopher Barr, Coleen D.
  Carlson, Angie Durand, Sarah Garnaat

2
Classroom Observations Let us count the ways
Target behavior, environment Measurement
amount, quality global specific Source self,
observer, participant Methods live/in-situ,
retrospective, recording surveys, protocols,
narratives, interviews structure open
closed All methods involve the balance between
validity and reliability, fidelity and
objectivity, accuracy and bandwidth.
We trained 44 master teachers as observers to
rate the quality of instruction and to mark the
occurrence of specific instructional behaviors at
118 campuses.
3
Instruments
Quality Ratings poor/mediocre/good Management
(2) Climate (2) Oral Language (4) Reading
Instruction (3) Writing (4) Assessment
(2) Curriculum Integration (1) Judged quality
for grade level. Single model of items measuring
each subscale to indicate dimensions of
grade-appropriate quality.
Behavioral Checklist present/absent PA Letters
(9) Word Structure(3) Word Recognition(3) Spelling
(9) Comprehension (14) Vocabulary (10) Simple
presence/absence. Grade-specific models of items
measuring each subscale unidimensional , then
multi-dimensional
4
Current Sample
1,225 teachers were randomly sampled to be
observed, producing a total of 1,650
observations.
The sample was 94 female, with an average of 11
years of teaching experience (SD10 years). 50
had 6 or fewer years of service.
Quality model ?279 523.2, CFI .96, RMSEA
.058
5
Quality Ratings Model Results
Manage
.82
.90
Climate
.92
.90
Oral Language
.54
.84
.86
.80
6
Reading
.75
.87
.72
Writing
.75
.75
.73
.84
Assessment
.81
.94
Curriculum

7
Quality Factor Correlations Means
8
Quality Ratings Implications
Within year, K-1 teachers teach better. Second
grade shows less change, and third grade is
stable. Across grades, third grade teachers are
judged to teach manage their classrooms
better. Items measure factors well, but may not
distinguish well among above-average
teachers. Factor correlations suggest high
consistency among aspects of quality
instruction. Ratings might not be truly
grade-specific, or early grade classrooms might
not have as much opportunity to demonstrate
higher quality behaviors (e.g., situations not as
complex).
9
Instruments
Quality Ratings poor/mediocre/good Management
(2) Climate (2) Oral Language (4) Reading
Instruction (3) Writing (4) Assessment
(2) Curriculum Integration (1) Judged quality
for grade level. Single model of items measuring
each subscale to indicate dimensions of
grade-appropriate quality.
Behavioral Checklist present/absent PA Letters
(9) Word Structure(3) Word Recognition(3) Spelling
(9) Comprehension (14) Vocabulary (10) Simple
presence/absence. Grade-specific models of items
measuring each subscale unidimensional , then
multi-dimensional
10
Correlation(PA, Syllables) .47
.74
.88
.90
.51
.93
.93
.92
.86
.32
Loadings
(correlation) .78
Syllables
11
Correlation (Structure, Recognition) .19
.77
.89
.88
.96
.58
.76
Loadings
Structure
Recognition
12
Correlation(Spelling, Blending) .82
.63
.77
.78
.78
.71
.53
.55
.52
.45
Loadings
Blending
13
not used in Kinder model
.79
.46
.70
.68
.46
.29
.43
.46
Loadings
14
.70
.77
.72
.55
.58
.59
.53
.63
Loadings
.68
.65
.80
.63
.81
.64
Knowledge
Strategy
Inference
15
Behavioral Checklist Implications
Factors reflect systematic approaches to
instruction. Profiles match grade-level
instructional expectations e.g., less PA
instruction for older children, more
comprehension instruction. Little evidence of
differential dimensionality across
grades. Multiple-group analysis might aid in
testing what aspects of the factors differ across
grades (e.g., means, variances,
covariances). Secondary analysis of the factor
scores might show higher-order organization of
instruction. Such organization might differ by
gradee.g., spelling might be used with PA, but
not with comprehension. How does instruction
relate to student outcomes?