VALUE-ADDED MODELS AND THE MEASUREMENT OF TEACHER QUALITY

1
VALUE-ADDED MODELSAND THE MEASUREMENT OF TEACHER
QUALITY

• Douglas Harris Tim R. Sass
• Dept. of Ed. Leadership Dept. of Economics
• and Policy Studies Florida State University
• Florida State University (tsass_at_fsu.edu)
• (harris_at_coe.fsu.edu)

IES Research Conference June 2006
2
Evaluating Value-Added Methodology

• The recent availability of panel data has
  produced a flood of research studies using
  various value-added approaches
• Research Questions
• Are assumptions underlying the value-added
  approach valid?
• Are some methods more likely to produce reliable
  estimates than others?
• What data are most important to obtaining
  consistent estimates?

3
Evaluating Value-Added Methodology

• Basic Model Types
• Cumulative Model
• Unrestricted Value-Added Model
• Value-Added Models with Persistence Restrictions
• Restricted Value-Added or Gain-Score Model
• Contemporaneous Model
• Specification Issues for Value-Added Models
• Treatment of teacher heterogeneity
• Measures of classroom/school inputs
• Treatment of student heterogeneity
• Aggregation

4
General Cumulative Model of Student Achievement
5
Basic Assumptionsof Value-Added Models

• Cumulative achievement function does not vary
  with age and is additively separable.
• Family inputs are time invariant.
• Parents do not compensate for poor school inputs
  or poor outcomes
• Todd and Wolpin (2005) reject exogeneity of
  parental inputs at 90 percent, but not at 95
  percent confidence level
• The marginal inputs of all school-based inputs,
  parental inputs, and the initial student
  endowment each decline geometrically (at
  potentially different rates) over time.
• Lagged achievement serves as a sufficient
  statistic for prior inputs
• We find twice-lagged inputs do not provide
  additional information

6
UnrestrictedValue-Added Model
7
Persistence Restrictions

• Restricted Value-Added or Gain-Score Model

• l is assumed to equal 1 (no decay in effect of
  past inputs)
• Alternatively, can interpret as an achievement
  growth model where growth is independent of past
  school inputs.

• Contemporaneous Model

• l is assumed to equal 0 (complete decay in effect
  of past inputs).

8
Decomposition of School-based Inputs in
Value-Added Model
9
Modeling Teacher Heterogeneity

• Substituting teacher time-invariant measured
  characteristics for teacher fixed effects

10
Classroom and School Inputs

• Exclusion of peer variables (P-ijmt)
• Number of peers (class size) and peer
  characteristics (gender, race, mobility, age)
• If peer variables are correlated with student and
  teacher characteristics (Xit and Tkt), omission
  will produce inconsistent estimates
• Exclusion of school fixed effects (fm)
• Given that teachers do not frequently change
  schools, omission of school effects will mean
  that teacher fixed effects will capture both
  teacher effects and some of the school effect,
  leading to inconsistent estimates

11
Modeling Student Heterogeneity

• Substituting measured time-invariant student
  characteristics for student fixed effects
• Race/ethnicity, foreign/native born, language
  parent speak at home, free-lunch status
• As with teachers, if unmeasured time-invariant
  student characteristics are correlated with
  independent variables, will get inconsistent
  estimates

12
Modeling Student Heterogeneity

• Fixed vs. random student effects
• Fixed effects allow for a separate intercept
  parameter for each student (equal to the mean
  error for that student) whereas random effects
  assume that the student-specific intercepts are
  drawn from a known distribution (typically
  normal)
• Since random effects are part of the error
  structure, they must be orthogonal to the model
  variables (Xit, P-ijmt, Tkt) in order to yield
  consistent estimates
• Given that fixed effects estimates are always
  consistent (whether or not unobserved student
  heterogeneity is correlated with other variables
  in the model), can test orthogonality assumption
  by applying a Hausman test
• Multilevel fixed effects models have been
  computationally burdensome

13
Aggregation

• Measuring characteristics of specific teachers
  vs. grade-level-within-school averages
• Since Texas data does not identify specific
  teacher, work by Rivkin, Hanushek and Kain (2005)
  relies on average characteristics of teachers
  within a grade
• Advantages/Disadvantages of aggregation
• Eliminates problems associated with non-random
  assignment of students to teachers within a
  school
• May reduce problem of measurement error since
  individual errors may cancel out at grade level
• May upwardly bias estimated impacts of school
  resources in the presence of omitted variables
• Tends to reduce precision of estimates

14
Data

• Floridas K-20 Education Data Warehouse
• Census of all children attending public schools
  in Florida
• Student records linked over time
• Covers 1995/1996 2003/2004 school years
• Includes student test scores and student
  demographic data, plus enrollment, attendance,
  disciplinary actions and participation in special
  education and limited English proficiency
  programs
• Includes all employee records including
  individual teacher characteristics and means of
  linking students and teachers to specific
  classrooms

15
Sample for Analysis

• Middle school students (grades 6-8) who took
  SSS-NRT (Stanford-9) math test in three
  consecutive years during 1999/2000 2003/2004
• Enrolled in a single math course in the Fall
• Up to 4 years of achievement gains
• 4 cohorts of students
• Includes a variety of math courses, from remedial
  to advanced and gifted classes
• Use random sample of 100 middle schools
• Reduces computational burden of estimating fixed
  effects
• Represents about 12 of middle schools in state

16
Value-Added Model EstimatesWith Varying Degrees
of Persistence
17
Correlation of Estimated Teacher Effects From
Models with Varying Degrees of Persistence
18
Restricted Value-Added Model EstimatesWith
Differing Controls for Teacher Heterogeneity
19
Restricted Value-Added Model EstimatesWith
Differing Classroom/School Controls
20
Correlation of Estimated Teacher Effects From
Models with Differing Classroom/School Controls
21
Restricted Value-Added Model Estimateswith
Differing Controls for Student Heterogeneity
22
Correlation of Estimated Teacher Effects From
Models With Differing Controls for Student
Heterogeneity
23
Restricted Value-Added Model Estimates
--Teacher-Specific vs. Within-School Grade-Level
Averages
24
Summary of Findings

• Model Selection
• Restricted value-added model seems to be a good
  approximation of the full cumulative model
• Specification
• Use of student and teacher fixed effects (rather
  than covariates) important
• Random effects may yield inconsistent estimates
• Important to include school fixed effects, but
  classroom peer variables relatively unimportant
• Aggregation to the grade level has some effect,
  though estimates not radically different from
  estimates with teacher-level data