Teacher Effectiveness in Urban Schools

1
Teacher Effectiveness in Urban Schools

• Richard Buddin Gema Zamarro
• IES Research Conference, June 2010

2
Research Questions

• How does teacher quality vary across classrooms
  and across schools?
• How do teacher qualifications affect teacher
  effectiveness?
• Traditional measures like experience and
  education
• Teacher licensure test scores
• Are the best teachers concentrated in a few
  schools?
• Are teachers with single-subject credentials more
  effective in middle school than teacher with
  multi-subject credentials?
• Does the quality of high school ELA instruction
  spill over into math and visa versa?

3
Focus on Los Angeles Unified School District

• LAUSD is 2nd largest district in the US
• 690,000 students
• 36,000 teachers
• 800 schools
• Eight years worth of student achievement data for
  students in grades 2 through 11
• Achievement scores in reading and math each year
• Students records linked longitudinally
• Teachers are matched with individual students
• Teacher licensure tests scores as well as
  traditional teacher credentials like experience
  and educational background

4
Teacher Licensure Test Scores

• Like other states, California relies on expert
  panels of teachers, administrators, and content
  experts to develop tests and set cut scores
• California requires passing scores on three tests
  as part of teacher certification
• General aptitude
• Reading pedagogy (elementary only)
• Subject matter knowledge
• Credentialing commission receives licensure
  scores directly, but they are not reported to
  districts or teaching candidates
• potential teachers know whether they pass or not
• districts know if a candidate is certified or not

5
How Do Teaching Candidates Perform on California
Licensure Tests?
6
Teacher Pass Rates Rise with College Grades
7
Empirical Issues

• Estimation methods control for prior learning and
  isolate the contribution of current teacher and
  school inputs to student learning
• No ideal methodmore flexible models require more
  years data and involve restrictions that may
  offset benefits
• We used three types of models
• Contemporaneous value-added student achievement
  is a function of student and teacher fixed
  effects
• Value-added gains Similar but uses year-to-year
  gains as dependent variable
• Lagged achievement Current student achievement
  as a function of prior year achievement

Similar results for all models for elementary,
middle, and high schoolsfocus on elementary here
8
Low Scoring Schools Have Much Different Mix of
Students and Teachers than High Scoring Schools
 Characteristic Lowest Quartile Schools Highest Quartile Schools
Student Characteristics
Reading Percentile 34.10 53.66
Math Percentile 40.79 62.31
Black 0.15 0.10
Hispanic 0.83 0.36
LEP 0.64 0.20
Parents not high school graduates 0.47 0.11
Teacher Characteristics
Years of Experience 6.36 9.37
Experience lt 3 yrs 0.44 0.30
Black 0.21 0.08
Hispanic 0.37 0.14
Master's/Doctorate 0.16 0.23
General Ability (standardized) -0.52 -0.08
Subject Matter (standardized) -0.43 0.06
Reading Pedagogy (standardized) -0.31 -0.01

• More minority, English learner, and low income
  students in low scoring schools
• Weaker teacher credentials in low scoring schools
• Less experience
• Fewer advanced degrees
• Lower licensure scores
• In past two years, LAUSD has cut young teaching
  staff
• With fewer young teachers, most of the gap in
  teacher characteristics has disappeared
• Unclear how this will affect achievement

9
Teacher Effects Are Much Larger than School
Effects
  Reading Math
1. Student Teacher Fixed Effects    
Student (?Student) 16.75 18.33
Teacher (?Teacher) 4.99 6.25
2. Student School Fixed Effects    
Student (?Student) 16.97 18.69
School (?School) 2.15 2.57
High quality teachers are spread across schools
and not concentrated in a few schools
10
Teacher Quality Has A Large Effect on Student
Test Scores, After Controlling for Student
Characteristics

• Teacher quality distribution has a large
  interquartile range in reading and math
• Students with a teacher at the 25th percentile of
  the teacher quality distribution score 9 to 12
  percentile points lower than do similar students
  assigned to a teacher at the 75th percentile
• Some teachers get much better classroom results
  than others

11
Do Traditional Measures of Teacher Qualifications
Predict Classroom Performance?

• Teacher experience is weakly related to student
  achievement
• A five year increase in experience increases
  achievement by less than one percentage point
• Teacher education level has no effect on student
  achievement
• Also, little evidence that student/teacher match
  affects classroom achievement
• Race/ethnic match
• Gender match
• High income students with better educated teacher

12
Do Higher Scoring Teachers Have Better Student
Achievement in Their Classrooms?

• Licensure scores are unrelated to student
  achievement
• General aptitude, subject-matter, and reading
  pedagogy score have no significant effects on
  student achievement
• Also looked at scores for each test separately
  and different tests at different grades, but the
  scores did not matter in either reading or math
• Examined whether teachers who initially failed
  licensure tests had lower student achievement in
  their classrooms
• Initial failures were also insignificantly linked
  to achievement scores

13
Limitations of Licensure Test Results

• Licensure tests and teacher performance are
  available only for teachers who pass the tests
• Potential teachers who fall below the cut scores
  on the licensure tests might have worse classroom
  outcomes than teachers who ultimately surpass
  those cut scores
• Raising the cut scores of current tests would not
  improve student achievement
• Stricter licensure screens might do a better job
  of differentiating teachers
• Tighter screens might significantly reduce the
  pool of available teachers and discourage
  qualified individuals from pursuing teaching
  careers
• Current licensure results provide no indication
  of how well teachers will do in the classroom

14
Special Findings from Middle and High School
Analysis

• Middle school results
• Teachers with elementary school credentials had
  slightly better success than did teachers with
  secondary school credentials
• Higher general knowledge or subject-matter skill
  (as measured on licensure exam) did not affect
  classroom effectiveness
• High school results
• Achievement outcomes differ substantially from
  teacher to teacher, and the effects of a good
  teacher spill over from one subject to another
• Even here, effectiveness was unrelated to
  licensure scores

15
How does teacher quality vary across classrooms
and across schools?

• Considerable variation in teacher quality across
  classrooms and less variation across schools
• Traditional teacher quality measures and
  licensure test scores explain little of the
  learning gap between these schools
• The teacher quality gap between low- and high-
  scoring schools is only about one percentage
  point
• Efforts to improve the teaching performance in
  low performing schools are unlikely to succeed,
  if they rely entirely on traditional measures of
  teacher quality
• Reshuffling teachers with better qualifications
  to low performing schools is unlikely to close
  the achievement gap

16
Final Thought

• Teacher quality measures may be weakly tied to
  student achievement because teacher effort is
  inversely related to those measures
• Perhaps more experienced or smarter teachers are
  better able to teach, but they do not
  consistently perform in the classroom
• Perhaps the current compensation system provides
  too little incentive for the best teachers to
  deliver their best performance in the classroom
  on a consistent basis