Teacher Views of Merit Pay: Lessons from the Achievement Challenge Pilot Project

1

2
Presentation Overview

• ACPP Description
• Teacher Views Research Questions
• Teacher Surveys Methods Results
• Teacher Interviews Methods Results
• Bonus Amounts Awarded
• Teacher Feedback
• Conclusions

3
Why try merit pay?

• Of all the inputs we can control, teacher quality
  matters most for student success.
• a good teacher will get a gain of 1.5 grade
  level equivalents while a bad teacher will get .5
  year for a single academic year.
• having five years of good teachers in a row
  (1.0 standard deviation above average, or at the
  85th percentile) could overcome the average
  seventh-grade mathematics achievement gap between
  lower-income kids (those on the free or
  reduced-price lunch program) and those from
  higher-income families.
• Hanushek, E.A. Rivkin, S.G. (2004). Similar
  findings by Sanders, W.L. Rivers, J.C. (1996)

4
Compensation Reform as Policy Tool
The Single Salary Schedule which rewards teachers
for credential and experience does not align pay
with the outcome we care about most Student
Achievement. Teacher compensation reform
focused on pay-for-performance can have
motivational (short term) and compositional
effects (long range)
5
Achievement Challenge Pilot Project Description
(ACPP)

• ACPP - Little Rock, Arkansas
• Merit pay program for all staff members in a
  school
• Reward personnel solely for increases in student
  achievement
• 3-Year Program
• Created in 2004-05 at Meadowcliff Elementary
• Expanded to include Wakefield Elementary, 2005-06
• Expanded to include 3 more schools (Romine,
  Mabelvale, Geyer Springs), 2006-07

6
ACPP Well Designed Plan

• Straightforward Based on growth, not
  complicated statistics
• Non-Competitive for Teachers Everyone could
  earn
• Significant Awards - up to 10,000 for teachers
• Focus on Growth of Students address student
  placement
• Everyone Participated from secretaries to
  principals

7
Achievement GAINS! BUT An Unqualified
Success???

• Evaluation Focus
• Two years of modest positive student gains
• (Winters et al., 2008) (Barnett et al., 2007)
• Should we think about other costs and benefits
  when evaluating a program?
• What can we learn from implementation?

8
The Hypotheses We Tested Proposed Benefits

• Possible advantages of merit pay
• Leads to greater innovation in teaching
• Leads to teachers working harder
• Leads to greater salary satisfaction

9
The Hypotheses We Tested Proposed Drawbacks

• Possible disadvantages of merit pay
• 1) Leads to a degraded school climate
• 2) Leads to an increase in counterproductive
  competition
• 3) Leads to the neglect of low-performing
  students

10
Teacher Views Research Questions

• Teacher Surveys (Compelling Control Groups)
• Do teachers in schools with a long-term merit pay
  (IA) program have differing attitudes and
  perceptions than teachers in new merit pay
  schools (LA)?
• Do the attitudes and perceptions of teachers in
  schools with a long-term merit pay program (IA)
  change over time?
• Teacher Interview
• Were teachers supportive of merit pay and the
  ACPP, and did their attitudes and behaviors
  change as a result of the merit pay program?
• Did teachers feel the goals of the ACPP were fair
  and attainable, and how would they improve the
  program?

11
Teacher Surveys Methods

• Surveys were administered to teachers in Fall 06
  Spring 07 at all five participating schools
• Survey participation was voluntary
• Teachers responses were measured on a scale of 1
  4, with a higher score representing stronger
  levels of agreement with that question
• Questions were grouped into constructs that
  examined attitudes about oft-cited positives and
  negative associated with merit pay

12
Teacher SurveysComparison of IA vs. LA Fall
2006
RQ1 Do teachers in schools with a long-term
merit pay (IA) program have differing attitudes
and perceptions than teachers in new merit pay
schools (LA)?




13
Teacher Surveys Comparison of IA vs. IA Fall
2006 to Spring 2007
RQ2 Do the attitudes and perceptions of teachers
in schools with a long-term merit pay program
(IA) change over time? ATTITUDES PERSISTED
AFTER FIRST YEAR
14
Synthesis of Survey Findings
15
Teacher Interviews Methods

• Conducted in Fall 2007 after bonuses were
  disbursed
• Questions explored the same areas as the surveys
  
• Teacher salary satisfaction was not covered in
  the interviews
• 43 interviews were conducted (40 teachers, 3
  principals) at the 5 ACPP schools
• Responses were coded as positive, negative, or
  neutral
• A positive response would advocate for
  performance pay
• Ex. Do teachers work harder at your school as a
  result of merit pay? Yes positive response
• No negative response
• No answer neutral response
• 21 coded as positive, 16 were negative, 6 were
  neutral

16
Teacher Interviews Results
RQ3 Were teachers supportive of merit pay and
the ACPP, and did their attitudes and behaviors
change as a result of the merit pay program?
RQ4 Did teachers feel the goals of the ACPP
were fair and attainable, and how would they
improve the program? Highlighted are different
findings.
Not evaluated on the survey
17
Teacher Bonus Amounts 2004-2007
Other than 0
18
Ways to Improve the Program Teacher Responses

• I think it should have been consistent across
  the board. The rules changed.
• I dont know if it was a lack of understanding,
  miscommunication, or it was literally changed
  after we were told that it would be this way, and
  then it wasnt done that way. It was done a
  different way. That caused a lot of conflict.
• I like the pretesting in the Fall when they the
  students got here. I like the post-testing at
  the end of the year because that told me how good
  of a teacheror what I needed to work on.
• I believe that no teacher should have gone
  without a check.
• They teachers would have to be in on the
  planning/making. And you would have the exact
  same discussion with themwhat do you think? What
  do we want? And they would have to have so much
  ownership of it.
• Have one set of rules for the entire district.
  Everything has to be transparent. Our situation
  wasnt transparent.

19
Conclusions Teacher Views of Merit Pay

• Teachers initially supported the ACPP and were
  somewhat supportive of merit pay as a
  compensation reform over time.
• Teachers did not associate merit pay with being
  more innovative, and responses varied in regards
  to working harder.
• Teachers did not associate merit pay with
  counterproductive competition or viewing
  low-performing students as a burden, but did feel
  that the school environment became more negative
  after bonuses were disbursed.
• Teachers reported higher salary satisfaction and
  felt more effective.
• Teacher expectations play a significant role in
  the success of a merit pay program.

20
Take away message

• Merit pay that ties bonuses directly to student
  achievement gains can lead to student
  improvement.
• Merit pay does not necessarily lead to divisive
  competition and nor undermine collaboration.
• Participant expectations are very important, so
  implementers must communicate clearly and openly
  to set and maintain expectations for
  participants.

21

• Contact InformationMarc HolleyOffice for
  Education PolicyUniversity of Arkansashttp//www
  .uark.edu/ua/oepEmail mjholley_at_uark.eduPhone
  [phone]