Title: Measuring the Link between Elementary Teachers and Student Achievement A Presentation of the Dissert
1 Measuring the Link between Elementary Teachers
and Student AchievementA Presentation of the
DissertationElementary Teachers and the
Mathematics Achievement of Urban Students
- Alan Spicciati, Ed.D.
- Seattle Pacific University, Class of 2008
- spicciad_at_hsd401.org
2Research Findings on the Variability of Student
Achievement by Teacher
- The difference between teachers one SD above and
below the mean is one years worth of achievement
(Hanushek, 1992) - Teacher effects are cumulative three years with
top vs. bottom quintile teachers opens a 54
percentile gap (Sanders Rivers, 1996) - Rowan, Correnti, Millers (2002) comprehensive
study of teacher measurement methodology
concluded 52-72 of student mathematics variance
lies between classrooms, with the rest between
students and between schools. - A one SD increase in teacher effectiveness is
equal to a reduction in class size from 25 to 15
(Nye, Konstantopoulos, Hedges, 2004) -
3Important Findings on Teacher Characteristics
- Experience
- Experience has a curvilinear relationship with
achievement. - Achievement rises with experience for between 2
and 5 years, with on-the-job training, then
levels off (Ferguson, 1991 Darling-Hammond,
2000 Rockoff, 2004 Rivkin, Hanushek, Kane,
2005).
4Important Findings on Teacher Characteristics
- Advanced Degrees
- Masters degrees are important in mathematics and
science in secondary (Goldhaber Brewer, 1997
Wenglinsky, 2000). - Findings on advanced degrees are split for
elementary. - Many studies find that advanced degrees do not
relate to elementary mathematics
achievement...(Hanushek, 1986 Rivkin, Hanushek,
Kain, 2005 Clotfelter, Ladd, Vigdor, 2007). - However, some reputable studies find a positive,
significant relationship (Ferguson Ladd, 1996
Greenwald, Hedges, Laine, 1996 Nye,
Konstantopolous, Hedges, 2004).
5Important Findings on Teacher Characteristics
- College Selectivity
- A teachers academic ability, particularly verbal
ability, is among the most established teacher
variables in relation to student achievement
(Hanushek, 1986 Rice, 2003). - College selectivity, often measured by Barrons
rankings, is a proxy for academic ability that is
moderately related to student achievement (Wayne
Youngs, 2003).
6Important Findings on Teacher Characteristics
- Mathematics Courses
- Mathematics content knowledge, as measured by
tests of teachers, relates to achievement
(Harbison Hanushek, 1992 Hill, Rowan, Ball,
2005). - Mathematics courses relate to math achievement in
secondary (Monk King, 1994). - However, Hill, Rowan, Ball (2005) found there
is little empirical evidence examining math
courses and achievement at the elementary level,
and their findings were not significant.
7Definition
- Teacher effectiveness. The present study is
focused on teachers, as opposed to teaching.
In this context, teacher effectiveness is
defined by the mathematics achievement of a
teachers students, as measured by growth on the
Measures of Academic Progress (MAP) test,
compared to expected growth. While teacher
effectiveness is a term used in the literature,
this will be a correlational study and will not
imply effects.
8Research Questions
- In terms of descriptive statistics, what is the
distribution of achievement growth at the
classroom level? - Is there a significant relationship between
advanced degrees, experience, college
selectivity, or total mathematics courses taken
at the university level and growth in mathematics
achievement? - What combinations of the above teacher variables
best explain the variance in student growth? - Since poor and minority communities generally
attract and retain less qualified and experienced
teachers than other communities, would the
achievement of diverse classes be significantly
higher if they had equal or even equitable access
to teachers with experience and advanced degrees?
9Participants
- 3,558 students
- 70.7 of all students in grades 3-6
- 84.2 of all students with complete scores,
excluding self-contained classes - 156 teachers
- 68.7 of all teachers in grades 3-6
- 89.7 of all eligible teachers
- Required teacher variable data was located for
all teachers
10Instrument
- Measures of Academic Progress (MAP)
- Published by Northwest Evaluation Association
(NWEA) - Computer adaptive item response theory
- Multiple choice typically 40 items
- Measures the content strands found on the math
WASL - Administered fall, winter, and spring
- Reliability and Validity
- Test-retest reliability r .88 to r .93
- Marginal reliability r .94
- Concurrent validity (with state tests) r .79
to r .89
11Procedures
- Permission granted by superintendent and SPU
Institutional Review Board - Gathered existing data
- MAP scores accessed in raw format from district
database - Teacher data accessed from Human Resources
- Degree database contained universities and
degrees - Highly Qualified Teacher database contained
record of course taking - Samples double checked against actual transcripts
12Variables
- Independent Variables
- Demographic
- Class Percent Non White (CPNW)
- School Free and Reduced Lunch Percentage (SFRL)
- Class Percent of English Language Learners (CPEL)
- Teacher
- Experience (EXP)
- Experience Dichotomized (EXPDI)
- Degree (DEGR)
- College Selectivity (COLL)
- Number of Mathematics Courses, Content and
Pedagogy (MC) - Math Courses Dichotomized (MCDI)
- Dependent Variable
- Class Percent of Expected Growth (CPEG)
- Fall to spring student level MAP growth, divided
by NWEA expected (normal) growth, aggregated to
class level
13Statistical Procedures
- Descriptive Statistics
- Overall
- Disaggregated by quartile level of diversity
- Correlation
- Multiple Regression
- Identification of best model for this dataset
- Regression equation used to estimate results with
various staffing scenarios
14Descriptive Statistics
15Means of variables, disaggregated by class
percent non-white (CPNW) quartile
- Performance
- Least diverse quartile grew most
- Demographics
- Poverty and ELL highly related to diversity
- Teachers
- Low diversity classes taught by more experienced
teachers - Other variables have weaker relationships
16Scatter of Classrooms by Diversity Level and CPEG
- Diversity level only explains 9 of growth
- Large range of growth at every level of diversity
- Many highly diverse classes outperform expected
growth
17Performance by diversity quartile, and growth
quartile within diversity quartile
- Explanation
- Each color represents a diversity quartile
- Each bar represents 9 or 10 classrooms, grouped
by growth, with average CPEG shown - Interpretation
- Top classrooms in every diversity quartile
outperform the average non diverse class
18Intercorrelation of demographic, teacher, and
classroom achievement variables
19Multivariate linear regression, preliminary/full
model
- Diversity explains 9 of CPEG scores
- Advanced degrees and experience explain an
additional approximately 9 - Experience does not significantly explain CPEG
scores beyond advanced degrees
20Multivariate linear regression, reduced model
- The reduced model includes only diversity and
advanced degrees - Advanced degrees explain more than 9 of variance
in CPEG scores beyond what diversity explains - The model as a whole explains about 18 of the
variance in CPEG scores
21Estimated achievement based on various scenarios
of teacher assignment
- Explanation
- Using Beta weights from the multiple regression
equation, achievement levels are simulated using
different allocation methods - Interpretation
- An equitable approach could close achievement gap
between Q1 and Q4 from 21 (in the status quo
model) to 7 - See limitations
- This approach would be more powerful with a
stronger measure of teacher quality or a
characteristic that varies more greatly across
schools
22Discussion
- Q1 Distribution of Achievement by Classroom
- 1 SD of classroom effectiveness nearly 3 months
of growth - Q2 Teacher Characteristics and Math Achievement
Growth - Advanced degrees
- Different findings may be attributable to small
n of colleges, local bargaining context, or
methodology that cannot link individual teachers
with their characteristics. - Experience
- Findings herein consistent with research.
- College selectivity
- Data lacks variability to show results.
- Mathematics courses
- Content knowledge appears to matter, based on
Hill, Rowan, Ball (2005), but coursework is a
poor proxy. - Q3 Combinations of Variables
- No significant interactions.
- Q4 Equal or Equitable Distribution of Teachers
- Simulations of this nature may be needed to
encourage policy.
23Implications for Practice
- Knowing and acting on data
- Disaggregating achievement data by classroom
- Using responsible and ethical assessment and HR
practices - Engaging in courageous conversations and
leadership actions - Teacher distribution and assignment
- Monitor teacher characteristics data to prevent
neediest schools from having disproportionately
inexperienced/less qualified teachers - Referee student assignment to avoid repeated
exposure to low performing classrooms (Sanders)
24Limitations
- Methodology
- Gain scores
- Small student n size per teacher
- Multiple regression vs. HLM
- Internal Validity
- Does measuring classrooms measuring teachers?
- Unidentified covariates
- External Validity
- Ability to generalize
- Assumptions that teachers would perform similarly
in different situations
25Suggestions for Future Research
- A multi-state study.
- A study of teachers that lasted more than one
year. - A study of other forms of mathematics content
acquisition. - A study that includes variables for teachers who
took a remedial mathematics course or who failed
a mathematics course. - A qualitative study of teachers whose students
significantly outperform.
26 Measuring the Link between Elementary Teachers
and Student AchievementA Presentation of the
DissertationElementary Teachers and the
Mathematics Achievement of Urban Students
- Alan Spicciati, Ed.D.
- Seattle Pacific University, Class of 2008
- spicciad_at_hsd401.org