Ways Teacher Work Samples Impact the Learning of All Students

1
Ways Teacher Work Samples Impact the Learning of
All Students
In R. S. Pankratz (Chair), Evidence of Teacher
Work Sample Impact on P-12 Student Learning,
Teacher Performance and Teacher Preparation
Programs
Peter R. Denner Idaho State University Stephanie
A. Salzman Western Washington University
AACTE Annual Meeting January 25, 2003
2
We looked at three things...

• The technical quality of the learning assessments
  provided by teacher candidates in their RTWS.
• Whether there was evidence for student learning
  gains for each learning goal and whether there
  was evidence for meeting the achievement goals.
• The disaggregations teacher candidates
  made with respect to student learning
  data in their work samples.

3
Technical Quality

• We developed a Quality of Learning Assessments
  (QLA) rating scale focused on 12 technical
  quality criteria.

See Handout Appendix B
4
Quality of Learning Assessments

• For the 12 Quality of Learning Assessment (QLA)
  items, the criteria were rated as
• 0 Does Not Meet Criterion
• 1 Partially Meets Criterion
• 2 Meets Criterion
• Summing the ratings across the 12 items provided
  a total score (possible range 0 - 24).

5
Source of the RTWS

• Second year TWS (N 87 June 2002 ) from the
  Renaissance Partnership for Improving Teacher
  Quality Project.
• The random set consisted of 10 TWS
• 1 Beginning
• 3 Developing
• 4 Proficient
• 2 Expert

6
Panel of Expert Raters

• The expert raters were 3 university faculty
  members representing different institutions
• Two raters held positions as director or
  coordinator for assessment at their institutions.

7
After training, the three QLA raters
independently scored the 10 TWS.

• The three rater coefficient of dependability for
  the QLA scores was .84.

8
Correlation of QLA Total Scores with Renaissance
TWS Total Scores

• The correlation was positive with r .70.
• Last year, we found a correlation of r .86
  for an exemplar set of TWS.
• Together, these data support the idea that
  teacher education candidates who scored well on
  the RTWS tended to use better assessments to
  demonstrate their impact on student learning.
• Caution is warranted in interpreting our findings
  because these correlations are based on small
  samples.

9
Correlation of QLA scores with RTWS sub-scale
scores for each standard.

• See handout Appendix C for table of correlations.
• The highest correlation was between the Analysis
  of Student Learning scores and the QLA total
  scores with r .91.
• The second highest correlation was between the
  Learning Goals scores and the QLA total scores
  with r .80.

10
Evidence for Learning Gains and Achievement of
Learning Goals
11
Evidence for Student Learning Gains

• Two Raters reached consensus on whether each TWS
  presented clear evidence for student learning
  gains for the targeted learning goals.
• The rated work samples were selected from N 117
  year three RTWS (January, 2003).
• There were 5 Expert, 10 Proficient, 10 Developing
  and 4 Beginning TWS in the randomly selected set.

12
(No Transcript)
13
Evidence for Achievement of Learning Goals

• The two raters also reached consensus on whether
  the same set of TWS contained evidence for
  student achievement of the learning goals
  according to teacher set criteria.

14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
Disaggregation of Achievement Data
21
Disaggregation Task

• Subgroups. Select a group characteristic (e.g.,
  gender, performance level, socio-economic status,
  language proficiency) to analyze in terms of one
  learning goal. Provide a rationale for your
  selection of this characteristic to form
  subgroups (e.g., girls vs. boys high- vs.
  middle- vs. low-performers). Create a graphic
  representation that compares pre- and
  post-assessment results for the subgroups on this
  learning goal. Summarize what these data show
  about student learning.

22
Frequent Disaggregation by Gender
23
Poor Disaggregation Focuses on Grades
24
(No Transcript)
25
Disaggregation by IEP Vs Whole Class
Good News Proportional impact was nearly the
same.
26
Conclusions

• Our findings provide support for the idea that
  successful performance on a teacher work sample
  is an indication of higher quality assessment of
  student learning.
• Better TWS performance is also an indication of
  better evidence for learning gains.
• The trend is similar (but not as clear )for
  evidence for attainment of achievement goals.

27
Conclusions

• More needs to be done to guide teacher candidates
  to state their assessment criteria so it can be
  determined how many of their students met their
  achievement goals.
• RTWS show evidence of the abilities of teacher
  candidates to disaggregate achievement data.

28
For more information about the Renaissance
Teacher Work Sample visit our web site
at http//fp.uni.edu/itq/