Relating teacher candidate performance to their students academic achievement

1
Relating teacher candidate performance to their
students academic achievement

• Mark J. Fenster
• Diane L. Judd
• Carolyn A. Cox
• Valdosta State University

2
Paper presented at the annual meeting of the
Southeastern Evaluation Association, Tallahassee
FL, February 3, 2006
3
Correspondence goes to

• Address correspondence to Mark Fenster,
  Department of Educational Leadership, Valdosta
  State University, Valdosta, GA 31698-0090, e-mail
  fenster_21stcentury_at_hotmail.com, or
  mfenster_at_valdosta.edu OR
• Diane Judd, Department of Early Childhood and
  Reading Education, Valdosta State University,
  Valdosta GA 31698-0092, e-mail djudd_at_valdosta.edu

4
Issue and problem

• Currently, teacher education preparation programs
  are under pressure from the public at the state
  and national levels to provide evidence that
  their programs and teacher candidates are
  improving student achievement.

5
Issue and problem

• Student achievement is not limited to teachers
  abilities, but comes from a myriad of different
  places students, home, school, peers, teachers
  and principals, to mention six pertinent examples
  (Hattie, 2003).

6
Issue and problem

• Hattie claimed most of the variation on
  achievement is attributable to students (50),
  the second biggest contribution is made by
  teachers (30), with home, peers and schools
  contributing about 5-10 each, with a small
  indirect effect attributed for principals.

7
Issue and problem

• Hattie (2003) goes on to conclude that the answer
  to improving student achievement,
• lies in the person who gently closes the
  classroom door and performs the teaching act-the
  person who puts into place the end effects of so
  many policies, who interprets these policies, and
  who is alone with students during their 15,000
  hours of schooling (p.3).

8
What this paper is about

• This paper is an analysis of those people who are
  in preparation to close that classroom door and
  teach.

9
Figure 1 Achievement variation
10
Hybrid Model used

• The model used for this paper was conceptually
  similar to the Western Oregon University model
  and supported by Renaissance Group 2
  institutions.

11
Hybrid Model used

• The teacher work samples contained information on
  the student achievement of the teacher
  candidates students with respect to learning
  goals, achievement, and analysis of students
  achievements presented by the teacher candidates.

12
Hybrid Model used

• The teacher candidate observation instrument was
  developed through the Georgia Systemic Teacher
  Education Program (GSTEP) .
• Teacher candidates were evaluated on 13
  indicators (The instrument can be found in
  appendix A of the paper).

13
NCATE standards

• The Program Standards for Elementary Teacher
  Preparation document developed by NCATE describes
  four major attributes for performance-based
  teacher preparation (National Council for the
  Accreditation of Teacher Education, 2000).
• Teacher candidate knowledge
• Teaching performances (methodology)
• Teacher dispositions (values and commitments)
• Positive effects on student learning

14
Three of the four NCATE standards were dealt with
in the paper

• The current paper focuses on three of the four
  attributes for performance-based teacher
  preparation with particular attention to
  analyzing impacts of student learning following
  the Western Oregon University model of teacher
  work samples. Teacher dispositions are not dealt
  with in this paper.

15
Ten components of Western Oregon Universitys
model

• Describing a unit of study
• Mapping the classroom context
• Identifying learning outcomes
• Developing assessments for outcomes
• Administering pre-instruction assessments

16
Ten components of Western Oregon Universitys
model

• 6. Developing a design for instruction and
  assessment for all pupils
• 7. Implementing the instructional plan
• 8. Administering the post-instructional
  assessment
• 9. Summarizing, interpreting, and reporting the
  growth of each pupil and selected groups of
  pupils in the class
• 10. Reflecting and evaluating the teaching and
  learning process for the instructional unit.

17
WOU model and VSU

• These ten tasks were incorporated into Valdosta
  States early childhood and reading teacher
  candidate preparation program.

18
Methods-observation instrument

• Teacher candidates were formally assessed by
  their university supervisor three times during
  their student teaching semester early during the
  period (usually in the second or third week),
  during the middle (six to eight weeks) and at the
  end (weeks twelve to thirteen).
• These supervisor ratings provided the basis to
  look at initial ratings (the baseline) and any
  change from those rating during the student
  teaching experience.

19
Methods-teacher work sample

• A pre-test and post-test was given to pre-k to
  grade 5 students on all learning goals for each
  unit taught by the teacher candidate.
• After collecting data from the teacher work
  sample on student achievement (the pre-test and
  post-test) we then looked the relationships
  between the supervisors evaluations of teacher
  candidates work and the student achievement of
  those teacher candidates students.

20
Methods-change score

• We looked at a change as a function of change
  hypothesis to analyze the relationship between
  the teacher observation instrument and student
  achievement.
• Since both variables were changing over the same
  time continuum, no causal statement could be made
  regarding whether changing performance of teacher
  candidates impacted changing student achievement.

21
Results
22
Teacher candidate performance

• Teacher candidate performance improved from the
  first to second observation, and improved again
  from the second to third observation.
• Mean on first observation 3.60 (out of 4)
• Mean on second observation 3.83
• Mean on third observation 3.91

23
Teacher candidate performance

• Overall scores went up
• 84 of teacher candidates improved their overall
  rating between the 1st and 2nd administration of
  the instrument and
• 51 teachers improved their overall rating
  between the 2nd and 3rd administration of the
  instrument.

24
Teacher candidate performance

• 4 of teacher candidates maxed out their score
  the first time they were formally observed.
• 34 of teacher candidates maxed out their score
  the second time they were formally observed.
• 67 of teacher candidates maxed out their score
  the final time they were formally observed.

25
Analysis on student achievement

• Teacher candidates identified learning goals and
  gave a pre-test to their students on the learning
  goals they were to cover in their class.
• On learning goal number 1, the mean performance
  of the teacher candidates students was 16.8.
• On learning goal number 2, the mean performance
  of the teacher candidates students was 20.2.

26
Analysis on student achievement

• On both learning goals 1 and 2, nearly 50 of the
  teachers had 0 of their students showing
  knowledge of the topic before the material was
  covered in class.

27
Analysis on student achievement

• At the end of the unit, teacher candidates gave
  an assessment on the material they covered,
  collecting data on the percentage of students in
  their classes exhibiting knowledge gain with
  respect to their chosen learning goals.
• The mean achievement gain for learning goal
  number one was 73 and for learning goal two was
  71.

28
Analysis on student achievement
29
Analysis on student achievement
30
Correlation analysis, the relationship between
teacher candidate performance and pre-k to 5
student achievement

• We see teacher candidate observation scores
  increasing and student achievement improving. We
  are now in a position to test our core research
  hypothesis, the relationship between teacher
  candidate observation scores and student
  achievement.

31
Correlation analysis

• There was almost no relationship between the
  teacher observation ratings (first, second and
  third observation score) and the learning goal
  number one change score.
• There was virtually no relationship between the
  change in teacher observation instrument to the
  learning goal number one or number two change
  score.

32
Limitations

• We recognize some major limitations of the
  present study. We had ceiling effects with both
  the teacher observation instrument and the
  learning gain scores.

33
Limitations

• At the third (and final) rating, two-thirds of
  the teachers topped out scores on the observation
  assessment. Additionally, one-third to two-fifths
  of the classes maxed out the learning goal gain
  measure, improving from 0 to 100.

34
Limitations

• The ceiling effects on both variables could have
  affected the strength of some of the reported
  rank order correlations.

35
Discussion

• We found that teacher candidates had large and
  positive learning effects. The pre-tests showed
  that students came into the units with little
  knowledge of the material (16-20) but showed
  considerable gains at the end of the unit (gain
  scores of 71 and 73).

36
Discussion

• Teacher candidates can be part of the 30 of the
  variance in student achievement attributed to
  teachers by Hattie (2003).

37
Discussion

• The learning goals were defined by teacher
  candidates. The learning goals had to be aligned
  with Georgia state standards, but the teacher
  candidates were free to choose from a wide range
  of learning goals. The teacher candidates were
  measured on their choice of topics. If the
  teacher candidates were measured on a body of
  knowledge not of their choosing, like student
  performance on state mandated assessments, the
  results of this study could have been different.

38
Discussion

• The measurement problems inherent in this kind of
  research were exacerbated with ceiling effects.
  Teacher candidates that showed the greatest
  improvement on the observation instrument were
  not the same teachers that had their students
  show the greatest gains on the learning goals.
  There was almost no relationship between
  improvement on the observation instrument and
  differential improvement on learning goal gains.

39
Discussion

• Learning goal gains were strong across the
  board. Nearly all classes showed large gains in
  both learning goals (one and two).

40
Discussion

• The very high percentage (nearly 100) of k-5
  students achieving the learning goals may mean
  that teacher candidates can conceivably set
  higher standards.

41
Discussion

• Lastly, the increases in leaning gain scores
  reported here was all short term.
• Pankratz (1999, 46) cautions us, what can be
  expected in a given time period?  Are the
  conditions different for short-term progress than
  for long-term gains?  Can short-term gains be a
  valid measure of teacher performance?