Rubrics: Using Performance Criteria to Improve Student Achievement

1
Rubrics Using Performance Criteria to Improve
Student Achievement

• Martha Ross, James Madison University
• Jeri Carroll, Wichita State University

2
Agenda

• Program and Unit Review
• Rubrics
• Rubrics for Program Reports
• Rubrics for Unit Data

3
Program Review

• To complete a program report, institutions must
  provide evidence of meeting program (SPA)
  standards based on data from 6-8 assessments.

4
Required Program Assessments

• Content
• State licensure exam for program area (if
  availableotherwise another content based
  assessment)
• Content Assessment
• Professional and Pedagogical Knowledge Skills and
  Dispositions
• Assessment of Planning (e.g., unit plan)
• Student teaching/internship assessment

5
Required Program Assessments

• P 12 Student Learning
• Assessment of candidate impact on student
  learning or providing a supporting learning
  environment

6
Program Review

• Section IV requires a narrative, assessment,
  scoring guide/criteria, and data tables plus a
  2-page maximum narrative for each of the 6-8
  assessments)

7
Program Review

• Common AFIs
• Data do not show evidence of candidates meeting
  the standard.
• Criteria/cells of the rubric do not contain the
  language of the standards.

8
Examples

• Elementary Education Data Table
• Rubrics
• Without standards language in rubric.
• With standards language in the rubric.

9
Unit Review

• Program Reports provide data for Standard 1
  Elements 1a, 1b, and 1d.
• Program or unit data needed to document
• 1c Professional and Pedagogical Knowledge and
  Skills,
• 1e Knowledge and Skills for Other School
  Professionals
• 1f Student Learning for Other School
  Professionals
• 1g Professional Dispositions for All Candidates

10
Professional and Pedagogical Knowledge and Skills

• Initial
• Teacher candidates can apply the professional and
  pedagogical knowledge and skills delineated in
  professional, state, and institutional standards
  to facilitate learning.
• They consider the school, family, and community
  contexts in which they work and the prior
  experience of students to develop meaningful
  learning experiences.
• They reflect on their practice.
• They know major schools of thought about
  schooling, teaching and learning.
• They are able to analyze educational research
  findings and incorporate new information into
  their practice as appropriate.

11
Professional and Pedagogical Knowledge and Skills

• Advanced
• Candidates in advanced programs for teachers
  reflect on their practice and are able to
  identify their strengths and areas of needed
  improvement.
• They engage in professional activities.
• They have a thorough understanding of the school,
  family, and community contexts in which they
  work, and collaborate with the professional
  community to create meaningful learning
  experiences for all students.
• They are aware of current research and policies
  related to schooling, teaching, learning and best
  practices.
• They are able to analyze educational research and
  policies and can explain the implications for
  their own practice, and for the profession.

12
Examples

• TWS Data Table
• Rubrics
• Without standards language in rubric.
• Rubrics mapped to NCATE Standards and Elements.

13
Rubrics Lesson Plans Elementary Education

• Learning Goals and Design for Instruction portion
  of the WSU Teacher Work Sample (TWS) served as an
  assessment of the candidates ability to plan
  instruction. Candidates were required to pass the
  whole TWS with a passing score of 80 or above
  (94 or more points of 117) and no less than 60
  in a section.

14
Rubrics Lesson Plans Elementary Education

• Learning Goals The candidate sets significant,
  challenging, varied and appropriate learning
  goals.
• Design for Instruction The candidate designs
  instruction for specific learning goals, student
  characteristics and needs, and learning contexts.

15
Rubrics Lesson Plans ElEd Health-Physical
Education

• Category
• Unacceptable Developing
• Acceptable Meets
• Target Exceeds
• Instructors Space--Grade and Comments
• WSU/KSDE Professional Education Standards
• KSDE Elementary Education Standards

16
Rubrics Lesson Plans ElEd Health-Physical
Education

• Standard 6 The kindergarten through sixth grade
  teacher knows, understands, and uses the major
  concepts of health education and human movement
  and physical activity as central elements to
  foster active, healthy life styles and enhanced
  quality of life for all students.

17
Rubrics Lesson Plans ElEd Health-Physical
Education

• Developing
• The lesson includes elements related to health,
  to be delivered as notes for students rather than
  meaningful uses and dispositions.
• Acceptable
• The lesson includes elements related to the
  health curriculum, and ways that students can
  have healthy lives.

18
Rubrics Lesson Plans ElEd Health-Physical
Education

• Target
• Lesson includes elements that focus on a
  meaningful application to life experiences, such
  that all students can be successful. A healthy,
  active lifestyle is the focus of the lesson.

19
Field Observation

• Generic Observation Form

20
Field Observation

• Content Specific Observation Form

21
Student Learning Generic
22
Student Learning Content Specific
23
WSU Lessons Learned

• Use the language of the standards in the rubrics.
• Use clear headings to show levels of performance.
• Avoid using the levels of criteria in the cells
  of the rubric define what levels mean.
• Provide requirements above the rubric as a
  checklist.

24
Modifying Unit Assessments to Meet Program
Standards

• The James Madison University Story

25
The Infamous GenericStudent Teaching Assessment

• Generic evaluation form since 1995
• Originally modeled after Pathwise
•   Framework
• Individual program interpretation of indicators
• And four area colleges/universities used the
  same form!

26
Last NCATE Visit Spring 2004

• SPA program reports submitted under old process
• No major questions raised at that time, but the
  hints were there
• Plus some programs and cooperating teachers
  also began expressing desire for more specific
  indicators

27
Phase 1 2004-2005

• Student teaching evaluation form revised to
  clearly assess
• content knowledge
• diversity
• technology
• impact on student learning
• Four instructional categories
• 23 indicators

28
ST-9 Instructional Categories

• A. Knowledge of Content (4)
• B. Preparation for Instruction (5)
• C. Instructional Performance (10)
• D. Reflection and Evaluation
• Impact on Student Learning (4)

29
A. Knowledge of Content

• A1. Demonstrates an understanding of appropriate
  content standards (Virginia PK-12 Standards of
  Learning, national professional standards)
• A2. Identifies key principles and concepts of
  subject matter
• A3. Uses examples to support basic principles of
  content
• A4. Links content to students prior experiences
  and to related subject areas

30
Three-Point Rubric

• A3. Uses examples to support basic principles of
  content
• 3.0 uses appropriate AND varied examples to
  illustrate basic content principles
• 2.0 uses some appropriate examples to illustrate
  basic content principles
• 1.0 uses inappropriate examples OR no examples
  to illustrate basic content principles

31
Questions for ReflectionA Generic Core

• Are the same examples used over and over when
  students need more clarity?
• Can the student teacher create new examples for
  further clarification?
• Are the examples appropriate for the age level
  and populations?
• Do the examples represent different modalities?
• Do the examples relate to diverse needs?

32
Phase 2 Spring and Summer 2006

• Student Teaching Reference Guide aligned with SPA
  standards
• 33 participants (10 teams/triads)
• arts and sciences faculty
• education faculty
• P-12 school personnel

33

34
One-Day Workshop

• Prior to workshop
• participants reviewed SPA standards, JMUs
  conceptual framework, Virginia PK-12 Standards of
  Learning
• English triad developed preliminary draft to use
  as a model
• Kick-off by Margie Crutchfield
• Individual team sessions for rest of day

35
Two Questions

• What specific SPA standards correlate to this
  indicator?
• What would this indicator look like at the 3.0
  level of scoring at your grade level and/or
  content area?

36
A3. Uses examples to support basic content
principles

• Example 1 Math
• NCTM Standards 1, 4, 5
• Does candidate recognize, use, and make
  connections between and among mathematical ideas,
  and in contexts outside mathematics, to build
  mathematical understanding?
• Does the candidate use varied representations of
  mathematical ideas to support and deepen
  students mathematical understanding?

37
A3. Uses examples to support basic content
principles

• Example 2 Music
• NASM Standards 3.b.4, 3.c.4
• Does the student teacher make effective use of
  explanation, illustration, modeling (vocal,
  keyboard, other instruments), conducting, and
  verbal imagery?
• Does the student teacher make appropriate use of
  negative as well as positive examples?

38
A3. Uses examples to support basic content
principles

• Example 2 Music
• NASM Standards 3.b.4, 3.c.4
• Does the student teacher employ music from a
  range of cultures and historical periods and
  place the music in context?
• Do warm-ups have a clear purpose? Are the
  relationships between any warm-ups and the music
  being performed made clear?

39
A3. Uses examples to support basic content
principles

• Example 3 Foreign Language
• ACTFL Standards 1C, 2A, 2C, 4A
• Do candidates use variations of language to show
  students the language differences of regions
  where the language is spoken?
• Do examples help students understand the
  similarities and differences between the target
  and heritage cultures?

40
Phase 3 Fall 2006

• Completed revision of SPA/program-specific
  Reference Guides
• Conducted training for university supervisors and
  all cooperating teachers
• Held refresher workshops for current clinical
  faculty
• Included new Reference Guides in training for new
  clinical faculty

41
Phase 3 Fall 2006

• Piloted new evaluation forms and Reference Guide
• 15 JMU programs / licensure areas
• 125 JMU student teachers
• Three of the four area colleges and universities

42
University Supervisor andCooperating Teacher
Ratings2006-2007
43
Plans for OngoingData Collection

• Unit Level
• Collect online from university supervisors
• Collect hard copy from cooperating teachers
• Aggregate and disaggregate by program,
  indicators, categories
• Unit and Program Levels
• Analyze for individual candidate performance
• Analyze for program improvement
• Analyze for improvement of unit operations

44
A Work in Progress Still Some Tweaking to Do !

• Review all of the SPA-specific questions for
  reflection for accurate placement within the
  indicators
• Consider changing the questions to statements
• Consider how best to analyze and present data

45
Some Comments

• I shouldnt tell you this, but I never looked at
  the earlier reference guide! I read this one
  each time I observe or evaluate a student
  teacher. Its very helpful!
• - Cooperating teacher
• It really helps in crafting concrete written
  feedback for a weaker student teacher.
• - University supervisor

46
More Comments

• This new reference guide is much more detailed
  and targeted to a specific subject or grade
  level.
• - Cooperating teacher
• I like the fact that we were included in
  revising the new guide!
• - Clinical Faculty

47
Access and Contact Information

• http//coe.jmu.edu/esc/Forms.shtml
• Click on Student Teaching for
• ST-8 - Student teaching observation form
• ST-9 - Student teaching evaluation form
• Student Teaching Performance Guide
• Reference Guides for each program
• Jane Smith smith3js_at_jmu.edu

48
Questions?