Developing and Validating Scoring Rubrics

1
Developing and Validating Scoring Rubrics

• Presented at CCRI
• September, 2005
• Peggy Maki
• PeggyMaki_at_aol.com

2
Scoring Rubrics

• A set of criteria that identifies the expected
  characteristics of a text and the levels of
  achievement along those characteristics. Scoring
  rubrics are criterion-referenced, providing a
  means to assess the multiple dimensions of
  student learning.
• Are collaboratively designed based on how and
  what students learn (based on curricular-co-curric
  ular coherence)

3

• Are aligned with ways in which students have
  received feedback
• (students learning histories)
• Students use them to develop work and to
  understand how their work meets standards (can
  provide a running record of achievement).

4
Developing Scoring Rubrics

• Emerging work in professional and disciplinary
  organizations
• Research on learning (from novice to expert)
• Student work

5

• Interviews with students
• Experience observing students development

6
Interpreting Student Achievement through Scoring
Rubrics

• Criteria descriptors (ways of thinking, knowing
  or behaving represented in work)
• Creativity
• Self-reflection
• Originality
• Integration
• Analysis
• Disciplinary logic

7

• Criteria descriptors (traits of the performance,
  work, text)
• Coherence
• Accuracy or precision
• Clarity
• Structure

8
Exercise

• Using your own student papers, list the criteria
  descriptors you will use (or do use) to score
  these papers, looking first at the best example
  and then at the least successful example.

9

• Performance descriptors (describe well students
  execute each criterion or trait along a continuum
  of score levels)
• ExemplaryCommendable Satisfactory-
  Unsatisfactory
• ExcellentGoodNeeds ImprovementUnacceptable
• ExpertPractitionerApprentice--Novice

10
Exercise

• Using your own student papers, list the
  performance descriptors you will use (or do use)
  to score these papers, looking first at the best
  example and then at the least successful example.
  Then establish descriptors that lie inbetween
  these two extremes.

11
Validating the Scoring Rubric Establishing
Inter-rater Reliability

• Apply to student work to assure you have
  identified all the dimensions with no overlap
• Schedule inter-rater reliability times
• -independent scoring
• -comparison of scoring
• -reconciliation of responses
• -repeat cycle