Developing a Rubric for Assessment

1
Developing a Rubric for Assessment

• Office of Assessment
• University of Nebraska at Kearney

2
What is a rubric?

• A systematic scoring guideline to evaluate
  students performance (papers, speeches, problem
  solutions, portfolios, cases) through the use of
  a detailed description of performance levels.
• Used to get consistent scores across all
  students.
• Allows students to be more aware of the
  expectations for performance and consequently
  improve their performance.
• http//www.utexas.edu/academic/mec/research/pdf/ru
  bricshandout.pdf

3
Why do we use rubrics for assessment?

• A rubric is a flexible tool that can be used to
  measure student learning related specifically to
  a departments objectives.
• Because rubrics provide descriptions of each
  score level, it is easier for different faculty
  to use a rubric to grade consistently across
  students.

4
Rubric Characteristics

• For assessment purposes, we usually want to
  develop a rubric that
• allows us to directly evaluate performance on
  program level student learning outcomes.
• is general enough that it can be used for
  different assignments (e.g. papers may be
  collected from different courses/faculty).
• is understood and agreed upon by faculty who will
  be using the rubric.

5
Step 1 in developing a rubric

• 1. Clearly define the assignment.
• What is the student expected to produce?
• What are the common expectations across
  instructors?

6
Step 2 in developing a Rubric

• 2. Consider what student learning outcomes will
  be assessed.
• Often with a culminating project, students are
  expected to demonstrate several of the
  department/program outcomes.
• For example, for a senior seminar paper, outcomes
  related to writing and critical thinking may be
  assessed, as well and more discipline-specific
  outcomes.

7
Step 3 in developing a Rubric

• 3. Determine the key criteria that you are
  interested in-- for example, for the senior
  seminar paper, what aspects of writing will be
  assessed?
• Coherence
• Organization
• Mechanics

8
Step 4 in developing a Rubric

• 4. Clearly define those key criteria.
• What do you mean by organization?
• What does organized writing look like?

9
Step 5 in developing a Rubric

• 5. Establish clear and detailed descriptors for
  each performance level for each criteria
• Determine what the different levels of
  performance look like within each criteria
• Use sample papers of high, mid and low performers
  to help
• It is usually easiest to begin by describing the
  highest level of performance
• Using specific language for the descriptors of
  performance levels increases the chances that
  multiple faculty members will apply the rubric in
  a similar manner.

10
Step 6 in developing a Rubric

• 6. Try out the rubric on a few students with
  several raters to see if the rubric works and
  gets consistent scoring from multiple raters.

11
Scales vs. Rubrics
Below Expectation Meets Expectation Exceeds Expectation
Citations X
12
Scales vs. Rubrics
Below Expectation Meets Expectation Exceeds Expectation
Citations The paper fails to cite sources using a consistent, formal, citation style Most follow a consistent style, although some contain minor errors or incomplete information All citations are complete, accurate, and consistently conform to a formal style
13
Initial Assessment Scale
14
Improved Rubric
15
Using Rubrics

• Train evaluators
• Sample work should be scored
• Discuss scores and reach agreement
• More than on evaluator should score papers
• If two evaluators disagree a third decides
• Frequent disagreements on an item indicate the
  item needs to be revised or removed

16
Practice

• Select a rubric that is currently being used in
  your department
• Is it a rubric or a scale?
• Are there multiple descriptions of the criteria
  (at least 3)?
• Are the descriptions clear and easy to use?
• Will other faculty agree on the descriptors?

17
Practice

• Rewrite the selected rubric to ensure it has
  multiple descriptors of the criteria (is not a
  scale), clear descriptors, and ratings that can
  be agreed upon by other faculty.
• Look at samples at httpwww.web.virginia.edu/iaas
  /assessment/assessrubrics.htm

18
Where to start

• Developing a rubric from scratch can be
  challenging.
• Get ideas by looking at what others have done
• Lots of sample rubrics for many disciplines at
  http//www.winona.edu/air/rubrics.htm
• Online tool for developing rubric (with more
  samples)
• Rubistar for Teachers http//rubistar.4teachers.o
  rg/index.php
• Online module for using the Rubistar at
  http//www.vcu.edu/cte/resources/videos/Rubistar_t
  utorial/index.html

19
Sources

• http//www.winona.edu/air/resourcelinks/rubric_sam
  pler.pdf (document of rubric development long)
• http//www.utexas.edu/academic/mec/research/pdf/ru
  bricshandout.pdf - good description of rubrics
  with focus on general analytic rubrics for the
  purpose of assessment