Designing Rubrics

1
Designing Rubrics

• Nancy Allen, Ph.D.
• College of Education
• Office of Faculty Development
• Qatar University

Adapted from Baggio, C. (n.d.). Tips for
designing rubrics. Retrieved on May 29, 2007,
from www.sdst.org/shs/library/powerpoint/rubrics.p
pt and
2
Designing Rubrics

• Students as Self Assessors
• Teachers as Focused Coaches

3
What is a rubric?

• A rubric is a guideline for rating student
  performance.
• Benefits
• The rubric provides those doing the assessment
  with exactly the characteristics for each level
  of performance on which they should base their
  judgment.  
• The rubric provides those who have been assessed
  with clear information about how well they
  performed.  
• The rubric also provides those who have been
  assessed with a clear indication of what they
  need to accomplish in the future to better their
  performance.

Asmus, E, (1999). Rubrics. Retrieved on May 29,
2007, from http//www.music.miami.edu/assessment/r
ubrics.html
4
What is a rubric?

• Quality Continuum
• A rubric must define the range of possible
  performance levels.  Within this range are
  different levels of performance which are
  organized from the lowest level to the highest
  level of performance.  Usually, a scale of
  possible points is associated with the continuum
  where the highest level receives the greatest
  number of points and the lowest level of
  performance receives the fewest points.

5
What is a rubric?

• A rubric is a lesson in quality
• A public declaration of expectations
• A communication tool
• A self-assessment tool for learners
• A gauge for examining performance
• A self-fulfilling prophecy

6
What is a rubric?

• Quality Continuum

PERFORMANCE LEVEL POINTS POINTS
Excellent 4 90-100
Good 3 80-89
Satisfactory 2 70-79
Needs Improvement 1 60-69
Clearly unsatisfactory 0 lt60
7
Rubric vs. Checklist

• Checklist for a friendly letter
• ______ Date, flush left at top
• ______ Address
• ______ Greeting
• ______ Body
• ______ Salutation
• ______ Signature

8
Rubric vs. Checklist

• Checklists have not judgment of quality.
• Checklists can only be used when present or
  absent is a sufficient criterion for quality.

9
Rubric vs. Checklist

• Rubrics include descriptors for each targeted
  criterion.
• Rubrics provide a scale which differentiates
  among the descriptors.

10
What is a rubric?

• Descriptors
• Each level of performance should have descriptors
  which clearly indicate what is necessary to
  achieve that level of performance. 
• Example
• Organization of Thought (4-points) Work is
  clearly organized and includes a diagram or
  step-by-step analysis.

11
The parts of a rubric
12
Determining Standards of Excellence

• How many degrees of quality should you include?
• Should you use language or numbers? If language,
  what descriptive terms should you use?
• Should you weigh the items?

13
Criteria

• The specific areas for assessment
• Focus areas for instruction
• Clear and relevant
• Age appropriate
• Form and function represented

14
Indicators

• Descriptors of level of performance for the
  criteria
• Conclusion includes whether the findings
  supported the hypothesis, possible sources of
  error, and what was learned from the experiment.
• Clear, observable language
• Examples for learners

15
How do rubrics alter instruction?

• The teacher commits to teaching quality.
• The teacher commits to assisting the student
  self-assess.
• The focus is on each product and/or performance.
• The labels are removed from students.
• Specificity appears in all communications.
• Everyone gives and receives feedback.

16
Whom does a rubric assist?

• It is a feedback system for students to judge a
  product or performance.
• It is a feedback tool for teachers to provide
  clear, focused coaching to the learner.
• It is a system that promotes consistent and
  meaningful feedback over time.
• It is a communication tool for parents.

17
What makes a quality RUBRIC?

• An even number of standards of excellence
• Clear essential criteria
• Realistic number of criteria
• Explicit, observable indicators

• If points clear to students upfront
• Deliberate sequence of criteria
• High interjudge reliability
• Tested out with students

18
What makes a good judge?

• Knowledge and experience with specific skill
• Practice with rubri.
• Objectivity
• Questions rubric in advance to be sure all
  participants understand

19
How do I get started?

• Critique current models.
• Ask students to define quality in relation to
  specific product or performance.
• Translate into a modest rubric.

20
Expert Input

• Experts agree
• Rubrics are hard to design.
• Rubrics are time-consuming to design.
• A rubric is only as useful as it is good. Using
  a bad rubric is a waste of time
• --Michael Simkins in Designing Great Rubrics
• Experts disagree
• How to design a good rubric
• Bottom line Is it working for you and for your
  students?

21
Holistic Or AnalyticWhich To Use?

• HOLISTIC
• Views product or performance as a whole
  describes characteristics of different levels of
  performance. Criteria are summarized for each
  score level.

22
Holistic Or AnalyticWhich To Use?

• Excellent Researcher
• included 10-12 sources
• no apparent historical inaccuracies
• can easily tell which sources information was
  drawn from
• all relevant information is included
• 2 - Good Researcher
• included 5-9 sources
• few historical inaccuracies
• can tell with difficulty where information came
  from
• bibliography contains most relevant information
• 1 - Poor Researcher
• included 1-4 sources
• lots of historical inaccuracies
• cannot tell from which source information came
• bibliography contains very little information

23
Holistic Or Analytic?

• HOLISTICpros and cons
• Takes less time to create.
• Effectively determines a not fully developed
  performance as a whole
• Efficient for large group scoring less time to
  assess
• - Not diagnostic
• - Student may exhibit traits at two or more
  levels at the same time.

24
Holistic Or Analytic?

• Analytic
• Separate facets of performance are defined,
  independently valued, and scored. Facets scored
  separately

25
Holistic Or Analytic?

• Analyticpros and cons
• Sharper focus on target
• Specific feedback (matrix)
• Instructional emphasis
• Time consuming
• Takes skill and practice

26
Task specific or general?

• Task specific Rubric designed for and
  references a specific assignment.
• General Rubric designed for and references a
  type of assignment frequently repeated.

27
Tip 1

• Use as many generalized rubrics as possible.
• Efficient
• Builds recognition of excellence

28
Tip 2

• If using pre-designed rubrics carefully consider
  quality and appropriateness for your project.

29
Tip 3

• Aim for concise, clear, jargon-free language
• in most instances, lengthy rubrics probably
  can be reduced to succinctmore useful versions
  for classroom instruction. Such abbreviated
  rubrics can still capture the key evaluative
  criteria needed to judge students responses.
  Lengthy rubrics, in contrast, will gather dust
  (Benjamin 23).

30
Tip 4

• Limit the number of criteria, but
• Separate key criteria.
• Very clear and very organized may be clear
  but not organized or vice versa.

31
Tip 5

• Use key, teachable criteria.
• Key Questions What are my objectives? Are
  there other generalized objectives that should be
  included?

32
Tip 6

• Use concrete versus abstract and positives rather
  than negatives
• Instead of poorly organized use sharply
  focused thesis, topic sentences clearly connected
  to thesis, logical ordering of paragraphs,
  conclusion ends with clincher.
• Key Question to ask yourself Would student
  know what quality looked like by this
  description?

33
Tip 7

• Use measurable criteria.
• Includes two or more new ideas instead of
  creative and imaginative

34
Tip 8

• Aim for an even number of levels
• Create continuum between least and most
• Define poles and work inward
• List skills and traits consistently across levels

35
Tip 9

• Include students in creating or adapting rubrics
• Consider using I in the descriptors
• I followed preciselyconsistentlyinconsistentlyM
  LA documentation format.
• I did not follow MLA documentation format.

36
Tip 10

• Provide models of the different performance
  levels.

37
The Assignment Sheet

• Link the assignment sheet and the rubric. Use
  same language.
• Include all non-negotiable items.
• On time
• Formatted correctly
• Follows standard conventions
• Etc.

38
Rubrics for formative assessment

• Encourage students to check progress using the
  rubric.
• Encourage / require self-assessment and/or peer
  assessment.

39
Steps in Developing a Rubric

• Design backwardsrubric first then
  product/performance.
• Decide on the criteria for the product or
  performance to be assessed.
• Write a definition or make a list of concrete
  descriptorsidentifiable-- for each criterion.
• Develop a continuum for describing the range of
  performance for each criterion.
• Keep track of strengths and weaknesses of rubric
  as you use it to assess student work.
• Revise accordingly.
• Step back ask yourself, What didnt I make
  clear instructionally? The weakness may not be
  the rubric.

40
Rubrics On Line

• "Rubistar Rubric Generator" (http//rubistar.4teac
  hers.org/) 
• "Teacher Rubric Maker" (http//www.teach-nology.co
  m/web_tools/rubrics/)
• Rubrician (http//www.rubrician.com/language.htm
  
• Rubrics for Web Lessons (http//edweb.sdsu.edu/tri
  ton/july/rubrics/Rubrics_for_Web_Lessons.html)
• An Online Rubric Maker (http//landmark-project.c
  om/classweb/rubrics/)

41
References

• Andrade, H.(2000). Using rubrics to promote
  thinking and learning. Alexandria, VA ASCD.
• Asmus, E, (1999). Rubrics. Retrieved on May 29,
  2007, from http//www.music.miami.edu/assessment/r
  ubrics.html
• Baggio, C. Designing rubrics Revising
  instruction and improving performance. Retrieved
  on March 1, 2007, from http//www.edutech.org.br.
• Baggio, C. (n.d.). Tips for designing rubrics.
  Retrieved on May 29, 2007, from
  www.sdst.org/shs/library/powerpoint/rubrics.ppt
• Benjamin, A.(2000). An English teachers guide
  to performance tasks and rubrics. Larchmont Eye
  on Education.
• Leavell, A. (n.d.). Authentic assessment Using
  rubrics to evaluate project-based learning.
  WEBLIBRARY.
• Matthews, J. (2000). Writing by the rules no
  easy task. Retrieved on October 25, 2000 from
  lthttp//washingtonpost.com/wp-dyn/articles/A63599-
  2000Oct23.htmlgt
• Simkins, M. (1999, August). Designing great
  rubrics. Technology and Learning.
• Wiggins, G. McTighe, J. (1998). Tips for
  developing effective rubrics. Understanding by
  Design. Alexandria, VA ASCD.