Effective grading and teaching through the use of rubrics

1
Effective grading (and teaching) through the use
of rubrics use of rubrics for assessment

• Sarah Murnen
• Kenyon College

2
Personal Background

• Although a psychologist, no particular background
  in this topic
• As Kenyons Assessment Coordinator from
  2002-2006 I learned that
• Some faculty resent the assessment process and
  see it as extra burden
• Departments with faculty who feel burdened by
  assessment dont get much out of it
• Departments that find a way to integrate
  assessment with what they already do benefit from
  the process they discuss what students are
  learning and change their teaching methods,
  assignments, and sometimes their entire curricula
  to help students learn

3
Benefits of Ohio-5 Project

• To help improve student learning (and faculty
  teaching) by encouraging the use of rubrics
• (Many other benefits to the use of rubrics)
• To specifically encourage a focus on creativity
  and critical thinking, important aspects of a
  liberal education
• To use rubrics for assessing critical thinking
  and creativity
• To show faculty how all of the work they put into
  grading can be used for assessment purposes

4
Background Information

• Presentation by Dr. Douglas Eder on Primary
  Trait Analysis
• Presentation by Barbara Walvoord on the use of
  rubrics for effective grading
• Walvoord, B. E., P Anderson, J. A. (1998).
  Effective Grading A tool for learning and
  assessment. Jossey-Bass San Francisco, CA.
• Stevens Levi (2005). Introduction to rubrics
  An assessment tool to save grading time, convey
  effective feedback, and promote student learning.
  Stylus Publishing Sterling, Virginia.

5
Our primary concern is student learning

• Good practice in undergraduate education
  (Walvoord, 1998, p. 15)
• Encourages student-faculty contact
• Encourages cooperation among students
• Encourages active learning
• Gives prompt feedback
• Emphasizes the time that students devote to the
  task
• Communicates high expectations
• Respects diverse talents and ways of learning

6
Walvoord asks (1998, p. 15)

• How many of these principles of good practice in
  some way involve the grading system in your
  class, the tests and assignments on which that
  system is based, and your ways of communicating
  with students about their work and their grades?

7
Walvoords Argument for Establishing Clear
Criteria and Standards for Grading

• Saves time in grading process
• Allows you to make the process consistent and
  fair
• Helps you explain to students what you expect
• Shows you what to teach
• Identifies essential relationship between
  discipline information and processes
• Help students evaluate their own and each others
  work
• Saves you from having to explain your criteria to
  students after they have handed in their work, as
  a way of justifying the grades they are
  contesting
• Helps student peers give each other constructive
  feedback on plans and drafts
• Helps team teachers or teaching assistants grade
  student papers consistently
• Helps teachers of sequenced courses communicate
  with each other about standards and criteria
• Form the basis for departmental or institutional
  assessment

8
How to Establish Clear Criteria Primary Trait
Analysis

• Developed to score essays on the National
  Assessment of Educational Progress by Lloyd-Jones
  in1977
• Creates a scoring rubric that can be used to
  assess any student performance
• Is assignment specific
• Can be used for grading
• For this project, we will develop rubrics using
  PTA to measure the critical thinking and
  creativity we are encouraging in our students

9
Why take the time to do PTA for grading?

• Makes grading more consistent and fair
• Saves time in the grading process once the rubric
  is developed
• Can diagnose students strengths and weaknesses
  very specifically in order to teach more
  effectively
• Can track changes in student performance

10
Process of PTA (Eder)

• Identify the primary traits essential or
  central components of the discipline to be
  learned by the student
• Build a scale for scoring the students
  performance on the trait
• Evaluate the students performance against those
  criteria

11
Key Stages in Constructing a Rubric (Stevens
Levi)

• Reflecting -
• What do we want from our students?
• Why did we create the assignment?
• What happened the last time we used the
  assignment?
• What are our expectations for the assignment?
• Listing Focus on particular details of the
  assignment and what specific learning objectives
  we hope to see completed (Sometimes helps to
  imagine the best and the worst performance on the
  assignment)
• Grouping and Labeling the goals together -
  Organize the results of our reflections in Stages
  1 and 2, grouping similar expectations together
  in what will probably become the dimensions of
  the rubric
• Application Transfer groupings to a rubric grid

12
Example Research Article Critique

• Reflecting -
• Want students to learn to be critical evaluators
  of psychological research
• The research article critique should start them
  thinking critically, will follow up with class
  discussion on each critique, and use multiple
  critiques throughout the semester to help develop
  their skills
• I expect students will move from description to
  analysis
• Listing
• I have seen students move through Blooms
  taxonomy in a semester using this assignment
  Most can show knowledge and comprehension at
  the beginning of the course, move to
  application, analysis, synthesis, and
  evaluation (hopefully). I want to try to
  capture this process.
• Grouping and Labeling the goals together
• see the questions that follow
• Application Transfer groupings to a rubric grid
• See the rubric that follows

13
Research Article Critique Assignment

• For each article analysis you are to answer the
  following questions
• What is the primary question posed by the study?
• Is there a hypothesis stated? If so, what is it?
• What is the theoretical explanation for the
  proposed hypothesis?
• Briefly describe the way the independent and
  dependent variable(s) were manipulated or
  measured
• How do the results of the study affect the
  originally posed hypothesis (or purpose of
  study)?
• Two strengths of the study?
• Two weaknesses of the study?
• What is a logical extension of the study?
  Briefly describe a study you could conduct to
  extend the research

14
Sample Rubric Designed for AssessmentACC
Accomplished, AVG average, DEV developing,
BEG beginning
15
Rubric Used for Grading weight each component
16
Describing the level of performance on the rubric

• Anywhere from 2 to 5 levels of performance on
  each trait
• Terms used to describe level of performance might
  be positive and active to encourage motivation in
  students (suggestions from Stevens Levi)
• Mastery, partial mastery, progressing, emerging
• high level, middle level, beginning level
• Sophisticated, competent, partly competent, not
  yet competent
• Exemplary, proficient, marginal, unacceptable
• Advanced, intermediate high, intermediate, novice
• Distinguished, proficient, intermediate, novice
• Accomplished, average, developing, beginning

17
Using Rubrics for Assessment

• We all need to attend to the issue of assessment,
  and Walvoord argues that we can make use of what
  we already do in the grading process

18
American Association for Higher Educations
Principles of Good Practice for Assessing
Student Learning

• Answer questions that people care about
• Lead directly to improvement in teaching and
  learning
• Be embedded in the context of learning
• Take place repeatedly over time

19
The rubrics we use for grading can be used for
assessment

• If we as faculty do not make our learning goals,
  tests, criteria, and standards explicit and
  understandable to legislatures, boards,
  accrediting agencies, and other audiences in ways
  that meet their needs and concerns, we face the
  very real possibility that some of the control we
  currently exercise in the classrooms will be
  taken away from us. We must deal with
  assessment, but we need not construct a parallel
  assessment structure that ignores the assessment
  we already conduct. (Walvoord, 1998, p. 5).

20
How to turn this into assessment

• Paper 1 Paper 2 Paper 3 Total
• Prof A
• Creative1 4 1 2 7
• Creative2 4 1 2 7
• Critical1 3 1 3 7
• Critical2 2 1 3 6
• Prof B
• Creative1 3 2 2 7
• Creative2 4 1 3 8
• Critical1 3 1 3 7
• Critical2 2 1 3 6
• Prof C
• Creative1 4 1 2 7
• Creative2 3 2 3 8
• Critical1 2 2 3 7
• Critical2 2 1 3 6

21
How have faculty been influenced by this process?

• I was able to clarify my expectations for the
  course by articulating the purpose behind each
  graded exercise. This helped dramatically in
  easing anxieties about performance and final
  grades. Students seemed to appreciate
  understanding the logic behind the assignments
  and seeing that each focused on building and
  assessing a particular skill, rather than merely
  providing me with another grade.
• Best of all, Im looking forward to sharing the
  process of developing PTA-based rubrics with my
  students. Specifically, students will work in
  groups to identify their own PTA-based critical
  thinking rubrics before they embark on writing a
  particular essay. I can hardly think of a better
  way to develop the students meta-cognition.,
  i.e., how they think about their own thinking!
• Yes, weve been disappointed with some aspects
  of student performance (for example synthetic
  ability) and using the rubric has helped us
  communicate better with students, and were
  thinking about ways to improve courses to help
  teach the skills needed to succeed on this senior
  exercise objective.
• I have learned a great deal, and I think that my
  teaching has been positively influenced by this
  important work. I can also say that my research
  and scholarship have been impacted by these foci,
  as well.

22
Go to Website for Rubric Samples

• http//www.wooster.edu/teagle/default.html