Grading Policies and Practices: A Survival Guide

1
Grading Policies and Practices A Survival
Guide for Pedagogical Dinosaurs in the Grade
Inflation Age

• The Grade Inflation Age
• Correlates of Students Perceptions of
• Fair Grading
• III. Test Construction and Administration
• IV. Grading Written Exercises
• Providing Feedback on Graded Exercises
• Tar Pits I Try to Avoid

2
Part IThe Grade Inflation Age
Ample evidence of grade inflation at the high
school level Juola (1980) - average GPA
increased by 0.39 between 1960 through
1978 Rojstaczer http//www.gradeinflation.com
Kutner et al. (2007) National Assessment of
Adult Literacy - prose literacy and document
literacy declined between 1992 through 2003 for
all levels of educational attainment beyond a
high school diploma
3
(No Transcript)
4
What are the Effects of Grade Inflation?
Grade inflation

• reduces students motivation to learn
• Olivares (2002)

• leaves students less prepared for
• future challenges
• Bonesrønning (1999)
• Johnson (2003)
• Wagner (2006)

• grade-driven course selection
• Johnson (2003) found that students did better in
  upper division courses if they took
• prerequisite courses from tough graders rather
  than easy graders who taught
• the same prerequisite courses.
• Bar, Kadiyali, Zussman (2007)

5
Part II Correlates of Students Perceptions
of Fair Grading

• Evidence from Rutgers
• survey - 400 sophomores and juniors in organic
  chemistry
• - 100 juniors in Principles of Management
  

6

• Please tell us how often you have a course in
  which each
• of the following practices is used. Estimate the
  percent of
• courses in which a particular practice is used by
  writing in
• a number from 100 (all of your courses) to 0
  (none of your
• courses) following each of the statements.
• In approximately what percentage of your classes
  do teachers
• Actual Advanced Preparation (AAP)
• provide study guides for the course material
• provide review sessions before major exams
• provide students with practice exams before
  regular exams
• Actual Post-hoc Grade Modification (APGM)
• drop the lowest test score before calculating the
  final grade
• give students the chance to take another test if
  they did poorly on the original test
• allow students to do extra work in order to raise
  their final grade
• curve grades upwards when scores are low on an
  examination
• modify grades awarded to students whose
  circumstances (e.g., car trouble) may have
  interfered with their performance on tests
• count the last tests in a course more heavily
  than the first tests if there
• is steady improvement in exam scores

7
70
60
50
40
30
20
10
0
8

• Please indicate your degree of agreement or
  disagreement
• with each of the following statements about the
  manner in
• which grading should be done in your courses.
• Teachers should
• Requested Advanced Preparation (RAP)
• provide study guides for the course material
• provide review sessions before major exams
• provide students with practice exams before
  regular exams
• Requested Post-hoc Grade Modification
  (RPGM)
• drop the lowest test score before calculating the
  final grade
• give students the chance to take another test if
  they did poorly on the original test
• allow students to do extra work in order to raise
  their final grade
• curve grades upwards when scores are low on an
  examination
• modify grades awarded to students whose
  circumstances (e.g., car trouble) may have
  interfered with their performance on tests
• count the last tests in a course more heavily
  than the first tests if there
• is steady improvement in exam scores
• make B the average grade in the course

9
Want strongly
Dont care
Dont want
10

• Grading fairness typically experienced in college
• 15 items (Cronbach Alpha .813) that assess
• Distributive justice
• Grades reflect what students learn in class
• Procedural justice
• Teachers will correct grading errors
• Interactional justice
• Teachers are unpleasant when discussing grades
  with students
• Overall justice
• Overall, teachers are fair when it comes to
  grading
• students

Background (control) variables - Sample
(business, chemistry) - GPA - Study time per
day on all courses
11
Regression Analysis
Standardized Model
Beta Significance Control variables
----- N.S. Actual grading
practices AAP .295 .000 APGM
.017 .736 Requested grading
practices RAP -.025 .609 RPGM
-.049 .319
Dependent variable Grading fairness
12
Part IIITest Construction and Administration
Rule No. 1 Students must recognize that your
procedures have afforded them every opportunity
to display what they know about the course
13
Test Construction
Students should be provided with - a review of
the material to be covered on the test - prior
explanation of the item formats that will be used
on the test - examples of proper answers if
students are unfamiliar with the test format -
prior sharing of the assessment criteria and
process
14
Test Construction - Rules for Writing Multiple
Choice Questions
Be wary of relying on publishers test item
banks - seldom prepared by the textbook
author(s) - often focus on rather trivial
material
The stem of the question should be written simply
and concisely - avoid superfluous information -
highlight words that change the sense of an
ordinarily affirmative statement - NOT or
not - e.g., Which of the following terms
does NOT belong with the other three? -
FALSE or false - e.g., Which of the
following statements is false? - EXCEPT or
except - e.g., All of the following are
characteristics of a bureaucratic organization
EXCEPT
15
Test Construction - Rules for Writing Multiple
Choice Questions
Response alternatives - list in alphabetic
order - make sure that all of the items are
spelled properly and have proper
punctuation - distractors should be plausible
but, nonetheless, incorrect - avoid all of the
above and none of the above - Haladyna
Downing, 1989
16
Test Construction - Rules for Writing Multiple
Choice Questions
Be careful not to provide unintended clues
Someone who starts a business is known as a.
entrepreneur. b. manager. c. transformational
leader. d. visionary.
a(n)
an
Bad item
Bad item
Planning a. involves identifying
organizational objectives and the steps
required to achieve them. b. is a management
skill. c. is not a management function. d. only
occurs at the top of the firm.
17
Test Construction - Sample Multiple Choice
Questions

• Taylor Mayo
• contingency approach organizational behavior
• scientific management human relations
• social man economic man
• systems contingency approach

• Which of the following statements about the
  Hawthorne studies is
• FALSE? The studies
• are associated with Harvards Elton Mayo.
• began by investigating the effects of
  illumination on productivity.
• gave rise to the concept of social man.
• generated interest in incentive (e.g., piecework)
  compensation.

18
Test Construction Short Essay Questions
Sample Compare and Contrast
Compare and contrast the following terms,
pointing out both similarities and differences
(5 points). E.g., Ham versus egg Both are
breakfast foods (1 point). Whereas ham comes
from a pig (2 points), eggs come from a chicken
(2 points).

• Comparable worth versus Equal Pay Act
• Both are legal issues pertaining to the
  compensation of female
• employees (1 point). Comparable worth is a
  concept that the pay of women in female-dominated
  jobs should be equal to the pay of men in
  male-dominated jobs if the two occupations are of
  equal worth.
• Although some public sector organizations have
  adopted this principle,
• comparable worth has not held sway in litigation
  involving private
• employers (2 points). The Equal Pay Act is
  enforceable statutory law
• that requires men and women who perform
  essentially the same job
• to receive the same pay (2 points).

19
Test Construction Short Essay Questions
Sample True-False Essay
Indicate whether the following statement is True
or False (1 point). Make sure that you
substantiate your answer by indicating the
reason(s) that you judged the statement to be
True or False (4 points).
All forms of employer discrimination when hiring
employees are illegal. False (1 point).
Discrimination is not illegal per se. As long as
employers make decisions about people on bases
that are job related, the discrimination is
legal (2 points). Problems arise, however, when
differentiation among people is not job related,
as when members of a protected class are
unfairly discriminated against (1 point). Note
that a Bona Fide Occupational Qualification
(BFOQ) allows a company to discriminate on the
basis of religion, gender, or age, but this is
because it has been determined that certain
individuals in these protected classes cannot
perform the job (1 point).
20
Test Construction - General Essay Questions
- bad essay question Discuss the following
proposition The Equal Pay Act requires
employers to pay men and women the same
wage. (15 points)
- good essay question What is the Equal Pay Act
and when was it passed? (4 points) To which
male and female employees does the Act apply?
(2 points) Are pay differentials between men and
women ever permitted under the terms of the
Act? (9 points)
21
Test Administration
Never administer an exam on a day that you cant
be present
Assure a quiet environment in which to take the
test
Start the exam at the appointed time
Allow students to ask questions during the exam
May entail accommodating the special needs of
certain students, including students with
learning disabilities and foreign students
Take steps to prevent cheating
Give make-up exam to students with legitimate
reasons for being unable to take the regularly
scheduled test
22
Part IVGrading Written Exercises
23
Grading Essay Questions
Mark each essay question separately for all
students rather than grading each students
entire paper before grading the next
students entire paper, etc. - despite having
to handle the papers more often, this process is
usually faster and more reliable
- reduces halo error

• regrade a few essays on papers that were marked
  first to assure
• consistency

24
Grading Writing Projects
Importance of clarifying the students
understanding about what the project entails
Meet with students - initial meeting is
mandatory - offer to meet again to discuss an
outline of their paper before they actually
write
Grade all of the projects at once
Check for plagiarizing - unevenness in the way
that the paper is written

• Turnitin an online plagiarism prevention
  system
• - www.turnitin.com

25
Grading Quantitative Work

• The following schedule of points will be awarded
  for each component of a good answer.
• Specify the null hypothesis(es) (H0 1 point
  each)
• Specify the research (alternative) hypothesis(es)
  (HA 1 point each)
• Write the formula for the appropriate test
  statistic (4 points)
• Correctly enter data values in the formula (2
  points)
• Correctly do the arithmetic to obtain the value
  of the test statistic (2 points)
• If appropriate, specify the formula for
  determining, and the actual value of, the degrees
  of freedom (2 points for each formula and actual
  value)
• Specify the critical region(s) for the test
  statistic (3 points)
• Correctly fail to reject the H0 or correctly
  accept the HA (1 point)

26
Sample Problem
In order to determine whether the average amount
of a sale is still 120, a store owner selected
a random sample of 12 sales invoices from the
population of current sales invoices. The
average sale in this sample was 112.85, and the
variance was 20.80. Using the .05 level of
statistical significance, is there any evidence
that the amount of the average sale has changed?

1
4
1
2
2
2
3
1
16
27
Part VProviding Feedback on Graded Exercises
28
Providing Feedback on Multiple Choice Questions
Lecture material used as the basis for a multiple
choice question

• Step 4 strategy formulation
• business strategy (p. 125) - defines the major
  actions used to build and strengthen competitive
  position
• identifies the role of functional areas in
  building competitive advantages
• responsibility of head of Strategic Business Unit
  (SBU)
• types of business strategy
• low-cost strategy - attempt to be efficient and
  offer a standard product
• differentiation strategy - attempt to be unique
  in its industry or market segment along some
  dimensions that customers value

29
Providing Feedback on Multiple Choice Questions
Tough question based on the previous lecture
material

• Determining the role of each major area in the
  organization (e.g.,
• marketing, quality assurance, and research and
  development) in
• building a competitive business advantage is an
  important aspect of
• business strategy.
• concentric diversification.
• corporate strategy
• functional strategy.

How to handle a poor question - increase
everybodys score by the value of the question -
keep percent correct letter-grade cutoffs the
same
30
Providing Feedback on Short Essay Questions
Read the answer you prepared and used as the
answer key Call on a student who received full
marks to read her/his answer aloud to the
class Ask students to carefully read the answer
key before requesting that you re-read their
papers Re-read any papers in the quiet of your
office or home, not at the end of a class with
students clamoring for more points Make any
corrections that are warranted Return papers
with appropriate notes
31
Part VITar Pits I Try to Avoid
32
Grading Problems I Try To Avoid
Dont overgrade - dont grade minor projects
- e.g., dont assign letter or numerical grade to
homework - satisfactory or redo
Simplify discussions with students who just
missed the next higher letter grade
33
Score f
Score f
Score f
Fall, 2003 185 students 49 different scores
8 breaks in the score distribution
34
Extra Credit Projects
I. Describe the external environment of a
specific organization.
II. Describe how a specific organization dealt
with a specific quality problem.
Purposes - learn how to analyze the environment
of a business - learn how the functions of
management are used - reduce the probability of
being embarrassed because you missed
being awarded a higher letter grade by
just a few points
35
How Extra Credit Projects Affect Grades
Distribution of Total Scores
Letter Grades
. 132 131 130 129 128 127 126 125 124 123 122 1
21 120 119 118 .
. A A B B A B B B B B B B B B B .
Total Score Exam 1 Exam 2 Exam 3
36
Grading Problems I Try To Avoid Practices
Requested by Students
Discount the Lowest Grade - weight it zero
(discard the lowest grade) - encourages
students to blow off one exam -
assumes that all measurement error is
negative

• weight it less than other grades if it is
• the first test
• - presumably to acknowledge genuine improvement
• -instructor should drop both the highest and
  lowest grades
• (akin to judging Olympic diving). The result
  (assuming that
• you have several other test scores) will probably
  be a better
• estimate of the student's 'true' level of
  ability.
• However, I don't advocate throwing away any data.

37
Grading Problems I Try To Avoid The Issue of
Student Participation
Rationale for grading participation - incentive
to come to class - encourage students to speak
in class - acknowledge students who are good
citizens by contributing to the class
Rationale for not grading participation - lack
of reliability system for recording
participation - validity facets of
participation that should be evaluated -
fairness cultural and/or disabilities that make
speaking up more burdensome
Does not refer to formal student classroom
presentations
38
Grading Problems I Try To Avoid Practices
Requested by Students
Second chances (a.k.a do over
or Mulligan)

• dont provide if students have the chance to
• take a make-up exam
• dont provide to one student unless you are
• prepared to offer the same opportunity
• to all other students who are dissatisfied
• with a grade

Extra work to improve a grade - see all of the
reasons for not providing a second chance
39
Grading Problems I Try To Avoid Practices
Requested by Students
Curving Grades
Formal definition Grading on a curve means that
a certain percentage of students receive each
grade e.g., 10 percent get As and so on.
Walvoord Anderson, 1998, p. 100
Student definition Generally is a plea for grades
that are higher than those deserved on the
basis of evaluated performance
40
So Be It
!