The Problem with Exams M. Selen, UIUC Physics

1
The Problem with ExamsM. Selen, UIUC Physics

• Why exams are a sensitive issue
• Hand graded versus multiple choice.
• How we do it in the intro Physics courses.
• What is the cost?

2

• Exams typically represent gt 50 of the final
  course grade.
• Students really focus on exams (perhaps too
  much).
• Any perceived problems (real or imagined) will
  generate enormous resentment.
• Faculty often dont focus on exams.
• Doing it right is time consuming, in particular
  of you are working alone.
• Fairness is not a trivial issue.
• Exam content.
• Grading.

3
Hand Graded vs. Multiple Choice

• Hand Grading (what we used to do)
• PROS
• Students feel like everything they write is taken
  into account (i.e. partial credit).
• Any mistakes in the exam can be adapted to (exam
  is more forgiving to the professorless quality
  control is needed).
• CONS
• Hard to make grading fair consistent.
• TA, Handwriting, time, before/after pizza etc..
• Whiners are rewarded (i.e. re-grades).

4

• Multiple Choice (what we do now)
• PROS
• Uniform Fair. Everyone is treated exactly the
  same.
• Fixing a bug in a problem help everyone.
• Lends itself to electronic publishing.
• WEB interface possible for practice (before exam
  night) and help/explanations (after exam).
• Very useful for analysis.
• tracking changes, education research, exam
  problem quality control, problem bank etc.
• CONS
• Harder to give partial credit (but not
  impossible).
• More care is needed when preparing exam.
• Considered inferior by some (mostly faculty).

5
About 1/3 of exam score is conceptual(2 3
choice)
Quantitative problems(5-choice) allow students
to select up to 3 answers. Partial credit !
Conceptual and quantitative problems are often
paired.
6

• Analysis of exam data is very interesting (and
  useful for education research).

Physics 101 Midterm Exam 1, Spring 2000
7
More sophisticated analyses can be used to rate
the effectiveness of each exam questions
8
We can look at the discriminating power of each
problem
We can learn, quantitatively, how to build better
exams.
9
Instant exam feedback is possible

• The minute they leave the exam, students can go
  on the web, enter their answers into a web
  version of the exam they just took, and see what
  their raw score is
• After the exam has been graded (next day)
  students can find detailed statistics on each
  problem on the web.

Students LOVE this !
10
The Cost (is it more work?)

• These courses are team-taught.
• Typically 3 faculty
• Lecturer, Discussion director, Lab director
• The team course works together to produce exam.
• The subject material is divided up and each
  faculty submits a set of problems.
• Old problems can be used for guidance.
• Format is fixed (MS word in our case).
• One of the faculty is in charge of assembling
  exam (secretarial staff can help).
• The team meets several times to discuss iterate
  the problems until the final draft is ready.
• A senior TA works the exam out and provides
  comments on difficulty, length etc.
• Three version of exam are produced by scrambling
  the order of the problems.
• Secretaries do this.
• After the exam, OIR machine reads the scantron
  forms and gives us the raw data. We have
  scripts that do the final analysis to yield
  grades etc.

11

• We are getting good at this. Exam averages are
  consistently 70-75.
• Curving is unusual.
• We never curve down.
• We can honestly tell students that they are not
  competing against each-other.
• Everyone could, in principle, get an A in
  Physics 11x.
• This is a great motivator.
• We can tell students on the first day of the
  semester what final semester score they need to
  get the various letter grades