Title: The Problem with Exams M. Selen, UIUC Physics
1The Problem with ExamsM. Selen, UIUC Physics
- Why exams are a sensitive issue
- Hand graded versus multiple choice.
- How we do it in the intro Physics courses.
- What is the cost?
2- Exams typically represent gt 50 of the final
course grade. - Students really focus on exams (perhaps too
much). - Any perceived problems (real or imagined) will
generate enormous resentment. - Faculty often dont focus on exams.
- Doing it right is time consuming, in particular
of you are working alone. - Fairness is not a trivial issue.
- Exam content.
- Grading.
3Hand Graded vs. Multiple Choice
- Hand Grading (what we used to do)
- PROS
- Students feel like everything they write is taken
into account (i.e. partial credit). - Any mistakes in the exam can be adapted to (exam
is more forgiving to the professorless quality
control is needed). - CONS
- Hard to make grading fair consistent.
- TA, Handwriting, time, before/after pizza etc..
- Whiners are rewarded (i.e. re-grades).
4- Multiple Choice (what we do now)
- PROS
- Uniform Fair. Everyone is treated exactly the
same. - Fixing a bug in a problem help everyone.
- Lends itself to electronic publishing.
- WEB interface possible for practice (before exam
night) and help/explanations (after exam). - Very useful for analysis.
- tracking changes, education research, exam
problem quality control, problem bank etc. - CONS
- Harder to give partial credit (but not
impossible). - More care is needed when preparing exam.
- Considered inferior by some (mostly faculty).
5About 1/3 of exam score is conceptual(2 3
choice)
Quantitative problems(5-choice) allow students
to select up to 3 answers. Partial credit !
Conceptual and quantitative problems are often
paired.
6- Analysis of exam data is very interesting (and
useful for education research).
Physics 101 Midterm Exam 1, Spring 2000
7More sophisticated analyses can be used to rate
the effectiveness of each exam questions
8We can look at the discriminating power of each
problem
We can learn, quantitatively, how to build better
exams.
9Instant exam feedback is possible
- The minute they leave the exam, students can go
on the web, enter their answers into a web
version of the exam they just took, and see what
their raw score is - After the exam has been graded (next day)
students can find detailed statistics on each
problem on the web.
Students LOVE this !
10The Cost (is it more work?)
- These courses are team-taught.
- Typically 3 faculty
- Lecturer, Discussion director, Lab director
- The team course works together to produce exam.
- The subject material is divided up and each
faculty submits a set of problems. - Old problems can be used for guidance.
- Format is fixed (MS word in our case).
- One of the faculty is in charge of assembling
exam (secretarial staff can help). - The team meets several times to discuss iterate
the problems until the final draft is ready. - A senior TA works the exam out and provides
comments on difficulty, length etc. - Three version of exam are produced by scrambling
the order of the problems. - Secretaries do this.
- After the exam, OIR machine reads the scantron
forms and gives us the raw data. We have
scripts that do the final analysis to yield
grades etc.
11- We are getting good at this. Exam averages are
consistently 70-75. - Curving is unusual.
- We never curve down.
- We can honestly tell students that they are not
competing against each-other. - Everyone could, in principle, get an A in
Physics 11x. - This is a great motivator.
- We can tell students on the first day of the
semester what final semester score they need to
get the various letter grades