Evaluating Students in CS Courses

1
Evaluating Students in CS Courses
John C. Ramirez University of Pittsburgh Departmen
t of CS TA Training Lecture
2
Outline

• Why is evaluation necessary?
• What do we need to evaluate?
• Evaluation via assignments
• Evaluation via exams and quizzes
• Dealing with cheating

3
Reference

• Some of the material for this talk was taken
  from
• The online TA Handbook, Chapter 8 Evaluating
  Students, by Dr. Carol E. Baker
• For the complete handbook, please see
• http//www.pitt.edu/ciddeweb/FACULTY-DEVELOPMENT/
  TA_HANDBOOK/index.htm

4
Why is evaluation necessary?

• In an idealistic sense, grades should not matter
• It is the learning that counts
• Grades only create pressure and competition for
  the students
• Practically speaking, grades DO matter
• Used by prospective employers
• Used by graduate admissions committees
• Help students assess themselves

5
Why is evaluation necessary?

• To assign valid grades, teachers must evaluate
  the students
• The evaluation must be fair
• The evaluation must reflect the students'
  performance in an absolute or relative way
• If an absolute scale is used, students are
  evaluated based on some goals or objectives for
  the course
• Grade depends on how many objectives they meet or
  on how well they meet them
• There is the potential for all students to get As
  or for all students to fail
• This technique is good because you know all
  students who pass the course have a certain
  consistent amount of knowledge and/or ability

6
Why is evaluation necessary?

• If a relative scale is used, students are
  evaluated with respect to each other
• Grades are not so much dependent upon what the
  student knows/does but rather on how much more or
  less that student knows/does than the other
  students
• One way of using a relative scale is the "normal
  curve" (breaking the curve into 5 groups
  containing 3.6 As and Fs, 23.8 Bs and Ds,
  and 45.2 Cs)
• But this is NOT required and probably not
  generally a good way to do it
• A relative scale is good because
  assignments/exams and even teaching vary from
  term to term.
• A more difficult assignment or exam for one class
  will not necessarily lower all of their grades
• This is also good because it shows students how
  they compare to their classmates
• Prepares them for competition after college
• The University does not specify which technique
  to use it is up to the instructor

7
Why is evaluation necessary?

• My evaluation process is more or less a hybrid of
  the two
• I start with an absolute grade scale (90, 80, 70,
  60)
• I adjust that scale somewhat based of the scores
  of the students for the current term
• Cutoffs should be clearly defined
• Of the utmost importance is making it clear to
  the students how you grade early in the term,
  especially if it is "unconventional"
• If you are a grader or recitation TA, you will
  not be assigning the final grades
• If you are the course instructor, talk with
  someone on the GREAT committee if you want some
  guidance with assigning grades

8
What do we need to evaluate?

• What needs to be evaluated depends on the course
  goals and content
• Many CS courses have a programming component that
  is evaluated through programming assignments
• Many CS courses have a theoretical component that
  is evaluated through problem sets
• Most CS courses also require examinations and/or
  quizzes
• Evaluation in a more regulated environment

9
Evaluation via assignments

• Programming Assignments
• Purposes
• Demonstrate knowledge/ability with some newly
  learned programming language
• Demonstrate logical reasoning and problem-solving
• Ex CS 0004, CS 0007, CS 0132, CS 0401, CS 0445
• Demonstrate knowledge/ability of some new
  algorithm/concept pertinent to the course
• Demonstrate advanced problem solving abilities by
  solving difficult tasks via programs
• Ex CS 1501, CS 1520, CS 1541, CS 1550, CS 1555,
  etc.

10
Evaluation via assignments

• Design
• Programming assignments should be carefully
  thought out by the instructor
• Keep in mind that most classes contain students
  with a range of abilities
• Assignments should not be so easy that the better
  students will be bored/uninterested
• Assignments should not be so hard that the
  majority of the class is unable to finish them
• Always realize what is being covered currently in
  class
• Assignments should be pertinent in some way to
  the current course material
• Often you can find good ideas in textbooks (don't
  limit yourself to the text for the course)
• Don't necessarily take the assignments exactly as
  they appear you may modify them to more
  specifically fit your class

11
Evaluation via assignments

• Try to make the assignments interesting to the
  students
• Students will be more likely to try them and to
  learn if it is something fun/interesting
• Be as specific as necessary for the assignment
  requirements
• More specific requirements are usually necessary
  for introductory courses
• More advanced courses typically allow more
  flexibility
• Also clearly state what you expect the students
  to submit, how they are to submit it, and when it
  is due
• For example, from CS 0445 (C), after covering
  dynamic memory and linked-lists, and from CS
  1520, after covering Graphics and Threads

12
Evaluation via assignments

• Grading
• Grading of assignments should be precise and
  unambiguous
• Establish grading criteria in advance and grade
  all assignments consistently using those criteria
• It is a good idea to make a grading sheet with
  each of the program requirements and points for
  each
• It is also a good idea to have students submit an
  "information sheet" to help you grade
• When taking off points, indicate the reason on
  the grading sheet (or perhaps on a hard copy of
  the code)
• If grading must be divided between 2 or more
  people, you should discuss the grading policies
  in detail to grade as consistently as possible
• Email or talk about unforeseen problems and/or
  issues so that everyone handles them in the same
  way

13
Evaluation via assignments

• Written Assignments/Problem Sets
• Purposes
• Enable students to demonstrate their knowledge
  and problem-solving abilities in courses that are
  not oriented toward programming or that contain a
  theoretical component
• Ex CS 0441, CS 1502, CS 1511
• Some courses may have both programming and
  written assignments
• Design
• The design of these assignments will vary greatly
  depending upon the course
• Some may be highly mathematical, some may be more
  algorithmic

14
Evaluation via assignments

• These can sometimes be quite difficult to "make
  up"
• Most textbooks provide Instructors' copies that
  provide problem sets
• Many faculty who teach these courses also have
  old assignments that can be partially reused
• Grading
• The grading for written assignments should follow
  the same guidelines as programming assignments
• The key is to grade fairly and consistently and
  to clearly and indicate what the students have
  done incorrectly

15
Evaluation via exams and quizzes

• Exams
• Exams have become a standard part of most college
  courses
• Exams allow instructors to evaluate students in a
  (somewhat) standard way
• They also allow instructors to compare students
  to one another in a more structured way than
  assignments
• For CS courses, they also have a lower
  probability of students' cheating
• See next section for cheating discussion

16
Evaluation via exams and quizzes

• Frequency
• Exam frequency depends a lot on the amount of
  material covered and the nature of the material
  covered
• If a lot of material is covered quickly, or if
  the material covered is not inter-related, more
  exams are desirable
• If material is covered more slowly (perhaps due
  to it being more difficult) or if it builds upon
  the same foundation, fewer exams are required
• As a TA, you should follow the exam guidelines
  for the course
• See Course Gatekeeper if you are unsure

17
Evaluation via exams and quizzes

• Design
• CS Exam structures can vary widely depending upon
  the content of the course
• A good idea is to get some old exams from the
  course you are teaching to get an idea of the
  types of questions to ask
• See faculty and/or TAs who taught the course
  previously
• If you eventually make up an exam from scratch,
  put a lot of thought and care into it. See TA
  Handbook for ideas and also consider the
  following issues
• Is the length of the exam appropriate?
• Different question types take different amounts
  of time
• Sometimes it is good to have someone else look at
  it or you can "take it" yourself to determine
  this
• If most of the class is still there at the end,
  it was probably too long

18
Evaluation via exams and quizzes

• What exactly is each question trying to test?
• Are you testing knowledge, analytical thinking,
  programming skill, memorization skill?
• Does the student need to know the material to
  answer the question correctly?
• Can the questions be graded easily and
  effectively?
• If a question is ambiguous it may have more than
  one "correct" answer. In this case you may end
  up giving most people credit or throwing it out
  altogether.
• If a question is very difficult to grade (ex.
  some programming problems, long technical essays
  and proofs) it gets frustrating to grade and you
  end up either grading very easily (don't have to
  read it too closely) or very harshly (no partial
  credit just right or wrong).
• Does the exam have the appropriate depth and
  breadth?
• Depth Are the topics covered in appropriate
  detail?
• Breadth Is the course content covered
  appropriately?
• Clearly you should devote more of the exam to the
  topics you covered most in class. A common
  student complaint "Nothing he told us to study
  was on the exam!".

19
Evaluation via exams and quizzes

• Avoid giving multiple versions (with different
  questions) of exams to the same class
• You may think they are equivalent, but one
  version may be more difficult than the other, and
  this is not fair to the students with the more
  difficult version.
• If you are afraid of cheating within a class
  during the exam, create different "versions" by
  simply mixing up the pages and by printing the
  exams on different color paper.
• Giving different exams to different sections of
  the same course is ok, as long as you grade each
  section independently
• Avoid giving optional questions (ex answer any 3
  out of the following questions)
• Again it is an issue of fairness. Students who
  pick the "more difficult" questions will be at a
  disadvantage
• Avoid giving extra credit questions
• Students take exams at different paces. Giving
  extra credit questions rewards the students who
  are "faster" in taking exams
• If you want to give students extra credit, make
  it in an assignment or extra project that gives
  all students an equal opportunity to complete it

20
Evaluation via exams and quizzes

• Grading
• As with assignments, you should determine a
  points breakdown for each exam question before
  you begin to grade
• Follow the breakdown consistently for all exams
• Some questions are, by nature, more difficult to
  grade than others
• For example, programming questions, proofs and
  essays can be difficult to grade
• Determine in advance what if any partial credit
  applies to these questions, and be consistent
  with it
• Try to grade each question across all of the
  exams so you can be more consistent in the
  grading
• After grading, go back and re-examine some
  (especially the first few) exams to make sure
  your grading did not change as you progressed
  through the exams

21
Evaluation via exams and quizzes

• Take breaks while grading the exams
• When you are tired and/or frustrated you tend to
  make more grading mistakes
• Double-check point totals when adding final
  scores
• Here is my technique For each incorrect answer
  or part of an answer I indicate how many points
  are lost and why. I also indicate the total
  points achieved for each question. I then add
  all of the points achieved together for the
  overall score. Finally, I subtract all of the
  points lost from 100 to get the overall score
  again. If the results do not match, I know I
  have made a mistake somewhere
• Either correct students' exams or make up a
  solution key for the exam
• Students need to know the correct answers so they
  can learn from their mistakes
• If Final Exam is cumulative, they can use
  previous exams' answer keys as study aides

22
Evaluation via exams and quizzes

• Quizzes
• Purpose
• Quizzes are "mini" exams typically used to keep
  the students current with material and/or to
  reduce the number of exams required
• Typically given in just a few minutes at the
  beginning or end of a class
• Guidelines
• Design and grading of quizzes will be similar to
  that of exams, but on a smaller scale
• Not all courses will have quizzes
• Check with the course Gatekeeper to see if you
  should give them or not

23
Dealing with cheating

• Clearly state your policy
• Have a written policy on cheating (give it out
  and keep it online)
• Clearly explain it in the first class
• Remind students of it when pertinent
• Follow it consistently
• If you are a TA for a course, check with the
  instructor for the policy
• Finding violations
• This is often difficult to do

24
Dealing with cheating

• If you worry too much about finding cheating
  violators, you will never finish grading an
  assignment
• Don't feel guilty if you don't catch every
  incident
• Most violations are caught "by accident"
• Look for incorrect submissions that are the same
• Same logic error or run-time error
• These tend to catch your attention
• With programming assignments, the "global
  replace" is a common way of cheating
• But the structure of the code is still clearly
  the same
• Output and logic errors (if any) will be the same

25
Dealing with cheating

• Handling violations
• If you are a TA, talk with the course instructor
  about violations
• Show offending work(s) to the instructor and let
  him/her make the decision
• If students complain or want to argue about it,
  send them to the course instructor
• If you are teaching a course, you must make the
  decision
• However, you can stop in to see anyone on the
  GREAT committee if you want some guidance
• Any questions?