Designing a Course for Evaluation:

1
Designing a Course for Evaluation
Triangulation of assessment to improve teaching
in a single course section

• Richard W. Stratton
• Economics Department
• The University of Akron
• November 7, 2003

2
Begin at the beginning

• How could I design a course that would allow me
  to evaluate its effectiveness objectively?

3
Begin at the beginning

• Set specific goals and objectives
• Identify course components
• Identify assessment tools
• Evaluate success

4
Set specific outcome goals

• Mine were based on Hansens proficiencies (2001
  JEE 32232-233)
• access existing knowledge
• demonstrate command of knowledge
• interpret knowledge
• find and interpret existing economic data
• apply existing knowledge

5
Set specific outcome goals

• Objectives (cont.)
• demonstrate their ability to apply economic
  knowledge
• effectively employ select analytical and critical
  thinking tools used by economists
• demonstrate their ability to apply these tools
• persist to the end of the semester

6
Two objectives

• Students will be able to write a well reasoned
  supporting a policy recommendation
• Students will demonstrate ability to work
  cooperatively with others (Teams)

7
Design course components to address goals and
objectives

• Worksheets prior to class
• Team work in class
• Participation in class
• CAT in class
• Essays after class
• Exams after class

8
Criterion basedMastery influenced

• Worksheets
• Pre-class encounter
• Basis of team exercises
• CAT student assessment of class
• Essays
• Rubric based grading
• Exams

9
How well did I do?

• Did Retention improve?
• no - 50
• Did the grade distribution improve?
• Yes - 40 of completers A or B

10
How well did I do?

• Analysis of Final Essay
• Used rubric to determine performance

11
How well did I do?

• Learning Gains
• Pre-test / Post-test
• Richard Hakes normalized gain
• (Post Pre)/(40 Pre)
• Average 13 gain
• 8 students more than 1 s.d. above 0
• 12 students within 1 s.d. of 0

12
How well did I do?

• Student CATs
• Increase rating of all three components
• 5 point scale
• Worksheets 2.48 to 2.88 sig. 1
• Essays 2.29 to 2.35 ns
• Class time 3.24 to 3.35 ns