Full-Cycle Assessment of Critical Thinking in a General Education Seminar

1
Full-Cycle Assessment of Critical Thinking in a
General Education Seminar
(Critical Thinking Assessment)

• Jennifer Blue
• Beverley Taylor
• Jan Yarrison-Rice
• Department of Physics, Miami University, Oxford,
  Ohio

2
Ethics Science Course

• First-Year Seminar at Miami University
• first semester first-year students
• Topic Ethics Science
• A case-study approach
• Lots of discussion, small group work, debates and
  written assignments
• Improvement of critical thinking skills is an
  explicit goal of the course

3
History of This Project

• Assessment project initiated mid-semester Fall,
  2004
• IRB approval sought
• Each semester about half of students consented to
  have their work used
• After course was over the Assessment Fellows read
  and assessed three papers per student

4
Miami Univ. Assessment Fellows

• Cross-disciplinary group formed in Fall, 2003
• Read about critical thinking
• Tried to define critical thinking
• Created lists of traits of critical thinking
• Created a variety of rubrics to assess critical
  thinking
• Finally settled on the Washington State rubric

5
Washington State CT Project
wsuctproject.wsu.edu
6
Washington State CT Rubric

1. Identify and present the problem/question
2. Identify and present students own perspective
   and position
3. Identify and present others perspectives and
   positions
4. Identify and assess key assumptions
5. Identify and assess supporting data/evidence
6. Consider influence of context
7. Identify and assess implications and consequences

7
Activity 1

• Use the rubric to assess Example 1.
• Concentrate on traits 2 and 7
• You have 10 minutes.
• Feel free to talk to your neighbors.

Scant (1) Min. dev. (2) Mod dev. (3) Sub. Dev. (4)
Students own position
Conclusion
8
Results from Year 1 - Averages
Early assignment Midterm
Identify problem 1.8 2.76
Student perspective 1.48 2.14
Other perspectives 1.32 2.10
Assumptions 1.24 1.67
Using evidence 1.2 1.71
Context 1.32 1.86
Conclusion 1.28 1.95
9
Year 1 - Frequency Data
10
Year 1 - Frequency Data
11
Modifying Assignments for Year 2
We decided that we needed

• More focused assignments
• To specifically teach critical thinking skills
• Assignments which
• Broke down CT skills to smaller portions
• Built from lower level skills to more difficult
  ones
• A grading rubric for each assignment to give
  students in advance
• Some assignments looking for all skills to see
  progress

12
Activity 2

• Use the rubric to assess Example 2.
• Concentrate on traits 1 and 6
• You have 10 minutes.
• Feel free to talk to your neighbors.

Scant (1) Min. dev. (2) Mod dev. (3) Sub. Dev. (4)
Problem
Context
13
Results from Year 2 - Averages
Early assignment Midterm
Identify problem 1.5 2.5
Student perspective 1.8 2.5
Other perspectives 1.5 2.1
Assumptions 1.2 1.7
Using evidence 1.4 1.8
Context 1.5 1.8
Conclusion 1.5 1.9
14
Year 2 - Frequency Data
15
Year 2 - Frequency Data
16
Compare Years 1 and 2

• Papers show improvement from early to later in
  the semester in both years
• Re-writing and use of Peer Review was significant
  in producing that change
• Instructor more comfortable with grading
• No real improvement in critical thinking from
  2004 to 2005
• Students felt the rubrics were very helpful as
  they worked on their essays

17
Student Perceptions of Rubrics
(1) Strongly Agree to (6) Strongly Disagree
Having rubric before wrote papers useful 1.12
Having rubric before wrote papers helped organize writing 1.24
Having rubric helped understand my grade 1.88
Rubrics helped incorporate critical thinking into writing 2.35
18
Discussion

• How do you see this rubric working in your own
  classes?
• Will you need to modify the rubric for your
  course? How?
• Other ideas?

19
Ideas for Next Iteration of Course

• Use of bulleted lists for identifying particular
  critical thinking traits a quick easy way to
  assess how students are doing
• A few assignments throughout the semester in
  which students draw evidence together from more
  than one case study
• Next offering will be to second semester
  first-year students will student CT skills be
  different after 1 semester in college?
• Your ideas?

20
Contact Information

• Jennifer Blue, bluejm_at_muohio.edu
• Beverley Taylor, taylorba_at_muohio.edu
• Jan Yarrison-Rice, yarrisjm_at_muohio.edu
• Thank you!