Making Sense of Assessing Student Learning Outcomes

1
Making Sense of Assessing Student Learning
Outcomes

• Southampton, UK
• June 26, 2008
• Randy Swing, Executive Director
• Association for Institutional Research

2
Assessment U.S. Context

• Not grading (in US)
• (but may use grades)
• Quality Assurance
• Program Improvement

3
Successful Assessment....
Creates Action

• To....
• Continue Effective Practice or
• Initiate Change (improvement)

4
FAQ Why do we need assessments beyond grading?

• U.S. Context
• Grades are a composite measure
• Learning
• Attendance
• Compliance with class rules
• Seldom validated
• More

5
Grades A Valuable Data Source?

• Researchers Dream
• Vary across and among participants
• Continuous variables in standard increments
• Readily available
• Widely held to be important indicators of
  success
• Researchers Nightmare
• Volunteer effect
• Unique treatments
• No random assignment not experimental design

6
Grades Matter!

• Major Predictor
• Persistence into the Sophomore Year
• Graduation

First Year GPA Bachelors in 6 yrs Less than
2.25 26 2.25 3.25 64 Over
3.25 75
Source Descriptive Summary of 1995-1996
Beginning Postsecondary Students Six Years Later
(NCES 2003-151)
7
How much failure should we tolerate?

• Identify the 5 courses that annually enroll the
  largest number of new students.
• Determine the DFWI rate for each course.

8
DFWI Rate
Five Courses with Highest Enrollment of New
Students

• Average DFWI Rate (percent of students)
• 25 (4-yr) 35 (2-yr)
• Range of DFWI Rates (percent of students)
• 1 to 79 (4-yr) 9 to 58 (2-yr)
• Percent of Courses Over 33 DFWI
• 25 (4-yr) 59 (2-yr)

Source Kathy Morley and Julie Alexander, Policy
Center on the First Year of College unpublished
2006
9
Alternatives to Grades(Selected examples)

• Standardized tests
• Admissions tests (SAT and ACT)
• Collegiate Learning Assessment (CLA)
• ETS Major Tests
• State Junior/Exit testing
• Proxy Measures
• National Survey of Student Engagement

10
Model for Assessing Learning in a Course
Outcomes
Out of Class Environments
StudentEngagement
Student Prior Experiences Characteristics At
Entry
CourseContents
Course Delivery Structures
Course Goals
Randy L. Swing, 2003 - Adapted from works by
Astin and Terenzini
11
Contact Information

• Randy L. Swing, Ph.D.
• Executive Director,
• Association for Institutional Research
• 1435 E. Piedmont Drive
• Tallahassee, FL 32309
• rswing_at_airweb2.org