Assessing Learning Outcomes in Programs Large and Small

1
Assessing Learning Outcomes in Programs Large and
Small

• Sue F. Phelps
• Linda Frederiksen

2
Introduction
3
Campus Learning Goals

• Critical Thinking
• Quantitative and Symbolic Reasoning
• Information Literacy
• Communication
• Self in Society
• Specialty

4
Assessment of Student Learning

• Assessment works best when the programs it seeks
  to improve have clear, explicitly stated
  purposes.
• Assessment works best when it is ongoing not
  episodic.
• Through assessment, educators meet
  responsibilities to students and to the public.
• American Association for Higher Education (AAHE)

5
ACRL Standards

• Standard One
• The information literate student determines the
  nature and extent of the information needed.
• Standard Two
• The information literate student accesses needed
  information effectively and efficiently.

6
Information Literacy Learning Goal

• Standard Five Example
• WSU Vancouver Website for Workshop
• http//www.vancouver.wsu.edu/vis/lib/acrlmidwinter
  08/

7
Rubrics Defined

• Rubrics are rating scales- as opposed to
  checklists- that are used with performance
  assessments. They are formally defined as scoring
  guides, consisting of specific pre-established
  performance criteria, used in evaluating student
  work on performance assessments. (Mertler, Craig
  A., 2001)
• Holistic and Analytical Rubrics

8
Step by Step Guide to Making a Rubric

• Define the task which the rubric will assess
• List the criteria that describes the parts of the
  task
• Describe how the beginning, intermediate and
  accomplished person would perform each part of
  the task. Use brief examples to illustrate, if
  necessary.
• Revise and refine as needed.
• (Adapted from Mertler, 2001)

9
Tips

• The language needs to be precise and free of
  jargon
• The criterion needs to be observable
• Easiest to describe the best and the worst first
• Use your experience of student work
• For qualitative outcomes, assign one or more
  numbers to each level
• (Adapted from Mertler, 2001)

10
Exercise

• Each table will be given an ACRL Standard
• Make a rubric based on this Standard
• Identify 3 main levels, beginning, intermediate,
  and advanced
• Identify two levels for each main level
• Be sure to include an observable criterion for
  each performance indicator within each main level
• Standard Five Example

11
Rubric Discussion

• Could a non-librarian use your rubric to assess
  student information literacy learning?

12
Using a Rubric for Assessment

• Collect student work
• Collaborate with teaching faculty to use student
  work from classes where bibliographic instruction
  is taught
• Writing Center
• Writing portfolios
• Institutional Repository
• Permissions for use of student work

13
Successful Rubric Use

• Raters
• Adopt the values of the Rubric
• Revisit the criteria to maintain consistency
• Consider their own biases
• Are adequately trained
• Are aware of other factors affecting judgment
• Have been involved in creating the rubric

14
Challenges

• Raters often want to evaluate work more
  holistically.
• Outcomes-based assessment is unfamiliar to many.
• Consistency of understanding and interpretation
  of rubric.
• Writing clear, precise rubric criteria.
• Difficulties with artifacts

15
Rater Exercise

• Pass your rubric to the next table.
• Each person at the table rates the artifact
  according to the rubric.
• The table needs to come to an agreement on the
  score.

16
Discussion

• How hard was it to use the rubric to rate the
  work?
• Were you able to come to a common understanding?

17
Closing the Feedback Loop

• Organizational commitment to change
• What are you doing with it
• One-time snapshot of skills
• Measure progress
• Share the results
• Rater respect

18
Discussion of Implementation

• How will you implement this?
• Barriers at your organization?
• Are there ways to get student work (is it already
  being collected)?
• Who needs to be involved?

19
Conclusion

• Final Questions
• Wrap up
• Look for your work on the website
• http//www.vancouver.wsu.edu/vis/lib/acrlmidwinter
  08/
• Thank you for attending and have a good
  conference!

20
Assessing Learning Outcomes in Programs Large and
Small

• Sue F. Phelps
• Linda Frederiksen