Using Portfolios to Assess Undergraduate Learning Outcomes

1
Using Portfolios to Assess Undergraduate Learning
Outcomes
Peter Keller, Brian Loher,Margaret Launius,
Francis Craig, Nancy Cooledge Department of
Psychology
2
What Makes a Portfolio?

• Purposeful collection of work
• Criteria for selection
• Criteria for judging merit

3
Benefits of Portfolio Assessment

• Student responsibility for learning
• Collaboration within and across disciplines
• Linking assessment and instruction
• Public accountability

4
Pitfalls of Portfolio Assessment

• Too many learning objectives
• Unclear directions
• Time involved
• Scoring

5
The Mansfield Portfolio in Context
PSY 1101 Intro to Psychology
PSY 1151 First Year Seminar

Core Requirements of Psychology Major
Courses identify outcomes projects suitable for
portfolio
Portfolio submissionExit examinationPublic
presentations
PSY 490, Senior Seminar
Assessment of Outcomesand Program Revision
Sample of portfolios evaluatedResults reviewed
at retreatProgram changes considered
6
PSY 1151 First Year Seminar

• Describe portfolio purpose and process
• Introduce and review portfolio materials in
  course/ department manual
• Illustrate portfolio structure and show examples
  of completed portfolio
• Answer questions on mechanics of producing
  portfolio
• Assignment Cover, spine, sections labeled, and
  Career Planning section includes initial Goals
  Statement Form and resume (both from PSY 1151)

7
Learning from the Portfolio Process
PSY 4490, Senior Seminar

• What is the instructors role as portfolio
  completion nears?
• Early review guidance
• Standardized format versus creative freedom
• Student-directed versus instructor-directed
• Graded versus not graded

8
Learning from the Portfolio Process

• How to assess the completed portfolio Varied
  purposes
• Peer assessment
• Instructor assessment
• Faculty assessment
• Reviewing all or some
• What to do with the completed portfolio
• Keep versus return to students

9
Reviewing Portfolios

• Preparing the portfolios for evaluation
• PSY 4490 Senior Seminar Evaluation sheet
• Sampling the portfolios
• Quantifying the outcomes
• Reviewers checklist
• 5 dimensions and a global rating Comments
• Analyzing the reviewers ratings
• Average ratings
• Inter-rater reliability
• Reviewers Comments

10
Reviewing Portfolios

• Utilizing the assessment results for enhancing
  the program
• Improving consistency in utilizing correct APA
  style across courses
• Clarifying our expectations of each other and our
  students
• Identifying areas in program not covered
  sufficiently
• Ethics, diversity, etc.
• Making curricular Changes

11
Reviewing Portfolios

• Preparing the portfolios for evaluation
• PSY 4490, Senior Seminar Evaluation sheet
• Quantifying the outcomes
• Review committee process
• Reviewers checklist

12
Reviewing Portfolios

• Inter-rater reliability
• 1998 2000
• Utilizing the assessment results for enhancing
  the program
• Improving consistency in utilizing correct APA
  style across courses
• Clarifying our expectations of each other and our
  students
• Identifying areas in program not covered
  sufficiently
• Ethics, diversity, etc.