Incorporating Assessment into Program Review

1
Incorporating Assessment into Program Review

• Dr. Maura Stevenson
• Ms. Mary Kate Quinlan
• Community College of Allegheny County
• Pittsburgh, PA

2
Session Objectives

• Participants will be able to
• Identify the steps needed to incorporate
  assessment into a community college program
  review process
• Create a plan to implement program assessment
• Determine the effectiveness of our process based
  on evidence presented

3
CCAC Facts and Figures

• 30,773 credit students
• Multi-campus 4 campuses 7 centers
• About 86 programs
• Unionized faculty
• Governance process

4
Assessment at CCAC

• Assessment of Student Learning Committee (ASL)
• Faculty led
• Support and participation by Administrators
• 5 - Faculty liaisons
• One for each campus
• One for student services
• Assessment focus
• Year 1 classroom
• Year 2 program general education

5
(No Transcript)
6
Embed Assessment into Existing Processes
7
Why We Chose Program Review

• Its required (annual and 5-year reviews)
• New Academic Affairs leadership valued the
  process
• Assessment was actually a requirement (but
  widely ignored)
• Credibility approved through governance
  (supported by union)
• Process in existence for 12 years (4th iteration)

8
Key Steps

• ASL recommended changes to the program review
  process to strengthen the assessment components
• Annual - program assessment
• 5-year - General education goal alignment

9
Key Steps

• ASL forwarded proposal to Academic Planning
  subcommittee
• The initiator distributes the proposal to the
  entire college community
• The subcommittee conducts a hearing and may
  recommend changes before deciding to forward the
  document to the full council

10
Key Steps

• College Council reviewed proposal and made the
  decision to forward to the president for approval
  
• The president approved the proposal

11
CCACs Experience

• The process went very quickly (about 7 weeks)
• College was under pressure by accrediting agency
  to conduct assessment at the program level
• Full ASL committee support and presence at the
  hearing
• Few people questioned the changes

12
Create a Process Thats Beneficial to Participants
13
Our Approach

• ASL is a resource to faculty conducting program
  assessment
• Develop tools
• Train the trainers deans and department heads
• Answer questions
• The academic deans maintained authority

14
3 Step Process

• Pick one program objective
• Pick two measures (one direct and one indirect)
• Discuss the results
• Based on Barbara Walvoords no-frills assessment
  plans

15
Timeline

• Fall semester (week prior to start of class)
  faculty meet to develop program objectives and
  develop assessment plan
• Submit plans by September 15th
• Fall semester faculty conduct assessment
• Spring semester (week prior to start of class)
  faculty discuss assessment results
• End of spring semester faculty submit a program
  assessment report on the use of results

16
Benefits of This Method

• Authentic utilizes whats happening in the
  classroom (not an add on)
• Flexible faculty choose what, who, and when
• Yields useful results
• Faculty worked together across campuses
• Focused on common values

17
The Process (in detail)
18
1. Start with Goals

• Our Hierarchy
• General education goals
• Program objectives
• Course outcomes
• Handout
• Worksheet writing program objectives

19
2. Create a Plan

• Handouts
• A Guide to Program Assessment for Faculty
• Definitions
• Examples
• Resources
• Sample Plans
• Good and bad fictional examples
• Tips for writing a good plan

20
3. Defining Expectations

• Handout
• Rubric for program assessment plans
• Benefits
• Clarifies expectations for participants
• Reduces subjectivity when evaluating plans
• Issue inter-rater reliability

21
4. Keep People on Track

• Handout
• Checklist
• Benefits
• A gentle reminder of required tasks
• Helps the deans manage the process
• Another opportunity for ASL faculty liaisons to
  make contact with instructors on their campus
• Sends a message assessment isnt going away

22
5. Collecting Reports

• Handouts
• Program Assessment Report Template
• Sample Report

23
Our Results (so far) Rubric Scores

• Received 59 plans
• No plans from13 programs without FT faculty (15)
• 14 unaccounted for (16)
• Average score 14 out of 15
• Areas of strength 1st and 2nd measures (3.0)
• Area of weakness statement of method (2.62)

24
Lessons Learned

• Rubric practice scoring to clarify
  understanding
• Clarify liaison role
• Discovered programs without FT faculty
• Forgot to require assessment criteria did not
  ask them to establish an acceptable level of
  performance
• Current debate should we have required writing
  5 program objectives at once rather than one at a
  time?

25
Contact

• Maura Stevenson
• mstevenson_at_ccac.edu
• Mary Kate Quinlan
• mquinlan_at_ccac.edu
• Our website
• www.ccac.edu/assessment