Beyond Program Review

1
Beyond Program Review

• Program Overview and Forecast
• PrOF

2
Program Overview and Forecast

• Central Questions
• How to link Student Learning Outcomes into
  Program Review and Planning processes
• Program definitions roles and functions

3
Program Overview and Forecast

• Key Issues
• Program SLOs
• Identification of Outcomes
• Assessment Process SLOs
• Development of a Response Plan
• Alignment of courses to Program SLOs
• Identifying Program achievements and challenges

4
Program Overviewand Forecast

• Key Issues (continued)
• Near Term Forecasting for Planning
• Resource Implications Longer term
• Budgeting (Operational and Capital)
• Staffing
• Facilities
• Other

5
What are Student Learning Outcomes

• Student Learning Outcomes Assessment
• Measures both achievement and monitors learning
  process
• Enables faculty to ask how the curriculum and the
  instructional process can be made more effective
  and better

6
SLO Assessment Process
Educational Goals, Course Objectives, etc
Utilize Assessment Analysis for Course Improvement
Acquire Assessment Information
Analyze and Interpret Assessment Information
7
Program Overviewand ForecastKey Elements

• Program Identification
• Strategic and Cultural Competence Plan
• College-wide SLOs
• Program SLOs
• Data Review and Analysis
• Curriculum Review Report

8
Program Overviewand ForecastKey Elements

• Program Delivery Support Services Review
• Program Strengths Challenges
• Planning Agenda Action Plan
• Near Term Forecast Planning
• Long Term Resource Planning

9
Program Overview and ForecastDescription of
Elements

• I. Program Identification
• Function the program needs to identify its
  primary function or functions
• Examples
• CIS (Office technologies Applications,
  Programming, Networking, Network Security, Web
  Applications)
• English (Composition, Literature, Reading,
  Developmental)
• Roles what roles or populations are served by
  the various functions of the program?

10
Program Overview and ForecastDescription of
Elements

• Strategic Planning
• Listing of Goals from Strategic Plan
• Assessment
• Planning Implications
• Cultural Competence Equity Plan
• Identification of relevant goals
• Program Strength with respect to goals
• Evidence of strength identified (in ii)

11
Program Overviewand ForecastDescription of
Elements

• III. College-Wide Student Learning Outcomes
• Listing of College-Wide Outcomes
• Identification of relationship of Program to
  individual outcomes
• Assessment, analysis and planning implications
  for programs

12
Program Overviewand ForecastDescription of
Elements

• IV. Program Student Learning Outcomes
• Identification of Program SLOs
• Identification of Assessment Process for Program
  SLOs
• Alignment of courses to Program SLOs (Matrix)
• Report on the Assessment of Program SLOs
• Analysis and Evaluation of Program SLOs and
  development of a response plan

13
Program Overviewand ForecastDescription of
Elements

• V. Data Review and Analysis
• Data Descriptions
• Identification of Trends and Anomalies in data
• Implications of Trends and Anomalies
• Identification of data needed but not available

14
Program Overviewand ForecastDescription of
Elements

• V. Data Review and Analysis (Expanded)
• Data Descriptions
• Program Demographics
• Program Characteristics
• Enrollment
• Productivity
• VTEA Core Indicators
• Other Data
• Student Achievement Data
• Course Completions/success
• Course/program persistence
• Program Success (open) i.e. degrees/certificates,
  employment or transfer data, other data
  (Licensure, etc)

15
Program Overviewand ForecastDescription of
Elements

• VI. Curriculum Review Status Report
• Status of Curriculum Review (status Matrix)
• Status Report on Course SLOs (Matrix)
• Identification of Course SLOs
• Assessment Process for Course SLOs
• Analysis of effectiveness of Course SLOs

16
Program Overviewand ForecastCurriculum Review
Elements
17
Program Overviewand ForecastCurriculum Review
Elements
18
Program Overviewand ForecastDescription of
Elements

• Program Delivery Support Services Review
• Delivery Options
• ID Delivery Options (D/E, Distance Ed, etc)
• ID Strengths Weaknesses
• Planning Implications
• Support Services
• ID Delivery Options (D/E, Distance Ed, etc)
• ID Strengths Weaknesses
• Planning Implications

19
Program Overviewand ForecastDescription of
Elements

• VIII. Program Strengths and Challenges
• Strengths and Accomplishments since last review
• Significant changes/impacts since last program
  review
• Implications for Plan based on strengths and
  challenges

20
Program Overview and ForecastDescription of
Elements

• IX. Planning Agenda and Action Plan
• a. Near Term Forecast and Planning
• Strategic Planning Implications
• Program SLO Planning Implications
• Equity Planning Implications, activities, or
  changes
• Curriculum Additions, changes, validations, and
  deletions (Program and Course)
• Internal Program Adjustments (intra-departmental)

21
Program Overview and ForecastDescription of
Elements

• Near Term Forecast Planning (Cont)
• Program Strengths Opportunities/Cooperative
  Opportunities
• Staffing (Near term 1-2 years)
• Budget (current budget cycle for operations and
  capital outlay)
• Other needs (Research, staff development, etc)
• b. Resource Planning
• Budget (Long term needs -- replacement upgrade)
• Staffing (Long term program changes
  expansion)
• Facilities
• Other (research, staff development, other)

22
PrOFWhere does the information go?
23
PrOF Reports

• Planning Committee
• Comment on report
• Identification of College-Wide Implications
• College Dialog

• Budget Planning Committees
• Resource Allocation Planning
• Planning Implications

• Internal Assessment and Response by Department
• Inclusion in Unit Planning Processes

• College-Wide Dialogue
• Implications of Program SLOs
• Discussion of College-Wide SLOs, Strategic Plan,
  and CCE

24
Beyond Program Review - PrOF
Faculty are introduced to SLOs and the PrOF
Faculty Define and Design PROGRAM OUTCOMES
Faculty design and conduct an ASSESSMENT of
outcomes and map an alignment of courses to
program SLOs
Challenges Faculty develop response plan to
change curriculum, pedagogy, student support, etc.
Success Revise outcomes to promote forward
movement of program
Analysis of Assessment Results
25
Implementing PrOF

• Introduce Faculty to SLOs and PrOF
• Identify strategies to create, assess, and evolve
  SLOs to improve
• Student understanding of course content
• Instructor ability to meet curriculum
  expectations
• Enhance and expand communication among faculty

26
Implementing PrOF

• Faculty Define and Design Program SLOs
• Review Course and College-Wide SLOs
• Identify Program SLOs that align with Course and
  College-Wide SLOs, Student Population
  Characteristics, and Program Effectiveness
  expectations
• Identify Articulation or Workforce expectations

27
Implementing PrOF

• Parameters for Program SLO development
• Process is iterative SLOs will evolve and be
  reviewed over time
• Not all Programs will develop and assess all
  program outcomes at the same time
• Program SLOs will be refined based on
  assessments and PrOF analyses

28
Implementing PrOF

• Mapping of the Alignment of Program SLOs and
  Course SLOs
• Courses will be aligned with the Program SLOs
  that are supported by individual courses
• Courses will be identified as to the level of
  intensity that they address/satisfy Program SLOs
• Introduction
• Practice and reinforcement
• Mastery

29
Implementing PrOF

• Assessment of Student Learning Outcomes
• Faculty are trained in assessment options
• Faculty select assessment processes that are
  appropriate to the SLO and the program
• Assessments can be both
• Embedded in Courses/Programs
• External to courses

30
Implementing PrOF

• Analysis of Assessment
• Results of assessments reviewed
• Assessment results are evaluated against
  pre-agreed criteria
• Outcomes are determined to be
• Successful
• Challenged

31
Implementing PrOF

• SUCCESSFUL Outcomes Assessment
• Target Outcomes are refined to raise the bar
  for the program and students (Improve learning)
• Other outcomes are selected for enhanced review

32
Implementing PrOF

• CHALLENGED ASSESSMENT RESULTS
• Program develops a RESPONSE PLAN
• Examine Curriculum
• Examine Course Syllabi
• Review teaching practices
• Review student support services/processes
• Review equipment/technology/other support needs
• Implement RESPONSE PLAN
• Recycle the PrOF process to assess outcomes

33
Implementing PrOF

• Reporting and Using PrOF
• PrOF includes a Planning Process that informs
  both Unit Plans and the Colleges broader
  Resource Allocation Processes
• PrOF reports are reviewed through the governance
  process
• PrOF conclusions are integrated into college-wide
  planning processes
• Identification of universal or broad based
  strategies and needs
• Revision of College-wide Plans, College Strategic
  Plan
• Institutional planning evolves as a result to
  PrOF to enable response plans through expanded
  resources or re-allocations

34
Implementing PrOF

• Next Steps
• As we saidThis is an iterative processPrOF
  and SLOs will evolve as we become more
  proficient and comfortable with them to improve
  student success!