Benchmarking in Community Colleges: Status of Two National Projects

1
Benchmarking in Community Colleges Status of Two
National Projects

• Jeffrey A. Seybert
• Director, Research, Evaluation, and Instructional
  Development
• Johnson County Community College
• George Malo
• Associate Vice Chancellor for Research and
  Assessment
• Tennessee Board of Regents
• John D. Porter
• Associate Provost
• The State University of New York

2
Two National Benchmarking Projects

• The Kansas Study
• Community College instructional costs and
  productivity
• Modeled on the Delaware Study
• Collects, analyzes, and reports data at the
  discipline level

• The National Community College Benchmark Project
• Involves a wide array of student outcomes,
  access, workforce development, faculty/staff,
  human resources, and finance variables
• Collects, analyzes, and reports data at the
  institutional level

3
The Kansas Study

• Supported by a three-year, 282,000 grant
• from FIPSE (USDE).
• Colleges will be able to analyze faculty
• workload and instructional cost at the
• academic discipline level of analysis.

4
Kansas Study History

• Summer 2002 FIPSE project approval and grant
  award
• Fall 2002-Fall 2003 Advisory committee
  identifies data elements, designs processes,
  and conducts two pilot studies
• Fall 2004 Aggregate reports distributed
  website opened for peer comparisons
• 2004 Year 1 project implementation 50
  institutions provided data
• Year 2 - 67 institutions participated
• 2006 Year 3

5
How Kansas Study Works

• Data Collection
• Excel Spreadsheets distributed electronically
• Data Verification
• Missing data and logical errors
• Partial Data OK (min. 10 disciplines)
• Confidentiality assured
• Annual Reports
• National Norms and Institutional Data
• Access to Kansas Study Website for Peer
  Comparisons

6
Kansas Study Timeline

• February 1 Data Collection Starts
• May 15 Data Verification Process Initiated
• June 18 Participant Institutional Data Due
• July 15 Data Verification Reports Sent
• July 5 Data Analyses Begin
• Early Fall Results Available Database
  Opened for Peer Comparisons/ Benchmarking

7
Web Site

• Kansas Study Website (www.kansasstudy.org)
• Public Information
• General Information
• Enrollment Form
• Sample Data Collection Template
• Sample Report Tables
• Advisory Committee
• Participating Institutions
• Information Available to Participants Only
• Log In Password
• National Norms by Discipline
• Peer Comparisons

8
Benchmarking Instructional Costs and
Productivity How a System and Campus Use the
Kansas Cost Study

• George Malo
• Assistant Vice Chancellor for Research and
  Assessment
• Tennessee Board of Regents

9
The Kansas Cost Model

• Purpose
• Colleges will be able to analyze faculty workload
  and instructional cost at the academic discipline
  level of analysis
• Provides comparative data important to
  accountability processes and decision making at
  both the system and institution levels
• Participants
• Community Colleges
• Context
• Modeled in part on the Delaware cost model for
  universities

10
TBR Context

• Why do we participate?
• Part of TBR and State initiatives
• Defining Our Future
• Strategic planning process
• Accountability reporting
• Performance funding
• Cost model replaces Boards former cost study
• Opportunity for national comparisons as well as a
  management tool for campuses
• Provides consistency across system institutions
• Useful for system policy and management decisions

11
Strategic Planning

• Provides benchmarks
• Annual monitoring of key program variables
• Documentation of activities
• Use in Decision-Making
• Part of presidential evaluations

12
Performance Funding

• Five points awarded as part of assessment
  standard
• All 13 community colleges in TBR system must
• participate
• Report on 4 key indicators (to be discussed
  later in
• presentation)
• Submit a report providing evidence of the usage
  of
• the Kansas model for institutional planning
  and
• improvement.

13
Campus Uses of Kansas Study

• Program Review or Academic Audit
• Look at high risk or outlier programs
• Look at Peer Costs
• Staffing tenure decisions
• Problemsolving tool
• SACS accreditation documentation

14
Caveats to Avoid Misuse

• Tool for institutional decision-making
• Support credible case-making and informed
• decision-making
• Must be used as trend data
• Should not be used for inferences of an
  institution
• as a whole
• Prerequisite for assessing the adequacy of
• accountability

15
System Level Ad Hoc Committee

• Institutional input through Academic Affairs
• committee for appropriate use of data
• Adoption of key indicators as standard for
• framing instructional productivity and
• effectiveness reports
• Development of common questions that would
• lead institutions to evaluate their
  decisions

16
System Level Key Indicators

• FTE students taught per FTE instructional
• faculty by discipline
• Student credit hours per FTE faculty as a
• percentage of national norm by discipline
• Percentage student credit hours taught by
• full-time faculty

17
Comparison Group Selection

• Each institution will construct its peer groups
  
• according to three standards, two for
  System
• use and the third for institutional use
  only
• System reporting as an aggregate
• System reporting per discipline
• At the institution level, each discipline may
  select peers

18
Evidence of Accountability/Productivity

• For each key indicator, the TBR System adopted
  questions to guide institutions in the analysis
  of their own data
• What significant changes can be detected over the
  last three years for the indicator?
• How does this three-year profile for the
  indicator compare to that of institutional peers
  by CIP?
• What factors have contributed to the changes at
  your institution around the key indicator?
• Are you satisfied? Why, and if not, how do you
  plan to make any alterations to adjust key
  indicator?

19
Case Making A Central Goal

• Data must be used for responsible decision
• making
• Can the institution make a case, from its
  analysis of the allocation of faculty, that it is
  moving toward improvement in instructional
  management?
• Can the institution make the case that it is
  effectively using its faculty ?
• Can the institution make the case that it is
  moving toward improvement in contributions to the
  institution, system, state, students, or the
  public?
• Do these contributions reflect a responsible use
  of resources ?

20
System Wide Assistance

• Programming for data collection
• Programming for analysis of data
• Templates for reporting of data
• Revisions to/formulation of policies and
• guidelines

21
Campus Uses of Kansas Study

• Documenting accreditation compliance
• Planning institutional change
• Predicting academic/financial impact

22
Accreditation

• Institutional Effectiveness
• Adequate Faculty
• Sound financial base and adequate
• resources

23
Annual Program Documentation
24
Document Staffing by Program
25
Document Cost by Program
26
New Program Planning
27
The National Community College Benchmark Project

• Involves a wide array of student outcomes,
  access, workforce development, faculty/staff,
  human resources, and finance variables
• Collects, analyzes, and reports data at the
  institutional level

28
Purposes

• To collect and report community college benchmark
  data on a national basis
• To provide data for comparisons and benchmarks of
  instructional, workforce-development, and other
  community college activities

29
NCCBP History

• 2003 Project Designed and Piloted
• 2004 First year implementation interactive
  project website designed and launched 110
  institutions participated
• - SUNY System (30 Colleges)
• - TN System (13 Colleges)
• 2005 Second year implementation 113
  institutions participated
• - SUNY System (30 Colleges)
• - TN System (13 Colleges)
• - PA Colleges (13 of 14)
• 2006 Third year implementation enrollment
  opened in Feb.
• - SUNY System (30 Colleges)
• - TN System (13 Colleges)
• - PA Colleges (14 Colleges)
• - FL System (28 Colleges)

30
How NCCBP Works

• Data collection
• Excel spreadsheets distributed electronically
• Data verification Missing data and logical
  errors
• Partial data OK no peer comparison for missing
  data
• Confidentiality assured
• Cost 1,000/year per institution
• Annual reports
• Aggregate data delivered electronically
• Access to NCCBP Web site for peer comparisons
• Website www.NCCBP.org

31
Data-collection Form

• FORM 4 Credit Students Who Enrolled Next Term
  and Next Fall
• Column 1 Enter unduplicated total credit students
  (including those who withdrew from all courses)
  at the end of the fall 2003 term. Do not I
  include high school students.
• Column 2 Enter total students from Column 1 who
  graduated or completed certificates before the
  next (spring 2004) term.
• Column 3 Enter total students from Column 1 who
  enrolled in the next (spring 2004) term.
• Column 4 Column 3 / (Column 1 - Column
  2)
• Column 5 Enter total students from Column 1 who
  graduated or completed certificates before next
  fall (fall 2004) term. Include graduates and
  completers in Column 2.
• Column 6 Enter total students from Column 1 who
  enrolled in the next fall (fall 2004)
  term.
• Column 7 Column 6 / (Column 1 - Column
  5)

32
(No Transcript)
33
Benchmark Categories

• Completion Transfer Rates
• Persistence Rates
• Transfer Student Performance
• Student Satisfaction
• Student Performance Measures
• Career Preparation
• Academic Success

• Access Participation
• Market Penetration
• Workforce Development
• Section Size, SF Ratio, Faculty Load
• Student Services Staff
• HR Statistics
• Instructional Professional Development Costs

34
2006 Timeline

• March Data collection begins.
• May Data-collection instruments are due.
• June Data confirmation reports are
• distributed.
• July Data updates are due.
• September Aggregate reports are distributed.
  Web site is opened for peer
  comparisons.

35
2004 Participants
36
College Characteristics

• Campus Environment
• Institution Type
• Institutional Control
• Academic Calendar
• Credit Enrollment
• Minority Students
• Percent State Revenue

• Operating Budget
• Faculty Unionized
• Service Area Population
• Unemployment Rate
• Household Income
• Service Area Percent Minority

37
National Community College Benchmark Project A
System/State Perspective

• John D. Porter
• Associate Provost
• The State University of New York

38
NCCBP A Valuable Resource for Systems States

• NYs CCs are funded based on annual full-time
• student equivalents (FTE)
• CCs need to benchmark their operations to
• maintain expand state support
• CCs are as complex as research universities,
• which is not understood by most decision
• makers
• NCCBP fills a critical void

39
SUNYs Community Colleges

• SUNYs community colleges enroll 208,374
• students
• 50.3 of SUNYs overall enrollment
• Campuses range in size from 21,000 to 1,500
• Located throughout New York Stake, including
• New York City
• One CC awards bachelor and master degrees
• (FIT)
• These institutions have every conceivable
• governance/funding arrangement

40
SUNY Support of NCCBP

• SUNYs benchmarking has here-to-fore focused
• on intra measures
• NCCBP offers the potential to benchmark
• against true peers and other states
• For the past three years, SUNY has encouraged
• campus participation by paying the
  subscription
• fee
• This year, all 30 community colleges will
• participate in NCBBP
• SUNYs hope is that other states and systems
• will see the value of this project and
  participate

41
NCCBP A Valuable Resource for Systems States

• NCCBP has generally been conceived as a tool
  for
• campuses
• Systems/States need this type of resource,
  since most
• community colleges are funded based on
  enrollment
• SUNY requires CCs to plan enrollments 5 years
  into
• the future also update the institutional
  mission every
• five years
• NCCBP has the potential for developing reports
• tailored to the needs of Systems and States

42
Issues?

• Participation needs to reach a critical mass
• 300 institutions?
• Gaining support for NCCBP on campus
• (some dont want to be compared)
• Funding shifting cost to the campus at some
• point in the future
• Accuracy/quality of data? How best to
• achieve?
• Important that NCCBP keeps the cost of
• participating low

43
National Community College Benchmarking Project

• George Malo
• Associate Vice Chancellor for Research and
  Assessment
• Tennessee Board of Regents

44
Uses of NCCBP

• Strategic planning
• Performance funding
• Documenting accreditation
• Policy development/analysis/evaluation

45
Where Does MCC Excel?Core Course Success
46
Example of Performance Funding Indicators
47
Outcomes of Educational Programs Career Program
Completers
48
Policy Questions
49
Questions