System Accountability Plan SAP Introduction

1
SYSTEM PERFORMANCE FUNDING PROGRAM
January 2005Administration BuildingDixon
University Center
2
Welcome and Introductory Remarks
Chancellor Hample
3
Purposes of the Workshop
President Grunenwald
4
Distributed Materials
A Copy of this Presentation Glossary of Terms (in
Order of Presentation) The Accountability
Matrix Performance Funding Naming
Conventions Distributed Prior to the
Meeting Performance Funding Guidelines (Board
of Governors) Performance Funding Definitions and
Documentation Glossary of Terms (in Alphabetical
Order) Peer Institutions by University PA Public
Peer Institutions CSRDE Peer Institutions
5
Presentation Agenda Overview
Morning Session 930 a.m. to 1230 p.m.
Accountability Objectives Data Collection and
Data Processing Performance Evaluation
Methodology There will be a 15 minute break
around 1100 a.m. Lunch 1230 p.m. to 100
p.m. Afternoon Session 100 p.m. to 330
p.m. Performance Funding Methodology There will
be a 15 minute break around 215 p.m. Closing
Comments 330 p.m. to 400 p.m. Chancellor and
University Presidents
6
Morning Session Topics Covered
Review of Accountability Objectives System
Accountability and Performance Funding
Programs System Strategic Plan Data Collection
and Data Processing Data Collection and
Processing Data Definitions and Documentation
Question and Answer Period Performance
Evaluation Methodology Development of
Baselines, Benchmarks and System Performance
Targets Question and Answer Period
7
System Performance Funding Program
Board of Governors approved the original System
Performance Funding Program in 2000 Substantial
revisions were made to the program in 2002 To
align the program with the System Accountability
Plan Eight of the 17 measures used in the
System Accountability Plan (SAP) were identified
for use in performance funding and a method for
allocation was described. Since 2002, the
Program has been expanded to include Evaluation
based on Benchmark comparisons Evaluation based
on System Performance Target comparisons
8
The Accountability Matrix and Accountability
Measures
9
Accountability Matrix
Organizes Accountability Measures and Narrative
Assessment Statements relative to System Values
and Standards
System Values Stimulating Intellectual Growth
Applying Knowledge Serving the Common Good
Fostering Citizenship, Social Responsibility,
Diversity and, Practicing Stewardship Standards
of Performance Enhancing Organizational
Effectiveness Pursuing and Rewarding
Excellence and, Enhancing Operational
Efficiency
10
The Accountability Measures
(10) Diversity of Entering Class (11) Minority
Enrollment (12) Staff and Faculty Diversity (13)
Programs with few Graduates (14) Personnel
Ratio (15) Private Support (16) Instructional
Costs/FTES (17) Faculty with Terminal Degrees
(1) Degrees Awarded (2)  Persistence Rates
(3)  Accredited Programs (4)  4 6 year
Graduation Rates (5)  Faculty Productivity (6)  Di
stance Education (7)  PRAXIS (Teacher
exams) (8)  Internships  (9)  Transfers from
Community Colleges (or Associate degrees
awarded)
11
The Performance Funding Measures
The 8 Measures Are

• Degrees Awarded (by level)
• Two-Year Persistence Rates (overall and by
  ethnicity)
• Six-Year and Four-Year Graduation Rates (overall
  and by ethnicity)
• Faculty Productivity (student credit
  hours/full-time equivalent faculty)
• Faculty Diversity (Minority Faculty - fall
  semester only)
• Personnel Ratio (Total EG Compensation/Total EG
  Expenditures)
• Cost Per FTES (annualized undergraduate and
  graduate) and
• Full-Time Tenured and Probationary Non-tenured
  (tenure track) Faculty with Terminal Degrees.

12
Data Collection and Data ProcessingThe purpose
of this section is to review the process of data
collection through the calculation of the measures
13
Data Collection and Data ProcessingTopics Covered
Historical Context for Performance Funding
Measures Data Collection Practices Edit
Checks, University Review and Reconciliation Dat
a Submissions Outliers and Missing Data Data
Definitions and Calculation of Measures
Question and Answer Period
14
Historical Context for Performance Funding
Measures

• PASSHE has been reporting data on performance
  measures for many years.
• Some measures have been reported each year in
  the Factbook since the 1980s.
• 24 Performance Indicators were developed in
  1998, including
• Second Year Persistence
• Six-Year Overall Graduation Rate
• Six-Year Underrepresented Graduation Rate
• Underrepresented Racial/Ethnic Faculty
• Instructional Costs (Undergraduate Graduate)

15
Historical Context for Performance Funding
Measures

• Performance and Outcomes Plan
• Adopted by Board of Governors in 2000
• Set three year targets set for both
  quantitative and qualitative measures
• Quantitative measures included
• Faculty productivity
• Second year persistence
• Minority faculty

16
Data Collection PracticesData Collection, Data
Integrity, University Review and Reconciliation
17
Data CollectionGeneral Observations

• Data submitted by the universities
• To be used in constructing the measures.
• External data sources
• Include state and federal reporting bodies as
  well as university consortiums and private
  agencies.
• Definitions and criteria are regularly reviewed
• To ensure relevance, consistency and
  comparability (particularly for external
  benchmarking data sources).

18
Data CollectionGeneral Observations

• System data collection has evolved
• More comprehensive and integrated trends that
  will continue with the implementation of Campus
  Management
• Future improvements
• Web-based tools and applications will enable
  staff at each university to see their data and
  predict performance evaluations before report is
  distributed

19
Data Collection Practices
20
Data Collection PracticesTopics Covered

• Data Integrity
• Types of Data Requests
• Sources of Data
• Internal Data
• External Data
• Data Collection Process

21
Data Collection PracticesData Integrity
Universities are the source of all internal
data Submissions are documented and audit trails
are established Edit checking and reconciliation
of data are next steps in the process Data
loaded to the Data Warehouse are reviewed by each
University University data is periodically
audited Internal Review Group Data Audits
22
Data Collection Practices Data Submissions
23
Data Collection PracticesTypes of Data Requests
Regular/Recurring Annual Data Collection
Plan Submissions to National and State
Agencies Ad Hoc Special Requests Extraction
and Compilation of National Databases
24
Data Collection PracticesSources of Data
Internal Sources (University Submissions) Student
Data Completions Terminal Degrees Common Cost
Accounting Financial Information (FIN)
Report External Sources Consortium for Student
Retention Data Exchange (CSRDE) Integrated
Postsecondary Education Data System
(IPEDS) National Center for Higher Education
Management Systems (NCHEMS)
25
Data Collection Practices Internal Data
Student and Completions Data Submitted multiple
times each year Terminal Degree Data Special
request made each Spring Common Cost Accounting
Data Submitted in December and reconciled
through the Spring Internal Financial
Data Financial Information (FIN) Report
submitted Report is official once audits are
completed.
26
Data Collection Practices External Data
IPEDS Data are obtained by using IPEDS Peer
Analysis System Degrees Awarded - IPEDS
Completions and IPEDS Fall Enrollment Faculty
Diversity - IPEDS Fall Staff Personnel Ratio -
IPEDS Finance CSRDE Database is purchased
annually from CSRDE Used for Second Year
Persistence and Graduation Four Year and Six
Year
27
Data Collection Practices Data Collection Process

• Data Submission and/or Extraction
• Data Loading (to Staging Area)
• Edit Checks and Error Reports
• University Reviews
• Resubmission Process (if required)
• Reconciliation and Acceptance
• Data Loading to Data Warehouse

28
Data Collection Practices Student and
Completions Submissions
Data submitted via FTP server Loaded into
staging area edit checks run If inconsistencies
are found, error report is generated University
reviews error report Data is reconciled and a
new resubmission is submitted Process begins
again until no errors are found Data loaded to
data warehouse
29
Data Collection Practices Terminal Degree
Submissions
University provided with a list of faculty as of
freeze date EIS supplies data from HRIS or
SAP-HR Data are updated by the
university Submitted and loaded into a staging
area where edit checks are run If errors are
found they are corrected Data loaded to the data
warehouse
30
Data Collection Practices Common Cost Accounting
Submissions
Data submitted via FTP server Data loaded into
staging area edit checks are run If
inconsistencies are found error report is
generated and sent to University contact
person Process repeats until no errors are
found Data is accepted by University Loaded
into data warehouse
31
Data Collection PracticesFinancial Information
Reports (FIN)
Data submitted to Office of the Chancellor
Submitted to Administration and Finance Edit
checks run by Administration and Finance If
inconsistencies are found, they are communicated
to universities and corrected FIN Report data
available for use after financial audits
32
Data ProcessingOutliers and Missing Data
33
Data Processing Treating Outliers and Incorrect
or Missing Data for Benchmark Comparisons
Outliers (in Benchmark Comparison
Data) Reported data values that are
sufficiently extreme to cause skewed results for
the calculated average and standard deviation of
comparison groups. Inclusion of outliers would
cause an inappropriate comparison of
performance, therefore they are excluded. A
statistical rule is used that excludes data that
are in excess of 2.8 standard deviations (above
or below the average) Incorrect or Missing
Data Occasionally, peer data may be missing or
in error. If correct data cannot be obtained,
the peer is excluded.
34
Data Definitions and Calculation of Measures
35
In this portion of the presentation Review
sub-measures and definitions for each of the 8
measures Provide representative calculations
for each measure using university data
Data Definitions and Calculation of Measures
36
Data Definitions and Calculation of Measures

• National (federal) definitions used whenever
  possible
• For some measures, System definitions are used
  (e.g., Common Cost Accounting)
• Benchmarks and System Performance Targets
  referenced
• Values for Benchmarks and System Performance
  Targets provided, but discussion will be deferred
  until later in the workshop
• Numerical examples provided
• For each sub-measure, examples of actual data
  and calculations will be presented

37
Data Definitions and Calculations Measure 1
Degrees Awarded

• Four Sub-Measures
• Bachelors, Masters
• Number and Degree to Enrollment Ratio for each
• Includes
• All degrees awarded and submitted by
  universities between 7/1-6/30 of current fiscal
  year
• Bachelors Degree
• An award (baccalaureate or equivalent degree)
  that normally requires at least four but not more
  than five years of full-time equivalent
  college-level work.
• Masters Degree
• An award that requires the successful completion
  of a program of study of at least the full-time
  equivalent of one or more academic years of work
  beyond the bachelor's degree.

38
Data Definitions and Calculations Measure 1
Degrees Awarded

• FTE for Degree to Enrollment Ratio
• Includes all Fall undergraduate or graduate, not
  just degree-seeking students
• Benchmark
• IPEDS data used for institutional peers for
  Benchmark
• System Performance Target
• For Degree to Enrollment Ratios
• Bachelors 21.5
• Masters 67.0

39
Degree to Enrollment RatioBloomsburg University
Measure 1 Degrees Awarded
40
Measure 1 Degrees Awarded
Bachelors Degree to Enrollment Ratio (1)
Count Applicable Bachelor Degrees Awarded
1,539 (2)  Three-Year Average of Undergraduate
FTE 6,528.8 (3) Degree to Enrollment Ratio
(1,539 / 6,528.8) 23.57 Masters Degree to
Enrollment Ratio (1) Count Applicable Masters
Degrees Awarded 308 (2) Two-Year Average of
Graduate FTE 411.5 (3) Degree to Enrollment
Ratio (308 / 411.5) 74.76
41
Data Definitions and Calculations Measure 2
Second Year Persistence

• Four Sub-Measures
• Number of persisters (Overall and Black and
  Hispanic combined)
• Retention Rate (Overall and Black and Hispanic
  combined)
• White and Hispanic used for Cheyney
• Includes
• All fall semester, first-time, full-time
  baccalaureate degree seeking freshmen (as coded
  and submitted by universities)
• Baccalaureate degree students who become
  associate degree students before they start their
  second year are counted in the cohort and as
  persisters

42
Data Definitions and Calculations Measure 2
Second Year Persistence

• Persisting student
• Cohort student who is still enrolled in the
  institution the following fall semester
• Second Year Retention Rate
• The number of cohort students still enrolled at
  the institution at the beginning of the
  subsequent fall semester divided by the number in
  the cohort group
• Benchmark
• Based on national cluster from CSRDE by SAT
  selectivity and Carnegie Classification, based on
  fall 2002 average combined SAT score
• System Performance Target
• Overall and Minority Rate 79.0

43
Average SAT Scores, Selectivity and Benchmark
Averages
44
Measure 2 Second Year Persistence Overall,
Black and Hispanic Combined, Millersville
University
45
Measure 2 Second Year Persistence
Overall Retention Rate (1) Count of Entering
Cohort 1,262 (2)  Count of Persisting Students
(Fall Freeze) 1,023 (3) Overall Retention Rate
(1,023 / 1262) 81.06 Minority Retention
Rate (1) Count of Minority Entering Cohort
163 (2) Count of Persisting Students (Fall
Freeze) 115 (3) Minority Retention Rate (115
/ 163) 70.55
46
Data Definitions and Calculations Measure 3
Graduation 4 and 6 Year

• Eight Sub-Measures
• Number of graduates and graduation rate for all
  students and Black and Hispanic combined (4 year
  and 6 year)
• White and Hispanic combined for Cheyney
• Includes
• All first-time, full-time baccalaureate degree
  seeking freshmen (as coded and submitted by
  universities)
• Graduates from fall, spring and summer
  commencements (through August 2003 graduation)

47
Data Definitions and Calculations Measure 3
Graduation 4 and 6 Year

• Special cases
• Associate degree completers who begin as
  baccalaureate degree-seeking freshmen are counted
  in cohort but NOT as graduates unless they go on
  and earn a bachelors degree
• Students who begin in cohort but transfer and
  graduate elsewhere (even in another System
  university) are counted in cohort but NOT as
  graduates
• Students not in the cohort who earn a bachelors
  degree (e.g., transfers) are NOT included in
  number of graduates or graduation rate

48
Data Definitions and Calculations Measure 3
Graduation 4 and 6 Year

• Graduation rate
• The number of graduated cohort students divided
  by the total number in the cohort group
• Totals can vary
• Due to changes in student ethnicity and
  exclusions to cohort
• Benchmark
• Based on national cluster from CSRDE by SAT
  selectivity, based on fall 2002 average SAT score
• System Performance Targets
• Four year 30.0
• Six year 55.0
• Same targets for Overall and Black Hispanic
  combined (White Hispanic)

49
Measure 3 Graduation 4 and 6 Year Overall
and Black and Hispanic combined, West Chester
University
50
Measure 3 Graduation 4 Year
Overall Graduation Rate (1) Count Entering
Cohort (Fall Freeze) 1,711 (2)  Count Initial
Cohort Students Graduated 437 (3) Percent of
Students Who Graduated (437/1,711)
25.54 Minority Graduation Rate (1) Count
Entering Cohort (Fall Freeze) 161 (2) Count
Initial Cohort Students Graduated 23 (3)
Percent of Students Who Graduated (23 / 161)
14.29
51
Measure 3 Graduation 6 Year
Overall Graduation Rate (1) Count Entering
Cohort (Fall Freeze) 1,576 (2)  Count Initial
Cohort Students Graduated 931 (3) Percent of
Students Who Graduated (931/1,576)
59.07 Minority Graduation Rate (1) Count
Entering Cohort (Fall Freeze) 176 (2) Count
Initial Cohort Students Graduated 79 (3)
Percent of Students Who Graduated (79 / 176)
44.89
52
Data Definitions and Calculations Measure 4
Faculty Productivity (Student credit hours per
FTE instructional faculty)

• Measure 4 has no sub-measures.
• Instructional faculty
• includes teaching faculty and release time for
  department chairs, assistant department chairs,
  graduate program coordinators, and internship
  coordinators (Common Cost Accounting standard)
• Data Source
• Universities submit student, cost, and faculty
  workload data via Common Cost Accounting
  submission (2002-03 data used for current year)

53
Data Definitions and Calculations Measure 4
Faculty Productivity

• One credit hour
• 15 contact hours, represents 50 minutes of
  instruction per week
• One FTE faculty
• 12 contract hours per semester, 24 per year.
  Full and part-time faculty included
• Benchmark
• System average of 545.95 (excludes outliers of
  California and Cheyney)
• System Performance Target
• 565.0

54
Measure 4 Faculty Productivity Slippery Rock
University
55
Measure 4 Faculty Productivity
Credits Per Instructional FTE Faculty (1) Total
Credit Hour Production 211,976.0 (2)  Sum of
FTE Instructional Faculty 378.47 (3) Faculty
Productivity (211,976 / 378.47) 560.09
56
Data Definitions and Calculations Measure 5
Employee Diversity

• Measure 5 Sub-Measures
• Number of minority faculty members
• Percent of faculty who are minority (fall 2003)
• Includes
• Full-time, tenured and tenure-track individuals
  (instructional and non-instructional)
• Ethnic groups
• Black, Hispanic, Asian, and Native American
  (White used instead of Black for Cheyney)
• Source of data
• HRIS
• Benchmark
• Based on institutional peers
• System Performance Target
• 15.0

57
Measure 5 Employee Diversity East Stroudsburg
University
58
Measure 5 Employee Diversity
Percent of Faculty who are Minority (1)
Overall Count of Faculty 240 (2)  Count of
Minority Faculty 31 (3) Faculty Diversity
(31 / 240) 12.92
59
Data Definitions and Calculations Measure 6
Personnel Ratio (inverted measure)

• Measure 6 (no Sub-Measures)
• Total Personnel Compensation Costs as a percent
  of Total Expenditures and Transfers (2002-03)
• Sources of data
• FIN reports and IPEDS Finance for institutional
  peers
• Measure was modified in 2001-02 to conform to
  GASB requirements
• Benchmark
• Based on institutional peers
• System Performance Target
• 73.0

60
Data Definitions and Calculations Measure 6
Personnel Ratio Formula

• (1) Numerator EG Total Personnel
  CompensationRestricted Total Personnel
  CompensationPlant Total Personnel
  Compensation-EG Compensated Absence adjustment
  - Sick-EG Compensated Absence adjustment -
  Annual-EG Postretirement in Excess of
  Pay-as-you-go-Restricted Compensated Absence
  adjustment - Sick-Restricted Compensated
  Absence adjustment - Annual-Restricted
  Postretirement in Excess of Pay-as-you-go-Plant
  Compensated Absence adjustment - Sick-Plant
  Compensated Absence adjustment - Annual-Plant
  Postretirement in Excess of Pay-as-you-go
• (2) Denominator EG Total Expenditures and
  TransfersRestricted Total Expenditures and
  Transfers-EG Postretirement in Excess of
  Pay-as-you-go-EG Student Aid
  Expense-Restricted Student Aid Expense
• (3) Personnel Ratio Numerator/Denominator

61
Measure 6 Personnel Ratio
62
Measure 6 Personnel Ratio
Personnel Ratio (1) Total Personnel
Compensation 53,352,339 (2)  Total Adjusted
Expenditures 67,001,587 (3) Personnel Ratio
(53,352,339 / 67,001,587) 79.63

63
Data Definitions and Calculations Measure 7
Instructional Cost (inverted measure)

• Measure 7 includes two Sub-Measures
• Undergraduate Cost per FTE Student
• Masters Cost per FTE Student
• Annualized data from 7/1/02 to 6/30/03
• Source
• Common Cost Accounting Report submitted by
  universities
• Benchmark
• System average 4,627 for undergraduate, 5,713
  for masters
• System Performance Targets
• 3.5 increase per year for undergraduate 7.0
  increase per year for masters

64
Data Definitions and Calculations Measure 7
Instructional Cost (inverted measure)

• Annualized FTE students
• One undergraduate FTE student equals 30 credit
  hours per year one graduate FTE student equals
  24 credit hours per year
• Total instructional cost
• General academic instructional costs of
  programs, including release time for department
  chairs, assistant department chairs, and
  internship coordinators (Common Cost Accounting
  standard)

65
Measure 7 Instructional Cost (inverted
measure)Shippensburg University
66
Measure 7 Instructional Cost (inverted measure)
Undergraduate Cost per FTE (1) Total
Undergraduate Cost 27,196,982 (2)  Total
Undergraduate FTE 6,374.77 (3) Undergraduate
Instructional Cost (27,196,982 / 6,374.77)
4,266 Masters Cost per FTE (1) Total Masters
Cost 3,161,569 (2) Total Masters FTE
505.71 (3) Masters Instructional Cost
(3,161,569 / 505.71) 6,252
67
Data Definitions and Calculations Measure 8
Faculty Terminal Degrees

• Measure 8 (no Sub-Measures)
• Percent of full-time permanent tenured and
  tenure track instructional faculty with terminal
  degrees (Fall 2003)
• Degrees approved are PhD, EdD, DEd, DBA, ScD, JD,
  and MFA
• Committee of CAOs considering inclusion of
  other terminal degrees
• Source
• Special data submission from universities
• Benchmark
• System average of 80.77
• System Performance Target
• 90.0

68
Measure 8 Faculty Terminal Degrees Clarion
University, Fall 2003
69
Measure 8 Faculty Terminal Degrees
Percent of Tenured or Tenure Track Faculty with
Terminal Degrees (1) Count of Tenured or
Tenure Track Instructional Faculty
247 (2)  Count of Tenured and Tenure Track
Instructional Faculty with Terminal Degrees
215 (3) Faculty Terminal Degrees (215 / 247)
87.04
70
Data Definitions, Documentation and the
Calculation of MeasuresQuestions Answers
71
Performance Evaluation Methodology
72
Performance Evaluation MethodologyTopics Covered
Development of Baselines Review of 4 Prediction
Methods Development of the Baseline Evaluation
and Reporting of Performance Establishing
Benchmarks Development of the Benchmark Evaluati
on and Reporting of Performance Performance
Target Attainment Development of the System
Performance Targets Evaluation and Reporting of
Performance Question and Answer Period
73
Performance Evaluation MethodologyDevelopment of
Baselines
Objective To develop for each measure historical
baselines and statistical bounds that provide an
appropriate expectation of future performance
outcomes Calculation Baselines are calculated
by averaging four different trend estimates, 2
short-term, 2 long-term Seven-year
trends, Three-year trends, Two-year change,
and Seven-year System trends
74
Performance Evaluation Methodology Development
of Baselines
Uses each universitys own data Thus, reflects
the individual institutions history Value of
methodology The combination of 2 short-term and
2 long-term projections reduces the impact of
data anomalies, and no one methodology drives
subsequent analysis Projected baseline values
represent expected performance Assuming no
significant changes from the trend line
75
Performance Evaluation Methodology
Development of BaselinesA detailed description
of statistical methodology
76
Development of BaselinesTopics CoveredThe Four
Methods of PredictionCalculating the
BaselineAdding Statistical BoundsEvaluation
of Performance
77
Performance Evaluation Methodology Development
of Baselines
Baselines are unique for each university and each
measure Uses the universitys own data Seven
years of historical data (for the 2003-2004
System Accountability Report) Current Year
Actual (Year 8) is evaluated But not included in
the baseline calculations shown on the
following graphs in yellow
78
Starting Point, University Seven Years of
Historical Data
79
The Four Prediction Methods
Method A Seven Year University Trend Method
B Three Year University Trend Method C Two
Year University Change Method D Seven Year
System Trend
80
Method A (Seven Year University Trend)

• Unique for each university
• Uses the universitys own historical data
• Seven years of history (for the 2003-2004 System
  Accountability Report) and fits the trend of
  those data points
• New data is added each year
• The prior year actual is added to the history
• Long-term trends are captured

81
Method A (Seven Year University Trend)
82
Method B (Three Year University Trend)

• Unique for each university
• Uses the universitys own historical data
• Uses the last three data points of the
  universitys own historical data and fits the
  trend
• In 2003-2004, the System Accountability Report
  three-year trend is calculated using years 5, 6
  and 7 of history
• Intermediate term trends are captured

83
Method B (Three Year University Trend)
84
Method C (Two Year University Change)

• Unique for each university
• Applies the calculated 2 year rate of change
• (between the two prior years) to the prior year
  to project the next year
• Reflects short term trends

85
Method C (Two Year University Change)
86
Method D (Seven Year System Trend)

• Uses historical data for all 14 universities
• Seven years of history (for the 2003-2004 System
  Accountability Report) and fits the trend of
  those data points
• Each university has a unique intercept, but all
  slopes are the same
• New data is added each year
• The prior year actual is added to the history
• Long term System-wide trends are captured

87
Method D (Seven Year System Trend)
88
Resulting Predictions from the 4 Methods
89
Resulting Predictions from the 4 Methods
90
Calculation of the Baseline Average of Four
Prediction Methods
For each year a baseline value is
calculated Baseline for Year X Sum of 4
Methods for Year X / 4 Sum of 4 methods
Method A (Seven Year Trend) Predicted Value
Method B (Three Year Trend) Predicted Value
Method C (Two Year Change) Predicted Value
Method D (Seven Year System Trend) Predicted
Value
91
Calculated Baseline as Average of 4 Prediction
Methods
92
Calculated Baseline as Average of 4 Prediction
Methods
93
Calculation of the Baseline and Bounds

• Statistical Bounds
• Calculated using the standard deviation of the
  seven years of university historical data
• Upper Bound
• Baseline 1 Standard Deviation
• Lower Bound
• Baseline 1 Standard Deviation

94
Final Baseline Used for Evaluation of Performance
95
Final Baseline Used for Evaluation of Performance
96
Performance Evaluation Baselines
97
Final Baseline Used for Evaluation of Performance
Target Exceeded
Target Met
Target Not Met (below lower bound)
98
Baseline Boundaries SAP Report Presentation
Current year data in Table 4-2 represents the
most recent year data points for the actuals,
baseline, upper lower bounds.
99
Baselines and Performance EvaluationsQuestions
and Answers
100
Establishing Benchmarks
101
Categories of External StandardBenchmark Peer
Groups
Institutional Peer Averages
15 peer universities for each of the 14
universities using national data
State-wide Public Averages
Pennsylvania Department of Education or other
state agencies
National Cluster Averages
National sources using selectivity in admissions
and institutional type
System-wide Averages
Most current data from all System Universities
102
Establishing Benchmarks
Benchmark Average For each measure, the average
and standard deviation of the appropriate peer
group are calculated The benchmark bound (Upper
Bound) is calculated by adding one standard
deviation to the average For Personnel Ratio
and Instructional Cost per FTE Student
(decreasing measures), the benchmark bound
(Lower Bound) is calculated by subtracting one
standard deviation from the average Benchmark
Performance Evaluation The university result is
compared to the peer average and the benchmark
bound
103
Establishing Benchmarks
104
Evaluation Based on Benchmark
105
Evaluation Based on Benchmark
106
Evaluation Based on Benchmark
Institutional Peer Averages
University Selected Peers
Data Sources IPEDS (National Center for
Education Statistics) Surveys Completions, Fall
Enrollment, Fall Staff, Finance
Degrees Awarded Degree to Enrollment
Ratio Bachelor's and Masters
Employee Diversity Percent of Faculty who are
Minority
Personnel Ratio Total Personnel Compensation as
a Percent of Total Expenditures and Transfers
107
Evaluation Based on Benchmark
Institutional Peer Averages
University Selected Peers
Example
Employee Diversity Percent of Faculty who are
Minority
East Stroudsburg University
108
(No Transcript)
109
Evaluation Based on Benchmark
110
Evaluation Based on Benchmark
National Cluster Averages
Peers Selected by Selectivity Range (Fall 2002
Average Combined SAT Score) and Carnegie
Classification
Data Source CSRDE (Consortium for Student
Retention Data Exchange)
Second Year Persistence Overall and Minority
(Black and Hispanic)
Graduation - Four Year and Six Year Percent of
Students who Graduated Overall and Minority
(Black and Hispanic)
111
Evaluation Based on Benchmark
National Cluster Averages
Example
Percent of Students who Graduated in Four Years -
Overall
West Chester University Peers Selective
Masters Institutions
112
(No Transcript)
113
Evaluation Based on Benchmark
114
Evaluation Based on Benchmark
System-wide Averages
Most current data from all System Universities.
Data Sources Common Cost Accounting Report,
University Submissions
Faculty Productivity Total Credits per FTE
Instructional Faculty
Instructional Cost Undergraduate and Graduate
(Masters) Cost per FTE Student
Faculty Terminal Degrees Percent of Full-time
Tenured or Tenure Track Instructional Faculty
115
Evaluation Based on Benchmark
System-wide Averages
Example
Instructional Cost Masters Cost per FTE Student
All Universities
116
(No Transcript)
117
Evaluation Based on Benchmark
System-wide Averages
118
Benchmark Evaluation Report Presentation
Type of External Standard
Actual (33.02) is higher than benchmark upper
bound (24.76) Benchmark Exceeded
Actual (4,376) is between the benchmark average
(4,718) and the benchmark upper bound (4,282)
Benchmark Met
119
Benchmarking and Performance EvaluationsQuestion
s and Answers
120
System Performance Targets
121
System Performance Targets

• Leading the Way
• System Strategic Plan
• Approved by Board of Governors on October 14,
  2004
• Adds third category of comparison
• Quantitative System performance targets
• To be reached by 2009
• Consistent with the strategic goals and vision
  of the Plan
• Progress and achievement
• Reflected in performance funding program
  allocations

122
System Performance Targets

• Designed to raise the level of performance
• By establishing targets for System-wide
  performance
• Established for each measure
• Sets a target for the System average level of
  performance
• Targets set using one of the following
• a) External benchmarks,
• b) System performance projections,
• c) Top peers for each university, and
• d) Top five performing System universities for
  each measure
• The targets and bounds
• Designed to be challenging yet achievable

123
System Performance Targets

• Performance targets are expectations of overall
  improvement with some universities expected to
  improve more than others.

124
System Performance Targets
125
System Performance Targets
126
Comparison to System Performance
TargetEvaluation of Performance
127
Comparison to System Performance
TargetEvaluation of Performance
128
Comparison to System Performance
TargetEvaluation of Performance
Inverted Evaluations Total Personnel
Compensation as a Percent of Total Expenditures
and Transfers and Masters/Undergraduate Cost per
FTE Student
129
System Performance Targets and Performance
EvaluationsQuestions and Answers
130
Afternoon Session Topics Covered

Performance Funding Methodology Performance and
Distribution Methodology Review of FY 2004-2005
Performance Funding Calculations and
Distributions Question and Answer Period
Wrap-up Session 330 p.m. to 400
p.m. Comments University Presidents Chancellor

131
Afternoon Session Topics Covered

Afternoon Session 100 p.m. to 330
p.m. Performance Funding Methodology Performanc
e and Distribution Methodology Review of FY
2004-2005 Performance Funding Calculations and
Distributions Question and Answer Period
Closing Comments 330 p.m. to 400
p.m. Comments University Presidents Chancellor

132
Performance Funding Program Guidelines
The distribution of performance funding dollars
to performance categories, measures, and
sub-measures based on performance is as
follows Divided equally among the three
comparison categories Institutional Improvement
(comparison to historical baseline) Comparative
Achievement (comparison to benchmarks) Performanc
e Target Attainment (comparison to System
Performance Target). Within the three
performance categories Dollars are then divided
evenly across the eight measures, and, equally
across the sub-measures.
133
(No Transcript)
134
Performance Funding Program Guidelines
Distribution to Exceeded vs. Met
Categories Dollars awarded in the Exceeded
performance category should be (approximately)
twice that awarded in the Met category.
Rule Universities with performance in the
Exceeded category receive an award that is
(usually) three times that of those with
performance in the Met category. Size
Adjustment The total University performance
award is adjusted to University size as measured
by full-time equivalent students at the
undergraduate and graduate levels.
135
(No Transcript)
136
Calculating Allocations to Performance
Categories Counts of Universities in Exceeded and
Met Performance Categories
8 Universities Exceeded
6 Universities Met
137
(No Transcript)
138
Calculating Per FTES Allocations to Performance
Categories Total FTES in Exceeded and Met
Performance Categories
Total FTES for Universities in Exceeded
Category 56,283.8
Total FTES for Universities in Met Category
35,640.6
139
(No Transcript)
140
(No Transcript)
141
(No Transcript)
142
(No Transcript)
143
System Performance Funding Program Guidelines
Special Cases Redistribution of unallocated
funds When No University performance falls in
either the Met or Exceeded categories The
funding pool for those measures is evenly
redistributed to the Met and Exceeded portions
of the other measures, depending upon the
performance area where this occurs When no
University performance falls in the Exceeded
category The total funds for that particular
measure are reduced to 1/3 the original
allocation to that sub-measure. The remaining 2/3
of these funds are redistributed to the Exceeded
portions of the other measures, depending upon
the performance area where this occurs
144
System Performance Funding Program Guidelines
Special Cases (continued) In the evaluation of
performance relative to historical baseline
(individual university target attainment) if
a Universitys performance result for a measure
is baseline/target not met and, the
Universitys performance result for a measure is
System Performance Target Exceeded then,
the Universitys performance result for
comparisons to baseline is modified to
baseline/target met for the purposes of
performance funding.
145
Allocation of Undistributed Performance Funds
146
Allocation of Undistributed Performance
Funds Resulting from no universities performing
at an Exceeded level
Total Undistributed Funds 1,954,477 Baseline/Ta
rget Attainment 0 Benchmarking
902,990 System Performance Target 1,053,488
147
(No Transcript)
148
Allocation of Undistributed Performance
Funds Benchmarking Pool 902,990
149
Allocation of Undistributed Performance
Funds Benchmarking Pool 902,990
BL received a total of 231,117.69 in performance
funding for exceeding benchmarks (across all
measures). 231,117.69 is 6.69 of 3,453,126,
the total Distributed to the benchmark Exceeded
category Therefore, BL receives 60,437.09 of
the pool of undistributed Exceeded funds of
902,990. 902,990 .0669 60,437.09
150
Allocation of Undistributed Performance
Funds Benchmarking Pool 1,053,488
151
Performance Funding Allocations Totals by
University
152
(No Transcript)
153
Performance Funding Allocations Totals by
University
154
System Performance Targets and Performance
EvaluationsQuestions and Answers
155
Closing CommentsUniversity PresidentsChancellor