Program Review Data Collection Project

1
Program Review Data Collection Project

• Brought to you by
• The Data Delivery Team
• (DDT)

2
What is the DOIs Program Review?

• Over the course of the summer our UHCC DOIs
  worked together to provide the system with a set
  of program review data requirements that would be
  known as the Irreducible Core Indicators.
• These program review data elements were intended
  to be the minimum data requirements for
  instructional program reviews.
• By creating a commonly shared language we will be
  better prepared to compare programs across
  campuses.

3
Where did the 13 data elements come from?

• Since committing to do our program reviews it
  became painfully clear that we would have to be
  moving quickly to determine what data we would
  need at the local level so we could begin to
  develop a tool to retrieve the data.
• With this in mind we selected 13 key data
  elements from the DOIs list so that we could
  begin the process and move forward with
  development.
• Since we currently do not have a reporting
  platform that could be used to retrieve this data
  it was necessary to design and develop a dynamic
  reporting tool, then test and retest the data
  output until it was being reported as specd.

4
What data is being provided to my program?

• We have taken the 13 chosen data elements from
  the DOIs Program Review document and have gone
  through a number of iterations to ensure that the
  data we are providing will prove to be useful.
• The primary purpose of this presentation is to
  provide clarity as to how the data is being
  collected, filtered, and displayed to you.
• The intention here is to reduce the number of
  questions you may have so you will feel confident
  about using the data in your program reviews.
• Click here for the list of data elements in part
  III.
• Program Review

5
A word about the timing of the data snapshot...

• Its imperative that the same data snapshot be
  used when doing analysis on data from different
  sources (MAPS reports, Program Health Indicators,
  etc.)
• The data in Banner is constantly being updated.
  It differs from the first day of class to
  census date to closing date.
• Unless otherwise noted, the data snapshot for the
  13 following data elements is
• Semester specific data as of the last update in
  Banner.
• For the most part data is last updated during
  closing each semester, with a few exceptions
  change of grade, incomplete grade make up, and
  administrative exceptions.

6
2 Distinct Program Course Categories

• Program courses are broken into 2 categories
• "Student Required Core" (SRC) courses are primary
  program courses required of the student. This
  does not include general ed, remedial, or
  elective courses. For example SRC courses for
  Accounting could include ACC 130, OAT 29, or ICS
  101. (note the course alphas are different).
  These apply to all except 8 - 10.
• "Program Specific Courses" (PSC) courses are a
  subset of the SRC, used to help identify staffing
  sufficiency. Note that the program and course
  alpha are the same. For example ACC 130, ACC 20,
  or ACC 201. These apply to 8 - 10.

7
1. Number of Majors (Unduplicated headcount of
declared majors for both semesters.)

• What it means to the DDT
• Headcount of ALL registered students with a
  declared HawCC major for each semester. For the
  academic year this will be reported as
  unduplicated headcount by program.
• Unduplicated headcount is defined as a student in
  Fall OR Spring semesters, OR both. By program
  they will only be counted once over the academic
  year.
• These are students that were registered as of the
  start of the semester, regardless of whether they
  may have dropped later.

8
2. Student Semester Hours for program majors in
all program classes.

• What it means to the DDT
• These are all students where the primary campus
  is HawCC (PriCampHAW) and they had an active
  registration at some point in the semester.
• Student semester hours (SSH) are the sum of all
  attempted credits in all Student Required Core"
  (SRC) classes.

9
3. Student Semester Hours for all program
classes. This will give a measure of crossover
interest in the program.

• What it means to the DDT
• Student semester hours (SSH) are the sum of all
  attempted credits in all Student Required Core"
  (SRC) classes, regardless of campus or major.

10
4. FTE Course Enrollment. SSH for all program
classes/15

• What it means to the DDT
• Using the output SSH from item 3 SSH
  attempted divided by 15 for any semester. These
  are students that are declaring ANY primary
  campus--not just HawCC.
• FTE for a single semester is SSH/15.This number
  can be reported for the academic year ((SSH Fall
  SSH Spring)/30) as well and reported as an
  average.

11
5. Number of classes taught (n).(n) corresponds
to (n) in 6 and 7.

• What it means to the DDT
• Number of all "Student Required Core" (SRC)
  courses.
• This is each and every section taught for this
  program.
• This does not include cancelled sections.

12
6. Average Class Size.

• What it means to the DDT
• (The sum of all class enrollments)/number of
  class sections taught (n) from 5.

13
7. Average Class Fit

• What it means to the DDT
• Average Class fit is the ((sum of enrollments/max
  enrollment)/number of classes taught) x 100
• For example when enrollments 15, 18, 20 and
  max enrollment for each is 30, 20, and 15
  respectively and 3 classes are taught 15/30
  18/20 20/15 .50 .90 1.33 2.73/3
  classes taught .91 x 100 91 Avg class fit.

14
7. Staffing Sufficiency

• Up to this point (1 - 7) we have been using
  "Student Required Core" (SRC) courses for all
  calculations.
• For the following items 8 through 10 we will be
  using "Program Specific Courses in order to
  calculate Staffing Sufficiency.
• For items 11 through 13 we will return to using
  "Student Required Core" (SRC) courses for the
  remainder of the calculations.

15
8. FTE of BOR appointed program facultyHeadcount
of all on tenure, tenure track, and multi-year
contract faculty

• What it means to the DDT
• This is further defined as Full time and Part
  time faculty only--not lecturers.
• FTE is for PSC courses taught by BOR Faculty.
• This is the FTE that were hired into the program.
• We will collect this information from the Program
  Initiators since the information is not kept in
  Banner.
• This data will be hard coded into the Program
  Review.

16
9. FTE for classes taught in programCredits
taught/full teaching loadFull teaching load is
generally defined as 27 or 21 credits depending
on program

• What it means to the DDT
• Credits taught here is defined as credits for
  program specific courses (PSC) taught by both
  faculty and lecturers.
• Full teaching load is 27 credits for Liberal Arts
  programs and 21 credits for CTE programs.

17
10. Student/Faculty ratio(Total SSH/15)/FTE
faculty -- Note FTE faculty is FTE for classes
taught in program (from 9)

• What it means to the DDT
• Item 4 FTE Course Enrollment (calculated using
  the PSC)/item 9, which is FTE faculty.
• This will be reported for the academic year using
  a full teaching load of 27 credits for Liberal
  Arts programs or 21 credits for CTE programs.
• The Student/Faculty ratio will be reported when
  results are gtor to .5, they round up to the
  next whole number.

18
11. Unduplicated number of Degree/certificatesNot
e All degrees and certificates awarded in a
program in a year to a single student are counted
as one item.

• What it means to the DDT
• The following items, 11-13 will be reported
  based on the Student Required Core (SRC) courses.
• This is the highest degree awarded for the
  program pulled directly from the degree table in
  Banner.
• Reported for the academic year.

19
12. Cost of program per student major.Regular
salaries, overload, lecturer costs, supply and
maintenance, and amortized equipment/(SSH of
majors /15)

• What it means to the DDT
• Costs/(item 2/15) for a single semester. Add
  both semester costs up for the academic year.
  Regular salaries, overload, lecturer costs,
  supply and maintenance, and amortized equipment
  will be included in the data request from the
  program leads.
• We will need the cost to run the program before
  we can run this item in our report.

20
13. Persistence of majors year to year

• What it means to the DDT
• A student with the declared major that moves from
  spring from one academic year to the fall of the
  next academic year.
• The declared major need not have a passing grade
  (fail or drop) to be included in the persistence
  pool.
• We will be reporting on our 3 year academic
  period. This means that we will report out
  Spring 03 to Fall 03, and Spring 04 to Fall 04
  persistence.

21
Opportunities

• The core (program) courses or "Student Required
  Core" for each program are not flagged as such in
  the Banner system or elsewhere. This requires us
  to use the General Catalog and using a "best
  guess" approach, then a review by the Program
  Initiators for accuracy. This in itself builds
  in an additional margin for error. We should
  consider the possibility of adding a field in
  Banner so that when the courses are added to the
  catalog they are accompanied with a code denoting
  "program course (SRC or PSC). This will aid in
  both speed and accuracy of report development
  going forward.

22
Opportunities continued

• There were also many courses that were reported
  by program initiators as being courses that
  needed to be included in their program profile
  but were NOT listed in the catalog. We need to
  find out why this is occurring so that we can
  begin to rely on the courses as they are listed
  in the catalog. What happens when a course is
  added after the catalog is printed? How come
  these courses are not being added the next year?
  Who is responsible for review of what goes into
  the catalog to ensure that it is accurate for
  their program year after year?

23
Opportunities continued

• The process of determining what program courses
  belonged to what programs took way too long.
• First, a course profile was built based on what
  was in the catalog.
• All Math, English, and elective courses were
  stripped out.
• The program profiles were then sent out to the
  Program Initiators for review.
• Then, the exceptions came trickling incourses
  that were not listed in the catalog or Gen Ed
  courses that actually were part of the core
  program.
• Then, we had to go through the exception process
  for each of these which added to the workload.
• Then, the program profiles had to be updated with
  all of the exceptions.
• All of this because we had no reliable source for
  what program courses belonged to what programs.
• Much of this unnecessary work seems to be
  centered around the fact that the catalog is not
  being properly validated for accuracy prior to
  being published.

24
Opportunities continued

• For 12 we sent out a template to aid in the
  collection of program costsnone were returned.
• We need to clarify when and who should be using
  contact hour to credit hour conversions.
• We need to build an example of an amortized
  equipment schedule and give it to the program
  initiators.
• We need to determine which account codes the
  secretaries need to use in order to determine
  program expensesmy understanding is that they
  all have access to FMIS.
• Instructions need to be built for collecting
  expense data with examples, and then shared with
  the Program Initiators and their secretaries.

25
Opportunities continued

• There are unique courses in Banner that throw off
  program review data. For example, independent
  study/directed study classes, experimental
  classes, and variable credit classesthese
  classes often need waiver/substitution statements
  posted to their transcript.
• How we handle waiv/subs needs to be further
  developed. We currently have about 2500 lines of
  comments. This process needs to be mechanized so
  we can easily locate and rename courses then
  sort, copy and paste them into our program
  profiles. This procedure will require a time
  investment for process development and data clean
  up.

26
Opportunities continuedAre we collecting all of
the data of interest? How about

• admitted students into a classified program that
  were not enrolled on the first day of the
  semester?
• regarding SSH? Are we only interested in students
  that stay enrolled? Or are we concerned about
  any student that drops at any time? If yes,
  dropped during the erase period, or after the
  erase period? For example, if 50 of the class
  withdrew by the end of the semester, do we need
  to know that data?
• interest in General Ed or remedial courses?

27
Opportunities continued

• Determining Full/Part time Faculty and who is a
  lecturer will be included in the data request
  from the Program Initiators until its determined
  how to load the information into Banner and then
  who will provide the necessary training.
• Determination of full teaching load (other than
  the reported 27 or 21 credits already reported
  for the program) will be included in the data
  request from the Program Initiators until its
  determined how to load the information into
  Banner and then who will provide the necessary
  training.
• Determination of Regular salaries, overload,
  lecturer costs, supply and maintenance, and
  amortized equipment will be included in the data
  request from the Program Initiators. We need to
  create an example expense template to help our
  program initiators better understand what account
  codes to use and how to handle equipment
  amortization.

28
Questions?

• If you would like to discuss how we have arrived
  at the data for your program please do not
  hesitate to contact us for clarification. We
  will work to get an answer to you.
• Thanks for your support in this important work as
  we move forward. This is the first time in a
  long time that program reviews have been required
  and we are all learning together. Lets try to
  enjoy the process!
• Thanks from the Data Delivery Team.
• Shawn Flood 974-7512 sgflood_at_hawaii.edu
• Marv Kitchen 974-7710 kitchen_at_hawaii.edu
• David Loeding 974-7753 loeding_at_hawaii.edu