Balancing Performance Measures in an Academic Setting

1
Balancing Performance Measures in an Academic
Setting

• Ramona Houmanfar, Jared Chase, Charna
  Mintz-Resudek, Thomas Boyce, Patrick M. Ghezzi,
• Linda J. Hayes, Michele D. Wallace Larry
  Williams
• University of Nevada, Reno

2
Purpose

• To provide a data-based description of the
  processes and outcomes associated with the
  behavioral system analysis of the Behavior
  Analysis Program at UNR.
• Discuss certain process related elements that
  are unique to an academic.
• Plans for ongoing enhancement.

3
Mission
 
Competitors (Schools and Businesses)
PROCESS (Graduate Training)
INPUT (Students)
OUTPUT (Alumni)
FEEDBACK
FEEDBACK
RECEIVERS (Employers)
4
Background Information

• Unique Graduate Program
• Self-capitalized model of training
• Important need to arrange contingencies to
  balance entrepreneurial and academic activities
• Collaborative effort with Abernathy and
  Associates
• TPS balances entrepreneurial and academic
  activities

5
Mission Statement

• To produce skilled behavior analysts who meet
  the needs of the community and contribute to the
  vitality of Behavior Analysis as a scientific
  discipline in a socially valid and cost effective
  manner.

6
Inputs students admitted
7
Inputs Total Students and Faculty
8
Total Performance System (TPS)

• TPS is an organization wide incentive-pay system
  based on targeted, objective, personal
  performance measures (Abernathy, 1999)
• It uses balanced performance scorecards that
  allow for the objective measurement of
  performance across an organization (Abernathy,
  1996)
• Cascading scorecards tie the performance of the
  individual employee to each level of the
  organization as well as to the overall objectives
  of the organization

9
Organizational Critical MeasuresBA Program At UNR

• Strategic Scorecard
• Measure Weight
• Revenue 20
• Expense/Revenue 20
• Cash Flow 10
• Academic Progress 20
• Professional Performance 20
• Consumer Satisfaction
  10

10
Academic Scorecard Measures
Driver Measures

• Critical Measures
• Grade Point Average
• Progress Toward Completion
• Program Participation
• Professional Development
• Research Productivity

11
Academic Scorecard Measures
Driver Measures

• Critical Measures
• Grade Point Average
• Progress Toward Completion
• Program Participation
• Professional Development
• Research Productivity

12
Academic Scorecard Measures
Driver Measures

• Critical Measures
• Grade Point Average
• Progress Toward Completion
• Program Participation
• Professional Development
• Research Productivity

13
Academic Scorecard Measures
Driver Measures

• Critical Measures
• Grade Point Average
• Progress Toward Completion
• Program Participation
• Professional Development
• Research Productivity

14
Academic Scorecard Measures
Driver Measures

• Critical Measures
• Grade Point Average
• Progress Toward Completion
• Program Participation
• Professional Development
• Research Productivity

15
Data collection

• Student and advisor meet at the beginning of the
  semester to set goals
• Student and advisor meet in the middle and at the
  end of the semester to review progress on
  previously set goals

16
Data collection

• Academic progress scores are calculated for each
  student
• (score goals met / goals set)
• Advisement scores are calculated for each advisor
  
• (score avg. score of students advised)

17
Driver Measures Progress Toward Completion
18
Key Academic Measures
.74
19
Cascading System
Advisor A
20
Academic ProgressCurrent Data
Research Participation Award
Trimester Based
Quarter Based
Semester Based
21
Professional Performance Measure

• A majority of students work for the BA program
• Faculty supervisors evaluate their performance on
  a semester basis
• Students are evaluated in two areas
• Productivity
• Quality

22
Critical Measure Productivity

• Students are expected to earn 100 in all
  productivity measures
• These driver measures include
• Task completion
• Timeliness of task completion
• Timeliness to meetings
• Meetings attended
• Students earn a percentage score at the end of
  the semester

23
Critical Measure Quality

• Students are rated on a scale from 1-4
• It is expected that a student earn 4s in all
  measures
• These driver measures include
• Satisfaction of task completion
• Interactions with clients
• Interactions with co-workers
• Interactions with the supervisor
• Level of independent task completion

24
Weight of Measures
25
Professional Performance Current Data
Research Participation Award
Trimester Based
Quarter Based
Semester Based
26
Consumer Satisfaction Measure

• This measure evaluates student satisfaction with
  advisement and supervision
• Students rate advisors and supervisors on a 4
  point scale in a variety of areas
• Ratings are averaged together resulting in a
  consumer satisfaction score

27
Critical Measures

• Faculty are evaluated in the following areas
• Adherence to BA program protocols
• Timeliness
• Accessibility
• Advisement
• Supervision
• Professional Interactions

28
Consumer SatisfactionCurrent Data
Research Participation Award
Trimester Based
Semester Based
Quarter Based
29
Outputs Graduate Alumni

• 17/20 M.A. Students (1990-2000) received degree.
• 15/38 Ph.D. Students (1990-1997) received degree.

30
Outputs Revenue
31
Alumni Feedback

• 7 of the 15 doctoral alumni responded to an
  on-line questionnaire
• Current placements (of 7 respondents)
• 4 academic, 1 clinical consultant, 1 internal
  consultant, 1 clinical research
• 7 of 7 respondents currently in desired work
  environment.
• Initial placements
• 6 of 7 in desired work environment

32
Processes Alumni Feedback Most Important
Training Experiences (7 respondents)
33
Processes Alumni Feedback Weakest areas of
training (7 respondents)
34
Competitors

• Academic
• BA programs, IO Programs, Business Management,
  Education, Rehabilitation
• Business
• Human service providers and consultation firms in
  Nevada Northern California

35
The Receiving End of the System

• Distribution of a survey among established
  placement sites (in progress)

36
Discussion

• The process has been long but successful
• Further development of the project
• Fine tuning of TPS measures to bring the current
  system at UNR closer to the ideal model
• Software installment for TPS
• Graduate programs can benefit from the
  implementation of the behavioral system analysis
  technology including the TPS model