Institutional Course Evaluation Solution

1
Institutional Course Evaluation Solution

• Faculty Senate Executive Committee
• September 12, 2012
• Carol VanZile-Tamsen, Ph.D. Associate Director,
  Office of University Accreditation and Assessment
• Joseph Mantione, Assistant Vice Provost,
  Institutional Analysis

2
Currently at UB

• Five different course evaluation solutions are
  used across the 12 units
• Additional systems used by academic support units
  delivering instruction

3
Currently at UB

• In some areas, preparation of evaluation forms
  for course offerings is an administrative
  nightmare
• Students enrolled in courses offered by different
  units do not have a single place in which they
  evaluate all of their courses
• In many areas, response rates are too low for any
  meaningful conclusions to be drawn

4
Currently at UB
5
ConnectEdu in the College of Arts and Sciences

• UBCATS - University at Buffalo Course and
  Teacher Survey - A web-based software
  application that allows students to anonymously
  evaluate courses and instructors online.
•  
•  
• Data Upload Process
• Upload of Course, Enrollment, Faculty, Student,
  TA, and Supplemental Instructor tables to the
  site
• Fall 2011
• 71,658 Student Enrollment Records
• 17,095 Student Import Records
• 2,350 Course Import Records
• 1,212 Faculty / Department Records
• 396 Teaching Assistants Records
•  
•  
• Departmental Access Process (Dept. Chair,
  Assistants)
• Set up the survey keep it open for a month
  close it two days after the last day of final
  exams.

6
ConnectEdu in the College of Arts and Sciences

• Communication Process
• Place student announcement and survey link on
  myUB
• Send Emails (17,078 initial emails) to Students
  to
• Inform them that the evaluation site is open
• Invite them to log in and do the evaluation).
• Send Emails (1,137 initial emails) to Faculty to
• Inform them that the survey is open
• Ask them to monitor response rates (real time)
• Remind their students to do the evaluation
• Send follow-up weekly emails/reminders to
  students with incomplete surveys
• (15,719, 14,072, 12,652, 11,733 emails in
  Fall11)
• Send final Email to Faculty at the close of the
  survey to let them know that they can log in and
  access the evaluation results the day after web
  grading closes for the semester.
• Reporting Process
• Distribute our own custom Summary Reports to
  Dept. Chair and Director of Graduate Studies.

7
ConnectEdu in the College of Arts and Sciences

• UBCATS Challenges
• 1 Data upload- creation of new HUB querying
  tools
• 2. Undeliverable Emails students not
  using/checking their UB email
• 3. Lack of Summary Reports
• 4. Departmental buy in lack of ownership
• 5. Limited technical support
• 6. Confusion among students not all UB courses
  are part of UBCATS
• 7. I cant log in!
• 8. Reports are not accessible to students
• 9. Response rate
• 10. Survey fatigue
•  
• To some degree, the other 11 schools, Athletics,
  MFC and EOC all do the same thing and incur the
  same PSR costs.
•  
• They also have the same challenges.

8
The Benefit of A Single Solution

• Uniformity in Administration leading to greater
  efficiency and more reliable results
• Greater transparency for students and one-stop
  evaluations may encourage greater response rates
• Availability of Data and Reports for Assessment
  Purposes
• Faculty review and development
• Program Review
• Accreditation Reporting

9
Course Evaluation Items

• For comparability of ratings across courses and
  instructors, a core of common questions could be
  useful
• 3-5 questions
• These would be developed jointly by Faculty
  Senate Executive Committee and Associate Deans
  Council
• Beyond that, we would promote the greatest
  flexibility possible for units in selecting
  questions

10
Vendors already on campus

• Campuslabs http//www.campuslabs.com/ (cost
  proposal submitted)
• ConnectEdu http//www.connectedu.com/postsecondary
  -solutions/college-evaluation-suite/college-course
  -evaluation.html (cost proposal submitted)
• Digital Measures http//www.digitalmeasures.com/co
  urse-evaluation-software/
• EValue https//www.e-value.net/
• SmartEvals http//service.onlinecourseevaluations.
  com/
• Blackboard http//www.blackboard.com/ (no course
  eval solution at present)

11
Conclusions

• Regardless of platform chosen, there are many
  good reasons to have an institutional course
  evaluation system
• We hope you will consider data use and reporting
  reasons, as well as fiscal reasons, as you
  consider this issue.