Briefing session on Online Student Evaluation of Teaching and Learning for colleagues supporting the administration of SETL

1
Briefing session on Online Student Evaluation of
Teaching and Learningfor colleagues supporting
the administration of SETL
Prepared by Social Sciences Research Centre
of The University of Hong Kong
1
2
SETL process

• Online process must cover all undergraduate
  courses (including Common Core courses) on 2
  semesters system
• TPG courses can be evaluated online if they also
  use 2 semester system. All other TPG courses
  must use the paper form for the evaluation
• The online SETL evaluation will be conducted for
  4 weeks until the start of the assessment period
  (i.e. this semester from Nov 9 to Dec 7, 2015)

2
3
SETL process

• SETL form covers course, teachers, tutors and
  demonstrators
• Uses SIS database record of course/teacher/tutor/d
  emonstrator
• An option in SIS is available for joining
  different courses in the evaluation, only one
  overall report will be generated
• For information, MBBS and Dentistry online SETL
  evaluation were handled separately

3
4
SETL process

• Must use standard forms unless Chairman of TLQC
  approves
• Online yields lower response rates, but better
  open-ended responses, which is good for
  improvement
• To ensure good response rates, need to ensure
  that students are given time at the end of a
  class to complete the form (although they can
  complete any time before the deadline)

4
5
SETL process

• Cannot currently change teacher names in system,
  which come from Human Resources
• For the Department Items, a maximum 5 extra
  questions for both on-line and paper evaluation,
  please contact and confirm with SSRC staff, Ms.
  C.C. Ho at 3917-1623 or hocc_at_hku.hk (on or before
  Oct. 23, 2015 for this semester)

5
6
SETL process

• For the online SETL, all courses with enrollment
  of at least 2 students must undertake SETL
• For the online SETL, the teacher/tutor/demonstrato
  r parts are only done if there are at least 2
  students per teacher/tutor/demonstrator (unless
  VP agrees)
• For the paper SETL, all courses with enrollment
  of at least 5 students must undertake the
  evaluation (unless VP agrees), although the
  teacher/tutor/demonstrator parts are only done if
  there are such persons for the class

6
7
SETL process

• A maximum list of teacher/tutor/demonstrator is
  25 names
• Separate online forms for course, teacher, tutor,
  demonstrator as needed. Up to 25 teachers can be
  evaluated in each course by a student, but only 1
  tutor/demonstrator selected by a student from a
  list (or evaluated as a group) of each course
• All teachers/tutors/demonstrators must have staff
  ID/University No. (HKU requirement and needed for
  staff review)
• Tutor means person running tutorials,
  demonstrator means person responsible for lab or
  field sessions (role descriptions)

7
8
SETL Workflow
Prepared by Social Sciences Research Centre
of The University of Hong Kong
8
9
SETL Workflow of the Online SETL

• Departments/Centres/Schools/Institutions should
  enter/update the course enrollment information
  including all undergraduate and TPG courses in
  the Student Information System (SIS) especially
  the staff number linked to the name of
  teacher/tutor/demonstrator in each course and
  sub-class information 10 weeks before the
  assessment date (i.e. before 25 September 2015
  for this semester)
• All information must be updated for the
  appropriate semester
• Student enrollment should be updated
  automatically,
• but the rest of the information relies on the
  departments/centres/schools/institutions to
  update

9
10
SETL Workflow of the Online SETL

• SIS unit exports the course enrollment
  information of all undergraduate and TPG courses
  including
• Course code
• Course title
• Term (Semester 1/2/full Year)
• Student list in each course (student name and
  university no.)
• Teacher name list in each course (teacher name
  and staff number)
• Tutor and Demonstrator name list in each course
  (teacher name and staff number) and
• Jointly evaluated courses (course list)

10
11
SETL Workflow of the Online SETL

• The enrollment information received from SIS
  except the student list and indication of whether
  online or paper process will be sent out to
  departments/centres/schools/institutions
  (i.e. on or before 9 October
  2015 for this semester), so
  departments/centres/schools/institutions can
  check for errors and missing information (Please
  refer to Course List, Teacher List, Tutor List
  and Demonstrator List)
• Departments/Centres/Schools/Institutions should
  update any data amendments in the SIS system
  between 9 and 23 October 2015 for this semester

11
12
SETL Workflow of the Online SETL

• The final enrollment information of courses with
  teacher/tutor/demonstrator names will be sent out
  to departments (i.e. on or before 2 November 2015
  for this semester). If any errors occurs, please
  contact SSRC through email setl_at_ssrc.hku.hk
  immediately (i.e. on or before 6 November 2015
  for this semester).
• For information, SSRC will send out the
  invitation emails to all HKU student on 9
  November and 2 reminder emails to all HKU student
  on 30 November and 4 December, 2015 for this
  semester

12
13
Instructions for Online SETL
Before administration

• The teacher concerned should inform the students
  in advance the date of the online evaluation
  (i.e. soon after 9 November 2015 for this
  semester), and that the students are encouraged
  to bring a mobile internet device (such as
  laptop, tablet or smartphone) to class at the
  scheduled time.

13
14
Instructions for Online SETL
On the day of administration

• Similar to the paper exercise, the SETL
  questionnaire should NOT be administered by the
  course teacher, but faculty/department staff.
• The teacher should briefly explain the objectives
  of the student evaluation and then leave the room
  leaving it to faculty/department staff to oversee
  the process.

14
15
Instructions for Online SETL
On the day of administration
Faculty/Centre/Institution/Department staff
should read out

• Please go to the weblink http//setl.hku.hk where
  you should see a list of all forms that you need
  to complete. Your evaluation will be saved
  anonymously, without any identification. There
  are separate forms for the course, teacher (and
  if appropriate) tutor and demonstrator. There are
  instructions, an FAQ link and a link for you to
  report any missing courses at the weblink.

15
16
Instructions for Online SETL
During the evaluation period
Students should

• Access the SETL system via the weblink
  http//setl.hku.hk/ where there are instructions,
  FAQ and a list of forms for completion (separate
  links for the course, teacher, tutor,
  demonstrator forms)
• Complete the forms for the class for four weeks
  until the start of the assessment period (e.g.
  this semester from Nov 9 to Dec 7, 2015)
• if they cannot find the course that they have
  enrolled in, send an email to SSRC using the link
  on the page so SSRC staff can rectify this
  promptly

16
17
Workflow for Paper SETL

• Departments/Centres/Schools/Institutions should
  be responsible for the accuracy of SIS data
  before SSRC generates the SETL reports and inform
  SSRC about the course enrollment information for
  printing purpose.
• For any amendment of the evaluation form (after
  received the Chairman of TLQC approval) should
  inform SSRC 4 weeks before the evaluation date.
• Departments/Centres/Schools/Institutions should
  inform SSRC for printing at least 2 weeks before
  the evaluation date. Departments collects the
  blank forms one week before the evaluation date
  and carries out the evaluation.
• For the paper evaluation forms, departments
  should deliver to SSRC all completed forms,
  completed cover sheet on the top of each envelope.

17
18
SETL Workflow for Reporting

• The SETL results will be generated upon notice
  from SSRC after the examination result release
  date (e.g. 21 January 2016 for this semester) and
  the following report will be provided to each
  department/centre/school/faculty/institution
• an individual report of every teacher in each
  course
• an overall teacher report of every course
• a report for the tutor/demonstrator of every
  course
• (if applicable)
• a course evaluation report of every course
• a soft copy of all open-ended questions answer
  using online SETL (in MS excel format by question
  number and course code)
• an overall report on the undergraduate courses
  and
• an overall report on the TPG courses

18
19
SETL Workflow for Reporting

• Additional requests may be entertained at cost
• SSRC will upload the SETL results to the APA on
  or after 22 January 2016 for this semester (after
  the examination result release date 21 January
  2016)
• SSRC will upload the SETL results to the SSRC
  online SETL report system on or after 22 January
  2016 for this semester (after the examination
  result release date 21 January 2016)

19
20
Instructions for Online SETL

• My colleague, Mr. Samson Lee, will now
  demonstrate Online SETL, so you can see how the
  process works in practice
• http//setl.hku.hk
• My colleague, Mr. Samson Lee, will now
  demonstrate Online SETL report system, so you can
  see how the process works in practice
• http//setl.hku.hk/report

20
21
Instructions for Online SETL

• The SSRC Guide (Third Edition, April 2015),
• Briefing session on Online Student Evaluation of
  Teaching and Learning (6 October 2015),
• Briefing session on Online Student Evaluation of
  Teaching and Learning for Administrative and
  Clerical Staff (6 October 2015) and SETL Online
  DEMO are available
• http//www.ssrc.hku.hk/setl.html

21
22
Thank you! I am happy to answer questions.
Prepared by Social Sciences Research Centre
of The University of Hong Kong
22