Student Evaluation of Teaching SET

1
Student Evaluation of Teaching (SET)
Briefing for Coordinators 10/17/2002

• Dr John Bacon-Shone
• Director, Social Sciences Research Centre, HKU
• Email johnbs_at_hku.hk

2
Student Evaluation of Teaching

• All the materials, including the SET handbook for
  download, course report samples and the last set
  of questionnaires can be found on the Intranet
  at
• http//intranet.hku.hk/ssrc/index.htm
• This presentation can be found at the SET link on
  the SSRC home page
• http//www.ssrc.hku.hk/

3
SET services

• Design and printing of form survey questionnaires
• Design of Web-based survey questionnaires
• Processing and reporting of all survey
  questionnaires

4
Services not includedin SET

• Surveys not under the guidelines of STQC
• Questionnaires which are too elaborate
• Surveys for research purpose
• Customized reports
• Processing of open-ended comments

5
The pivotal roles of the academic co-ordinators

• Determine the number of questionnaires required
  to evaluate courses, different types of
  programmes, teachers, tutors
• Draft/modify these questionnaires
• Determine the number of forms for each
  questionnaire
• Schedule administration of survey
• Schedule the dispatch of the completed forms to
  SSRC, normally in a single submission
• Collect suggestions from colleagues on the
  revision of questionnaire for the next round

6
SET procedure in 2002-03

• At least 4 weeks before the evaluation date
• The SET coordinator provides the SSRC with the
  following information for each course
• Course code and title
• Instructors (and tutors if needed)
• Enrollment figure
• Number of blank forms needed
• and reviews with SSRC staff the questionnaires
  used last year.

1.
7
SET procedure in 2002-03

• At least 2 weeks before the evaluation date
• The SET co-ordinator finishes proof-reading the
  camera-ready questionnaire and returns it to SSRC

2.
The department collects a survey package for
evaluation upon notice from SSRC and conducts the
surveys in due course
3.
8
SET procedure in 2002-03

• The department returns to SSRC
• All completed forms in a single submission
• A completed cover sheet on top of each envelope
  of completed forms
• A completed Excel file summary and a packing list

4.
SSRC scans the returned forms in the order of
arrival and generates reports
5.
The department collects reports upon notice from
SSRC
6.
9
STQC policy recommendations

• Operate under STQC guidelines
• Faculty may endorse core items to be used by all
  departments
• Course Evaluation results must be shared with
  students
• Student evaluation is made one of the established
  components of the Universitys quality assurance
  system

10
STQC policy recommendations (continued)

• Evaluation results should be treated by the SSRC
  as confidential and returned only to the heads of
  departments/teaching units.
• Evaluation results must be used to improve
  teaching.

11
STQC guidelines for operation

• Some standard questions required
• Conditions for separate Questionnaires instead of
  an integrated Questionnaires
• Co-ordination of courses jointly taught by more
  than one department
• All courses must normally be evaluated
• Questionnaires are to be designed by dept,
  endorsed by faculty and processed by SSRC.
• Formative evaluation may be undertaken by
  teachers with the help of the SSRC

1.
2.
3.
4.
5.
6.
12
STQC Guidelines for operation (continued)

• Two concluding overall items, one in part I (on
  course) and one in Part II (on teachers)
• Use standard descriptors and possibly a five
  point scale
• Standard reports to the heads of departments/
  teaching units
• Customized reports can be done by departments
  themselves using the data file supplied by SSRC

7.
8.
9.
10.
13
STQC Guidelines for operation (continued)

• All reports must use a course code
• A single submission by each dept of completed
  forms to SSRC
• Exceptions to Guideline 12.
• To add more specific items in Part III
• Issues for evaluating teachers of a course
  jointly taught by a large number of teachers

11.
12.
13.
14.
15.
14
STQC Guidelines for operation (continued)

• Give SSRC sufficient time to design, print and
  process questionnaires.The printing quantity
  estimates should be based on student enrolment
  figures.
• Department are encouraged to trial on-line
  Web-based evaluation with the help of SSRC
• Computer Centre to assist departments and SSRC in
  on-line operation

16.
17.
18.
15
A Sample Questionnaire
16
A Sample Questionnaire (continued)
17
A Sample Questionnaire (continued)
18
A Sample Questionnaire (continued)
19
A Sample Questionnaire (continued)
20
A Sample Questionnaire (continued)
21
A Sample Questionnaire (continued)
22
Reports disseminationa suggested matrix
23
Administration for SET
Pre-evaluation essentials

• at least 4 weeks in advance of the number of
  copies required for each questionnaire
• SSRC will notify departments by fax or telephone
  to collect the printed questionnaires, copies of
  the instruction sheet and the cover sheet
• Department should, if possible, help their
  teachers fill in the cover sheets for all the
  courses and teachers to be evaluated and then
  paste them on top of each envelope of forms
  before handing them over to the teachers in
  charge of the courses

24
Administration for SET

• Teachers should supervise the evaluation by
• first explaining the objective of the evaluation
  to the students
• reassure students of anonymity
• tell them how much time they will have to
  complete the form
• explain how they should fill in the course code,
  teacher identity, and student group information
  on the front page of the questionnaire and get
  them to do so immediately.

25
Administration for SET

• Record of courses evaluated
• Departments should count the returned envelopes
• prepare a packing list of the course evaluated,
  in a printout and in soft copy on a computer disk
  to accompany the envelopes of forms to SSRC

26
Administration for SET

• Order of processing
• Based on the order in which envelopes of forms
  are received by the SSRC
• SSRC cannot allow queue jumping
• Department needs the reports urgently, it must
  make a written request in advance and their forms
  must reach the SSRC at least fourteen working
  days before the date when the reports are required

27
Administration for SET

• Reporting
• SSRC provides department heads with a set of
  standard reports
• A disk contains the course reports and teachers
  report in a Rich Text Format (RTF) and the
  scanned raw data in a comma-delimited format

28
Administration for SET

• New for this year
• Trial of department summary for overall course
  effectiveness question for every course
• Request from HKU admin to add questions on
  Contact hours, English used and Hours of oral
  presentation

29
Online evaluation

• Advantages
• No need to hand out and collect paper forms
• Comments are compiled
• Disadvantages
• Very low response rates unless administration is
  supervised.

30
Online evaluation

• Two ways
• SSRC homebrew
• replicates paper process, except online
• uses student university number as password
• WebSET
• Instructor/Coordinator can arrange design using
  UIS system