Student Feedback on Learning and Teaching

1
Student Feedback on Learning and Teaching

• Peter Daly

2
Development of Student Feedback Working
Group(SFWG)

• QESC Initiative
• CELT-based Project (2004-2005)
• CED-based Project (2005-2006)

3
Terms of Reference (SFWG)

• Identify current practices for gathering and
  using student feedback at QUB
• Consider how the student feedback system may be
  enhanced
• Make recommendations on the development of staff
  and student support for the use of student
  feedback gathered at module level
• Consider examples of best practice on the
  collection an use of student feedback
• Review progress and report to the QESC

4
Sources of Information

• Government/government agency publications
  relating to Higher Education policies and
  practices in England and Scotland
• Review of recent research literature
• Questionnaire survey of QUB schools/Faculties
• Focus groups with undergraduate full-time
  students from each Faculty
• Focus groups with academic staff from each
  Faculty

5
Emerging Inter-related Themes

• Purposes of Teaching and Learning Feedback
• Collection and Analysis of Data
• Uses made by Stakeholders of Information Obtained.

6
Organisational Levels

• Student
• Tutor
• Module/course unit
• Programme of Study
• Subject
• Department/School
• Faculty
• Institution

7
Background Literature Policy Reports

• Cooke Task Group Reports 2001, 2002
• HEFCE HEI Report (2003)
• Brennan and Williams (2004) LTSN Guide
• SHEFC HEI Report 2005

8
HEFCE (2003)Main Purposes

• Enhancing the students experience of learning
  and teaching
• Contributing to monitoring and review of quality
  and standards
• Informing prospective students about the quality
  and standards of Higher Education Programmes of
  Study

9
HEFCE (2003)

• There is a need for clarity about differences
  between student feedback on satisfaction, on
  learning processes (study methods) and on student
  objectives and their achievement. The dangers are
  that questionnaires muddle these different kinds
  of feedback although it is not impossible to
  combine them into a single instrument (p.49)

10
Summary of HEFCE HEI Report (Ruth Williams)

• The need for greater clarity and agreed
  understanding within institutions about the
  purpose or purposes of collecting student
  feedback
• The need for greater clarity and agreed
  understanding about the use or,more likely, the
  uses to which feedback data will be put
• The need for greater clarity and agreed
  understanding about the needs of different users
  and the types of information, analysis and
  presentation they might require from feedback
  data
• The need for improved dissemination to students
  of the results of.feedback data

11
Background Literature Independent
Investigations

• Marsh Associates 1992, 1994, 1997
• Cashin (1995)
• Harvey (2001, 2003)
• Penny Coe (2004)
• Deem and Brehony (2005)
• Moore and Kuol (2005)
• Richardson (2005)
• Cohen (2005)

12
QUB Focus Groups

• Aims
• To review current module-level student feedback
  practices at QUB
• To identify areas for enhancement and improvement
• Representation (Schools/Faculties)
• Academic staff
• Students (Full-time undergraduates)

13
QUB Focus Groups Staff and Student Themes

• Purposes of Module Evaluation
• Evaluation Data Collection and Analysis
• Uses made of feedback by stakeholders

14
Purposes

• Monitoring students experience of learning and
  teaching
• Evaluating teaching quality
• Evaluating module content
• (Quality enhancement for the benefit of the next
  student cohort was seldom mentioned by students)

15
Data Collection and Analysis

• Staff-Student Consultative Committees had strong
  support from staff and students
• Major student demand for more evaluation during
  the module delivery period e.g. half-way stage or
  earlier by whatever means. End-of-module
  questionnaires cannot address this problem.
• Students welcomed mid-module evaluation
  strategies including SSCCs, staff-student
  seminars, minute papers at end of lectures etc.

16
Data Collection and Analysis

• Some students desired opportunities to provide
  feedback on all module presenters, separately,
  including teaching assistants
• Staff noted QUB Quality Handbook core questions
  did not address all significant issues raised in
  LTSN Guide e.g. library and ICT provision,
  pastoral care.
• Staff and students worried about questionnaire
  fatigue
• On-line questionnaire feedback- mixed reactions

17
Qualitative responses and more direct
staff-student contact

• Qualitative feedback was viewed very positively
  by staff and students
• The best type of information gets through the
  discussion, qualitative kind of discussion about
  hows it going?
• More direct contacts between staff and students
  had strong support.
• If there are difficulties and things we need to
  fix, tell us week 1, week 2, -we have a great
  e-mail, great Queens on-line system..they can
  give feedback any day, any time, any place and
  you dont wait.. or they come to see your staff.
  (Former Head of School)

18
Uses Source of Handbook Information

• Some module handbooks have information on
• Modifications made partly because of previous
  student criticism
• Grade performance fromm previous student cohort
• Views of previous external examiners
• ( Handbooks not available for all modules or
  lack basic information)

19
Uses

• A not unusual student perception was
• I know its a module review thats the extent of
  it. It is never brought back to us and we dont
  know where to find the information that comes
  from it .

20
Uses

• But if the tutors are, you know, seeing these
  comments and perhaps, like, they got negative
  evaluations or you know ,are they happy that they
  have to change the way that they do things? or
  like, are they just doing it because they (are)
  being told basically they have to?. it would be
  useful to know what the tutors think of their own
  evaluations
• (Student remark)

21
Uses

• I have to say I really do not know how that
  information is used, I mean, I think theres what
  is said about how it is used, how important it is
  do you remember the time we had balanced
  excellence (general laughter!) and I am not so
  sure - you look and see what people say about
  how it is used and you look and see what actually
  happens and I mean were just like students are,
  maybe, a bit picking up sub-texts, obviously we
  are too and you see that perhaps where, you know
  , good teaching, God, doesnt really matter at
  all, it seems . (Tutor observation)

22
Main Findings

• A significant amount of effort is invested in
  collecting feedback, but there are significant
  differences in approaches to how data is
  collected
• A lack of clarity and transparency as to who sees
  evaluation feedback and how it is used, emerged.
  Confidentiality and staff development were
  issues of concern to staff.
• Dissemination of results of feedback to students,
  including action taken, was very widely seen by
  students as inadequate.
• Some staff and students are cynical about the
  value of student feedback as it is currently
  collected and used.

23
Main findings

• A lack of clarity about and wide variability in
  provision of various mechanisms for interim
  (ongoing)
• feedback
• Significant interest was shown in increasing the
  mechanisms available for interim formative
  evaluation, including on-line questionnaires.
• ( End- of -module evaluations are only summative
  )

24
Recommendations

• The University should consider
• Preparing a clear statement on the purposes of
  student feedback, how the process will operate
  and how the information collected will be used
  and by whom(separate documents for staff and
  students)
• Reviewing and modifying the current procedures on
  student feedback as outlined in the University
  Quality Handbook

25
Recommendations

• The University should consider
• Developing a clear strategy on how the results of
  student feedback should be disseminated to
  students ( e.g. a summary of key points, action
  to be taken and timeline, reasons for not
  addressing specific issues etc.)
• Further developing the role of the Staff Student
  Consultative Committees as a mechanism for
  formative interim evaluation
• Piloting an on-line questionnaire as a mid-point
  formative module evaluation mechanism

26
Recommendations

• The University should consider
• Seeking more direct support from the Students
  Union for educating and training student to
  participate fully in evaluation exercises.

27
Way ahead Scotland as an Example

• The Scottish QAA/Higher Education Funding Council
  document (2005) is viewed by some commentators as
  state-of-the-art.
• Culture Change seen as a possibility in Scottish
  Higher Education Institutions
• A change from evaluation for accountability to
  the evaluation of student views for the express
  purpose of enhancement
• Students viewed as evaluation partners

28
Way ahead Scotland

• Increasing informal face-to-face staff/student
  consultation (not confined to reps)
• Fast-return type methods of data collection and
  analysis
• More investment in developing feedback skills
  for students and staff

29
Summative Evaluation Scottish Criticism

• Summative evaluationhas become self-sustaining.
  In many instances information is collected with
  no clear end use in sight- the purpose of
  evaluation has become the gathering of
  information rather than the analysis of
  information to promote change.