Enhancing the Education Environment at Queens First Annual Conference of the Centre for Educational

1
Enhancing the Education Environment at Queens
First Annual Conference of the Centre for
Educational DevelopmentQueens University
Belfast, 18-19 Sept 2006

• Enhancing Learning and Teaching
• What Role Can Research Evidence Play?
• Dai Hounsell
• University of Edinburgh
• www.ed.ac.uk/etl

2
BACKGROUND AND INTRODUCTION

• Evidence and Practice
• the lure of evidence-based and evidence-informed
  practices and policies
• evidence on teaching-learning and assessment
  practices
• how far can we generalise, and about what?
• the challenge of contingency
• Feedback and Its Discontents
• pervasive evidence of variable feedback
• (e.g. National Student Survey, 2006 QAA
  Learning from Subject Review, 2003 Krause et al.
  2005 Hounsell, 2003 Hounsell et al. 2005
  Carless, 2006)

3
RESEARCH AND FEEDBACK TO STUDENTS

• compelling evidence of the role of feedback and
  formative assessment in facilitating high-quality
  learning ( see e.g. Black et al. 2003 Nicol and
  MacFarlane-Dick, 2006)
• evolving conceptions of feedback (Sadler, 1998)
• what makes for effective feedback
• knowledge of results ?
• support and encouragement ?
• grasp of what high-quality achievement entails
• closing the loop (waxing and waning)
• action taken to close the gap, between desired
  goal and actual performance

4
KEYNOTE FOCUS AND AIMS

• research findings on guidance and feedback to
  students
• draws on biosciences data from the ETL Project
• aims to review
• findings from 1st round of data-gathering, and
  subsequent action by course team
• findings from 2nd round of data-gathering, on
  impact of measures taken
• outcomes of subsequent analysis
• implications for evidence-informed efforts to
  enhance learning and teaching

5
Enhancing Teaching-Learning Environments in
Undergraduate Courses (ETL Project) RESEARCH
DESIGN

• Aims
• to investigate ways of enhancing the quality of
  undergraduate learning and teaching, in a range
  of subject areas settings
• Samples and settings
• first- final-year modules in three departments
• Data-gathering
• student questionnaires and interviews with
  students staff
• Enhancement focus
• collection, analysis joint review of baseline
  data
• ? ? evidence-based collaborative initiatives

6
BIOSCIENCE STUDENTS PERCEPTIONS OF GUIDANCE AND
FEEDBACK

• The students overall perceptions of their
  courses were broadly positive across all of six
  bioscience course units surveyed
• Their experiences of the provision of guidance
  and feedback on assessed work, however, were much
  more variable
• In some units, students reported favourably in
  others, there were significant student concerns

7
BIOSCIENCE STUDENTS PERCEPTIONS OF GUIDANCE AND
FEEDBACK (Questionnaires)

• insert figure 2

8
STUDENTS CONCERNS ABOUT GUIDANCE AND FEEDBACK
(Interviews)

• Where guidance and feedback was a significant
  student concern, it could take various forms
• uncertainty about what staff expected from
  students in set i.e. formally required work
• dissatisfaction with the variable quantity and
  helpfulness of feedback comments from staff
• frustration with delays in receiving feedback
• (in a small number of instances) uncertainty
  about the ground-rules for buttonholing tutors

9
STUDENTS CONCERNS ABOUT GUIDANCE AND FEEDBACK
(Interviews)

• S5 I got 8 out of 20, and I've got nothing
  written on my feedback sheet at all.
• S3 Mine's the same. I got 10, and it's got no
  comments on it whatsoever.
• S5 And they tell you to do it in
  double-spacing, so they can write things in, but
  they never do .
• S3 I mean, if we're getting half marks, it
  must have a lot wrong with it . . S5 Exactly.
  But it's not telling us anything.
• --------------------------------------------------
  -------------------
• S1 Sometimes they say Be more concise but then
  another time I thought Well, Ill try being more
  concise this time and actually I got less for
  doing that! So then the next time I thought
  Ill go back to my other way and it worked
  better! So its been confusing.

10
STUDENTS CONCERNS ABOUT GUIDANCE AND FEEDBACK
(Interviews)

• S1 We write the thing, hand it in S Yeah and
  we get it back with a few comments on Mainly
  spelling mistakes. Laughter
• S3 It's postgrads who mark the work, and it's
  quite, sometimes inconsistent.
• S2 It's very inconsistent. S Yeah. And
  also, I don't think that they are marked for us.
  They are marked for them. ..
• I don't think they are writing in the margins so
  we will know not to do it again. They're writing
  it in the margins so they will remember that
  we've done it wrong when they add up the marks, I
  think. It isn't done as feedback.

11
TWO CASE STUDIES

• Case 1 - A Large First-year Course Unit
• Case 2 - A Small Final-year Honours Module

12
Case OneA LARGE FIRST-YEAR COURSE UNIT

• Over 600 students and 25 staff in varied roles
• 50 of overall grade from coursework, incl. a
  debate, a group poster, an advisory letter to a
  GP (the pertussis enigma exercise)
• findings from initial questionnaires and
  interviews
• low questionnaire scores on clarity about
  assessment and feedback
• general concern about limitations of
  pre-assignment guidance and post-assignment
  feedback
• particular concern with the pertussis enigma
  exercise

13
Case OneA LARGE FIRST-YEAR COURSE UNIT

• The collaborative initiative agreed with the
  course team to address the concerns identified
• strengthened guidance to lab demonstrators about
  assignments and assessments
• (incl. the pertussis enigma exercise)
• adoption of a structured marking and feedback
  proforma for the pertussis enigma exercise

14
Case OneFINDINGS ON IMPACT

• Pre-Collaborative Initiative
• S1 We didn't actually get much feedback on the
  actual marking of the pertussis exercise. Mine
  had no written comments on it at all and had 10
  out of 20 or something, which I wasn't too happy
  with.
  
• I So you didn't understand why you'd got that
  mark?
• S1 Yeah, well no comments were on it at all
• Collaborative Initiative
• S Yeah. ... I thought the feedback on the
  pertussis assignment was good because it had
  written comments and how you'd done in each bit.
  So it wasn't just a mark out of nowhere, you knew
  where you'd let yourself down, whether it was the
  presentation,or whether it was the content, or
  what.

15
Case OneFINDINGS ON IMPACT

• With apparently highly similar student cohorts
  
• More positive perceptions of advance guidance and
  feedback about the pertussis enigma exercise in
  every interview following the introduction of the
  initiative
• No evidence in the questionnaire data of impact
  across the module
• Suggests difficulty of change across multiple
  assignments with many staff involved

16
Case TwoA SMALL FINAL-YEAR HONOURS MODULE

• A total of 14-15 students and two staff
• Took the form of student-led seminars, assessed
  by oral presentations and essays
• Findings from initial questionnaires and
  interviews
• questionnaire scores low on two feedback items
• interviews indicated, for presentation and
  essays
• uncertainty about assessment criteria
• relative paucity of feedback

17
Case TwoA SMALL FINAL-YEAR HONOURS MODULE

• The collaborative initiative agreed with the
  course team to address the concerns identified
• more guidance about assessment criteria in
  introductory class briefing
• handout on assessment criteria for presentations
• anonymous written peer feedback on presentations
• private feedback meeting between staff and
  student-presenters

18
Case TwoFINDINGS ON IMPACT

• With apparently highly similar student cohorts
  
• improvement in questionnaire scores on all the
  teaching-learning environment scales
• largest change on scales relevant to the
  collaborative initiative
• similarly very positive comments in the student
  interviews

19
Case Two FINDINGS ON IMPACT ( agree or
agree somewhat)
clear expectations
how to tackle it
fdbk for learning
staff support
fdbk to clarify
20
Case TwoFINDINGS ON IMPACT

• Pre-Collaborative Initiative
• No, theyre really weird essay titles and Ive
  just been like, Whoah, where do you start? Like,
  theyre really bizarre.
• Collaborative Initiative
• S4 They have given us good guidance about the
  essays -
• S2 Yeah, they did didnt they?
• S3 Yeah, one of them particularly, its not
  really anything we can find references for
  So, its something weve really got to kind of
  think about, and draw on our knowledge of what we
  already know

21
Case TwoFINDINGS ON IMPACT

• Collaborative Initiative
• I So do you think having feedback from other
  students on your presentation is worthwhile?
• S1 I think it is, cause then you realise what
  you did wrong and how you can improve it. It is
  actually really useful.
• S2 Especially from people that, you know, if we
  do something blatantly stupid theyll tell us.
  Its quite good to get opinions from people
  whove been listening to you but not marking.

22
REVIEW OF CASE FINDINGS

• (Bearing in mind the need for caution about the
  scale and limitations of the research) these
  research findings would seem to indicate that
• students concerns about the effectiveness of
  guidance and feedback took various forms
• areas of particular concern could be pinpointed,
  and steps taken to try to address these
• there was follow-up evidence of impact in
  interviews (in both cases) and in questionnaires
  (in case 2)
• findings from these and other cases suggest that
  enhancing the quality of feedback and guidance
  may be harder to achieve in larger team-taught
  courses

23
A CODA MODELLING GUIDANCE AND FEEDBACK

• Unfinished business
• analysis and writing-up of research evidence as
  ongoing and recursive
• Remodelling guidance and feedback as an
  integrated loop

24
The guidance and feedback loop
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
The guidance and feedback loop
32
REMODELLING GUIDANCE AND FEEDBACK

• Remodelling guidance and feedback as an
  integrated whole
• the guidance-and-feedback loop
• takes in both coursework and exams
• illuminates potential troublespots
• shows how steps can be inter-related
• findings as data and evidence
• in tandem with
• findings as tools for diagnosis enhancement

33
KEY REFERENCES
Black, P., Harrison, C., Marshall, L. and Wiliam,
D. (2003). Assessment for Learning. Putting It
into Practice. Maidenhead Open University
Press. Carless, D. (2006). 'Differing perceptions
in the feedback process', Studies in Higher
Education, 31.2, pp. 219-233. Hounsell, D.
(2003). 'Student feedback, learning and
development'. In Slowey, M. and Watson, D. ed.
Higher Education and the Lifecourse. Maidenhead
SRHE Open University Press/McGraw-Hill. pp.
67-78. Hounsell, D. in press. 'Towards more
sustainable feedback to students.' In Boud, D.
and Falchikov, N., eds. Rethinking Assessment for
Future Learning. London Routledge Hounsell, D et
al. (2005) Enhancing Teaching-Learning
Environments in Undergraduate Courses
End-of-Award Report to ESRC on project
L139251099. Universities of Edinburgh, Durham and
Coventry ETL Project. http//www.ed.ac.uk/etl/pu
blications
34
KEY REFERENCES
Hounsell, D., McCune, V., Hounsell, J. and
Litjens, J. The quality of guidance and feedback
to students. Submitted for journal publication,
Sept 2006 Krause, K., Hartley, R., James, R.
and McInnis, C. (2005). The First Year Experience
in Australian Universities Findings from a
Decade of National Studies. Final Report to DEST.
Melbourne University of Melbourne, Centre for
the Study of Higher Education.
http//www.cshe.unimelb.edu.au/ McCune, V. and
Hounsell, D. (2005). The development of
students' ways of thinking and practising in
three final-year biology courses. Higher
Education, 49(2), 255-289. Nicol, D. and
Macfarlane-Dick, D. (2006). Formative assessment
and self-regulated learning a model and seven
principles of good feedback practice. Studies
in Higher Education, 31(2), 199-218 QAA (2003).
Learning from Subject Review, 1993-2001 Sharing
Good Practice. Gloucester Quality Assurance
Agency for Higher Education. http//www.qaaa.ac.uk
Sadler, D. R. (1998) Formative assessment
revisiting the territory, Assessment in
Education 5(1) 77-84.