Higher Education Academy Network for Hospitality, Leisure, Sport and Tourism CONFERENCE ON ASSESSMEN

1
Higher Education Academy Network for Hospitality,
Leisure, Sport and TourismCONFERENCE ON
ASSESSMENTOxford, 15 September 2005

• The Role of Assessment and Feedback
• to Students in Bringing About
• High-Quality Learning
• Dai Hounsell
• University of Edinburgh
• www.ed.ac.uk/etl

2
WHATS THE USE OF FEEDBACK ?

• to inform and justify an evaluation of the
  progress, performance or achievement of students
• to encourage and support students
• to raise the quality of students learning, by
• helping them to identify strengths and
  limitations
• developing their grasp of what counts as
  high-quality work in a subject ...
• .... and of how they might go about achieving
  work of this standard

3
GUIDANCE, FEEDBACK AND FORMATIVE ASSESSMENT

• Striking an optimal balance between
• FORMATIVE ASSESSMENT
• (assessment-for-learning)
• and
• SUMMATIVE ASSESSMENT
• (assessment-for-grading and certification)
• Guidance pre-and post-assignment and feedback
  as key elements in formative assessment
• Evidence of the benefits of well-designed
  formative assessment

4
GUIDANCE, FEEDBACK AND FORMATIVE ASSESSMENT

• Evidence of the benefits of well-designed
  formative assessment
• The research reported here shows conclusively
  that formative assessment does improve learning.
  The gains in achievement appear to be quite
  considerable, ... among the largest ever
  reported for educational interventions.
• (Black and Wiliam, 1998 61)
• Pressures on the quantity and quality of guidance
  and feedback in an age of mass higher education
• e.g. larger classes, more diverse students,
  constrained resources, end-loaded assessment

5
AIMS OF PRESENTATION

• Drawing on the work of the Enhancing
  Undergraduate Teaching-Learning Environments
  (ETL) project
• to outline initial research findings from one
  subject area on guidance and feedback to students
• to report what steps were taken by course teams
  to tackle concerns raised in these findings
• to review evidence on the impact of these
  initiatives
• ? to explore some practical implications for
  guidance and feedback to students

6
THE ETL PROJECTEnhancing Teaching-Learning
Environments in Undergraduate Courses

• Funding (2001-2005)
• Economic and Social Research Council / Teaching
  and Learning Research Programme
• Aims
• to investigate ways of enhancing the quality of
  undergraduate learning and teaching, in a range
  of subject areas and institutional settings
• Subject Areas
• Biosciences, Economics, Electronic Engineering,
  History

7
RESEARCH DESIGN

• Samples and settings
• first- and final-year course units in three
  partner departments in each subject area
• Data-gathering
• questionnaires
• interviews with students and staff
• course documentation and data
• Enhancement focus
• collection, analysis joint review of baseline
  data
• evidence-based collaborative initiatives

8
BIOSCIENCE STUDENTS CONCERNS ABOUT GUIDANCE AND
FEEDBACK

• Students overall perceptions of their courses
  were broadly positive across all of the
  bioscience modules/course units surveyed
• Experiences of the provision of guidance and
  feedback on coursework, however, were much more
  variable
• In some units, students reported favourably in
  others, there were significant student concerns

9
BIOSCIENCE STUDENTS CONCERNS ABOUT GUIDANCE AND
FEEDBACK

• Where guidance and feedback was a significant
  student concern, it could take various forms
• uncertainty about what staff expected from
  students in set i.e. formally required work
• dissatisfaction with the variable quantity and
  helpfulness of feedback comments from staff
• frustration with delays in receiving feedback
• (in a small number of instances) uncertainty
  about the ground-rules for buttonholing tutors

10
BIOSCIENCE STUDENTS CONCERNS ABOUT GUIDANCE AND
FEEDBACK

• S2 We dont really get feedback on exams - you
  get a mark - but even in course work you just
  get a mark and maybe a couple of ticks or (S1
  Good) or put this in capital letters instead or
  something, and that would be it.
• S1 You dont really get any feedback on
  anything.
• I So that must be quite difficult then if
  youre struggling to figure out what to do?
• S1 (Agrees). Thats another way its very
  different from school. But how did you know what
  areas to improve?
• S2 I didnt really, I just threw everything in
  and hoped something would be okay.
• S1 If it did work, you dont know what it was
  that worked and what it is youre wasting your
  time on.

11
BIOSCIENCE STUDENTS CONCERNS ABOUT GUIDANCE AND
FEEDBACK

• S5 Assignments should be explained better, and
  there should be more feedback. (Others yes,
  definitely feedback ).
• S5 - 'Cos that's the only way you can really
  improve yourself, if you know what to improve
  yourself on, and which parts to improve.
• -----------------------
• S5 And I got 8 out of 20, and I've got nothing
  written on my feedback sheet at all.
• S3 Mine's the same. I got 10, and it's got no
  comments on it whatsoever.
• S5 And they tell you to do it in
  double-spacing, so they can write things in, but
  they never do .
• S3 I mean, if we're getting half marks, it
  must have a lot wrong with it . . S5 Exactly.
  But it's not telling us anything.

12
BIOSCIENCE STUDENTS CONCERNS ABOUT GUIDANCE AND
FEEDBACK

• S1 We write the thing, hand it in S Yeah and
  we get it back with a few comments on Mainly
  spelling mistakes. Laughter
• S3 It's postgrads who mark the work, and it's
  quite, sometimes inconsistent.
• S2 It's very inconsistent. S Yeah. And
  also, I don't think that they are marked for us.
  They are marked for them. .. I don't think they
  are writing in the margins so we will know not to
  do it again. They're writing it in the margins
  so they will remember that we've done it wrong
  when they add up the marks, I think. It isn't
  done as feedback.

13
BIOSCIENCE STUDENTS CONCERNS ABOUT GUIDANCE AND
FEEDBACK

• S1 Sometimes they say Be more concise but then
  another time I thought Well, Ill try being more
  concise this time and actually I got less for
  doing that! So then the next time I thought
  Ill go back to my other way and it worked
  better! So its been confusing.
• -----
• S4 I think every time they are corrected by
  different people anyway. Some of the correctors
  have said You should do it like this when the
  person before had said to do it the other way, so
  then we get marks taken off because we try to
  make an effort.

14
Case OneA LARGE FIRST-YEAR COURSE UNIT

• Over 600 students and over 25 staff in varied
  roles
• 50 of overall grade from coursework, incl. a
  debate, a group poster, an advisory letter to a
  GP (pertussis enigma exercise)
• findings from initial questionnaires and
  interviews
• low questionnaire scores on clarity about
  assessment and feedback
• general concern about limitations of
  pre-assignment guidance and post-assignment
  feedback
• particular concern with the pertussis enigma
  exercise

15
Case OneA LARGE FIRST-YEAR COURSE UNIT

• The collaborative initiative agreed with the
  course team to address the concerns identified
• strengthened guidance to lab demonstrators about
  assignments and assessments (incl. the pertussis
  enigma exercise)
• adoption of a structured marking and feedback
  proforma for the pertussis enigma exercise

16
Case OneFINDINGS ON IMPACT

• Pre-Collaborative Initiative
• S1 We didn't actually get much feedback on the
  actual marking of the pertussis exercise. Mine
  had no written comments on it at all and had 10
  out of 20 or something, which I wasn't too happy
  with.
  
• I So you didn't understand why you'd got that
  mark?
• S1 Yeah, well no comments were on it at all
• Collaborative Initiative
• S Yeah. ... I thought the feedback on the
  pertussis assignment was good because it had
  written comments and how you'd done in each bit.
  So it wasn't just a mark out of nowhere, you knew
  where you'd let yourself down, whether it was the
  presentation,or whether it was the content, or
  what.

17
Case OneFINDINGS ON IMPACT

• With apparently highly similar student cohorts
  
• More positive perceptions of advance guidance and
  feedback about the pertussis enigma exercise in
  every interview following the introduction of the
  initiative
• No evidence in the questionnaire data of impact
  across the module
• Suggests difficulty of change across multiple
  assignments with many staff involved

18
Case TwoA SMALL FINAL-YEAR HONOURS MODULE

• total of 14-15 students and two staff
• took the form of student-led seminars, assessed
  by oral presentations and essays
• findings from initial questionnaires and
  interviews
• questionnaire scores low on two feedback items
• interviews indicated, for presentation and
  essays
• uncertainty about assessment criteria
• relative paucity of feedback

19
Case TwoA SMALL FINAL-YEAR HONOURS MODULE

• The collaborative initiative agreed with the
  course team to address the concerns identified
• more guidance about assessment criteria in
  introductory class briefing
• handout on assessment criteria for presentations
• anonymous written peer feedback on presentations
• private feedback meeting between staff and
  student-presenters

20
Case TwoFINDINGS ON IMPACT

• With apparently highly similar student cohorts
  
• improvement in questionnaire scores on all the
  teaching-learning environment scales
• largest change on scales relevant to the
  collaborative initiative
• similarly very positive comments in the student
  interviews

21
Case Two FINDINGS ON IMPACT ( agree or agree
somewhat)
clear expectations
how to tackle it
fdbk for learning
staff support
fdbk to clarify
22
Case TwoFINDINGS ON IMPACT

• Pre-Collaborative Initiative
• No, theyre really weird essay titles and Ive
  just been like, Whoah, where do you start? Like,
  theyre really bizarre.
• Collaborative Initiative
• S4 They have given us good guidance about the
  essays -
• S2 Yeah, they did didnt they?
• S3 Yeah, one of them particularly, its not
  really anything we can find references for
  So, its something weve really got to kind of
  think about, and draw on our knowledge of what we
  already know

23
Case TwoFINDINGS ON IMPACT

• Collaborative Initiative
• I So do you think having feedback from other
  students on your presentation is worthwhile?
• S1 I think it is, cause then you realise what
  you did wrong and how you can improve it. It is
  actually really useful.
• S2 Especially from people that, you know, if we
  do something blatantly stupid theyll tell us.
  Its quite good to get opinions from people
  whove been listening to you but not marking.

24
CONCLUDING COMMENTS/IMPLICATIONS

• (Bearing in mind the need for caution about the
  scale and limitations of the research) these
  research findings would seem to indicate that
• the students concerns about the effectiveness of
  guidance and feedback took various forms
• it proved feasible to pinpoint areas of
  particular concern and to take steps to try to
  address these
• there was follow-up evidence of impact in
  interviews (in both cases) and in questionnaires
  (in case 2)
• findings from these and other cases suggests that
  enhancing the quality of feedback and guidance
  may be harder to achieve in larger team-taught
  courses

25
some questions for discussion
26
QUESTIONS FOR DISCUSSION

• Thinking of a course on which you teach, which
  aspects of the guidance and feedback provided
  would the students find most/least helpful to
  them as learners, do you feel?
• Would it be helpful to know more about the
  students experiences and perceptions of guidance
  and feedback . . . and if so, how might this be
  done?
• (If there were indications of student concern) In
  what ways could guidance and feedback on the
  course feasibly be strengthened, do you think?
• e.g. by reviewing
• where and when guidance and feedback are provided
• by whom guidance or feedback are provided
• in what forms guidance or feedback are provided