Assessment for learning: the benefits

1

• Assessment for learning the benefits
• of generating feedback
• David Nicol
• Professor of Higher Education
• Centre for Academic Practice and Learning
  Enhancement (CAPLE
• Director, Peer project (www.reap.ac.uk)
• University of Strathclyde, Scotland
• University of Cadiz, March 9-11, 2011

2
Plan

• Background
• Re-engineering Assessment Practices (REAP)
  project www.reap.ac.uk/reap/index.html
• Concepts and example of practice
• Institutional considerations
• PEER project www.reap.ac.uk/peer.aspx
• Discussion

3
Background

• Departments and faculties educational
  improvement projects, including REAP project
• Policy/strategy development of educational
  policy and principles of assessment feedback
  (based on REAP)
• Students Feedback as dialogue campaign
• Quality procedures redesigning course
  documentation and review procedures
• PEER project developing students ability to
  evaluate the quality of each others work
• See www.reap.ac.uk

4
REAP Re-engineering Assessment Practices

• Scottish Funding Council for Universities (1m)
• 3 Universities - Strathclyde, Glasgow Glasgow
  Caledonian
• Large 1st year classes (160-600 students)
• A range of disciplines (19 modules 6000
  students)
• Many technologies online tests, simulations,
  discussion boards, e-portfolios, e-voting,
  peer/feedback software, VLE, online-offline
• Learning quality and teaching efficiencies
• Assessment for learner self-regulation
• www.reap.ac.uk

5
Background (1)

• Gibbs, G. Simpson, C (2004) Conditions under
  which assessment supports students learning,
  Learning and Teaching in Higher Education, 1,
  3-31.
• See
• Formative Assessment in Science Teaching (FAST)
  project at http//www.open.ac.uk/science/fdtl

6
Gibbs and Simpson (2004)

• Assessment tasks Conditions 1-4
• Capture sufficient study time (in and out of
  class)
• Are spread out evenly across timeline of study
• Lead to productive activity (deep vs surface)
• Communicate clear and high expectations
• i.e concern here is with time on task how much
  work students do - their active engagement in
  study

7
Background (2)

• Literature Review
• Nicol, D. Macfarlane-Dick, D. (2006). Formative
  assessment and self-regulated learning A model
  and seven principles of good feedback practice.
  Studies in Higher Education, 34 (1), 199-218
• Background
• Student Enhanced Learning through Effective
  Feedback SENLEF project funded by HE Academy
• REAP project www.reap.ac.uk

8
Rethinking assessment and feedback

• 1. Consider self and peers as much as the teacher
  as sources of assessment and feedback
• Tap into different qualities than teacher can
  provide
• Saves time
• Provides considerable learning benefits (lifelong
  learning)

• 2. Focus on every step of the cycle
• Understanding the task criteria
• Applying what was learned in action

• 3. Not just written feedback
• Also verbal, computer, vicarious, formal and
  informal

9
Seven principles of good feedback

• Good feedback
• Clarifies what good performance is (goals,
  criteria, standards).
• Facilitates the development of reflection and
  self-assessment in learning
• Delivers high quality information to students
  that enables them to self-correct
• Encourages student-teacher and peer dialogue
  around learning
• Encourages positive motivational beliefs self
  esteem
• Provides opportunities to act on feedback
• Provides information to teachers that can be used
  to help shape their teaching (making learning
  visible)
• Source Nicol and Macfarlane-Dick (2006)

10
Principle 1 Clarify what good performance is
(the context of dialogue)
EMPOWERMENT/ SELF-REGULATION

• Students create criteria
• Students add own criteria
• Students identify criteria from samples of work
• Exemplars of different performance levels
  provided
• Students rephrase criteria in own words
• Provide document with criteria

ENGAGEMENT
11
Principle 2 Facilitate reflection and
self-assessment in learning
EMPOWERMENT/ SELF-REGULATION

• Students create MCQs including feedback for right
  and wrong (distractors) answers
• Students self-assess using MCQs and confidence
  ratings
• Students self-assess own performance using online
  MCQs

ENGAGEMENT
12
Two meta principles

• 1. Meta-PRINCIPLE 1 time and effort on task
  (structured engagement) i.e. steers on how much
  work to do and when Gibbs and Simpson 4
  conditions
• 2. Meta-PRINCIPLE 2 developing learner
  self-regulation (empowerment/self-regulation) i.e
  steers to encourage ownership of learning the
  seven principles discussed above.
• Key task for teacher is to balance 1 and 2

13

• Example
• Psychology

14
Psychology

• 560 first year students
• 6 topic areas (e.g. personality, classical
  conditioning), 48 lectures, 4 tutorials, 12
  practicals
• Assessment 2 x MCQs (25), tutorial attendance
  (4), taking part in experiment (5), essay exam
  (66)

15
Problems identified

• No practice in writing skills but required in the
  exam
• More detail provided in lectures than mentioned
  in exams (not enough independent reading)
• No feedback except on Multiple Choice Questions
  (percent correct)
• Didnt want to increase staff workload

16
Psychology Redesign

• Discussion board in Learning Management System
• Students in 85 discussion groups of 7-8, same
  groups throughout year
• Also open discussion board for class
• Friday lectures cancelled discover material
  themselves
• Series of online tasks

17
Structure of group tasks

• 6 cycles of 3 weeks (one cycle x major course
  topic)
• First week light written task (e.g. define
  terms) 7 short answers (all answer)
• Second week guided reading
• Week three deep written task students
  collaborate in writing a 700-800 word essay on
  the same topic.
• Within each week
• The Monday lecture introducing material
• Immediately after lecture, task posted online
  for delivery the following Monday
• Model answers (selected from students) posted for
  previous weeks task

18
The teaching role

• Participation in the discussions was compulsory
  but not marked (in subsequent years there was 2
  mark for participation)
• Course leader provided general feedback to the
  whole class often motivational
• He encouraged students to give each other
  feedback
• The group discussions were not moderated but
  monitored for participation

19
An example of deep task

• The Task 800 word essay
• Assess the strengths and weaknesses of Freuds
  and Eysencks theories of personality. Are the
  theories incompatible?
• readings suggested
• questions provided which all students should try

20
Relation to the Gibbs Simpsons four
assessment conditions

1. Tasks require significant study out of class
   (condition 1)
2. Tasks are distributed across topics and weeks
   (condition 2)
3. They move students progressively to deeper levels
   of understanding (condition 3)
4. There are explicit goals and progressive increase
   in challenge (condition 4)

21
Relation to 7 feedback principles

• Standard format and model answers provide
  progressive clarification of expectations
  (principle 1)
• Students encouraged to self-assess against model
  answer (principle 2)
• Course leader provides motivational and
  meta-level feedback and selects model answers
  (principle 3)
• Online peer discussion aimed at reaching
  consensus is core feature of design about
  response (principle 4)
• Focus on learning not just marks, sense of
  control/challenge enhanced motivation
  (principle5)
• Repeated cycle of topics and tasks provide
  opportunities to act on feedback (principle 6)
• VLE captures all interactions allowing course
  leader to monitor progress and adapt teaching
  (principle 7)

22
Benefits

• Students worked exceptionally hard
• Written responses of exceedingly high standard
• Students took responsibility for learning
• High levels of motivation atmosphere in class
  improved
• Online interactions showed powerful scaffolding
  and community building
• Feedback with 560 students through peer and
  self-feedback (model answers)
• Easy for tutors to monitor participation
• Improved mean exam performance (up from 51-59,
  plt0.01) weaker students benefit most

23
Has it worked?
24
(No Transcript)
25
(No Transcript)
26
Guidelines for Implementation

1. A single principle or many?
2. Tight-loose maintain fidelity to the principles
   (tight) but encourage disciplines to develop
   their own techniques of implementation (loose)
3. Balance teacher feedback with peer and
   self-generated feedback
4. Focus on developing students own ability for
   critical evaluation
5. Create opportunities for learning communities
   to emerge
6. The more actively engaged students are, the
   better the course design

27
Developments since REAP

1. Principles of Assessment and Feedback approved by
   University Senate and embedded in policy (2008)
2. Use of principles to inform curriculum renewal
   and Quality Assurance processes
3. Feedback as Dialogue campaign to gain
   commitment of students
4. PEER Project (Peer Evaluation in Education
   Review)
5. Work on the way we document our courses and
   programmes

28
(No Transcript)
29
(No Transcript)
30
Peer Review in Education Evaluation PEER

• The aims of the PEER project are to
• Review evidence base for peer review
• Develop educational designs for peer review (and
  self-review)
• Identify software support for peer review
• Pilot implementations of peer review with large
  student numbers
• Produce guidelines for higher education why do
  it, how to do it, pitfalls and solutions and
  software possibilities.
• see http//www.reap.ac.uk/peer.aspx

31
PEER Project Core Ideas

• All graduate development in higher education
  requires that
• Students learn to evaluate critically the quality
  and impact of their own work (e.g. academic
  texts, problem solutions, designs)
• Students learn to evaluate critically the work of
  others (e.g. peer review and peer feedback)
• Ref Nicol, D (2010) The foundation for graduate
  attributes developing self-regulation through
  self and peer assessment. www.reap.ac.uk/peer.aspx
  

32
The research on peer feedback

• Mainly about peers marking each others work
• Where peer feedback is the focus it is usually
  about peers augmenting teacher feedback by
  increasing the quantity and variety of feedback

33
The focus of PEER project

• Scenarios where students make evaluative
  judgements about the work of peers and provide a
  feedback commentary, usually written
• Not talking about
• .....collaborative tasks where students give
  each other informal feedback
• .....scenarios where the focus is on students
  evaluating each others performance in group
  working
• .....scenarios where the focus is on students
  grading/marking each others work although some
  rating of performance might be part of the peer
  design

34
Benefits of feedback construction (1)

• Constructivist rather than transmission
  perspective
• Producing is cognitively more demanding than
  reading feedback from others students cannot be
  passive
• Students actively exercise assessment criteria
  (process/reprocess) from many different
  perspectives
• See how others tackle assignments and learn that
  quality can be produced in different ways
• Develop writing skills through commentaries
• Deepens critical thinking in the discipline

35
Benefits of feedback construction (2)

• Learn to assess own work, to monitor and evaluate
  their own productions - same skills involved
• Develops self-regulation for life beyond
  university.
• Changes the power relationship in class (shares
  responsibility across students).
• Can develop learning communities

36
Giving and receiving feedback

• Giving and receiving feedback within same domain
  enriches the whole process.
• Students are able to compare the feedback they
  receive with the feedback they produce
• And learn how different reviewers perceive their
  work a single source of feedback is never
  satisfactory
• Can develop a sense of shared responsibility
  across students for learning

37
Example peer feedback

• Students write essay on one topic from three
• Each student provides feedback on three essays in
  another topic anonymously using rubric
• The rubric write a short summary of the essay,
  comment on and rate (four point scale) the
  structure, arguments, evidence, writing, suggest
  ways of improving the essay.
• Students receive peer reviews of their own essays
• Then review, comment on and rate their own essay
  using same rubric.
• Graded for participating in the task, for their
  own essay and for their review of it.
• Finally students rate 3 reviews (on others work)
  and comment on how useful they think they would
  be to author.
• Software used to support administration

38
Six learning opportunities in peer review

• Producing the target assignment
• Reviewing and producing peer feedback
• Receiving feedback from peers
• Responding to peer reviews (on own reviews of
  those given to others)
• Self-reviewing ones own work (e.g. after peer
  review)
• Receiving feedback from the teacher e.g. on the
  assignment, peer review or self-review
• 1-3 comprise the essentials of peer review while
  4-6 are further options.

39
Principles of effective peer review

• Peer review should
• Engage students in active use of criteria and
  standards
• Involve students in constructing commentaries in
  relation to peer judgements, not just marks
• Lead to practice in both analytic (componential)
  and holistic (configurational) judgements about
  quality
• Facilitate dialogue around the object and quality
  of the review
• Ensure an atmosphere of mutual trust and respect
• Integrate self-review activities within peer
  review designs
• Ensure that making peer judgements is a regular
  activity and not a one-off event
• www.reap.ac.uk/PEER.aspx

40
Some of my Publications

• Nicol, D (2010) The foundation for graduate
  attributes developing self-regulation thorugh
  self and peer assessment, QAA Enhancement Themes,
  Scotland
• Nicol, D (2010) From monologue to dialogue
  Improving written feedback in mass higher
  education. Assessment and Evaluation in Higher
  Education. 35(5), 501 -517
• Nicol, D and Draper, S (2010) A blueprint for
  transformational organisational change in HE
  REAP as a case study, Published by the Higher
  Education Academy, UK (see reap.ac.uk website)
• Nicol, D (2009) Transforming assessment and
  feedback Enhancing integration and empowerment
  in the first year, Published by Quality Assurance
  Agency, Scotland (http//www.enhancementthemes.ac.
  uk/documents/firstyear/FirstYear_TransformingAsses
  s.pdf
• Nicol, D (2009), Assessment for learner
  self-regulation Enhancing achievement in the
  first year using learning technologies,
  Assessment and Evaluation in Higher Education,
  34(3), 335-352
• Nicol, D (2007), Laying the foundation for
  lifelong learning cases studies of technology
  supported assessment processes in large first
  year classes, British Journal of Educational
  Technology, 38(4), 668-678
• Nicol, D (2007) E-assessment by design using
  multiple-choice tests to good effect, Journal of
  Further and Higher Education.31(1), 53-64.
• Nicol, D. Milligan, C. (2006), Rethinking
  technology-supported assessment in relation to
  the seven principles of good feedback practice.
  In C. Bryan and K. Clegg, Innovations in
  Assessment, Routledge.
• Nicol, D, J. Macfarlane-Dick (2006), Formative
  assessment and self-regulated learning A model
  and seven principles of good feedback practice,
  Studies in Higher Education, 31(2), 199-218.
• See www.reap.ac.uk/resources.aspx for copies.