Re-engineering Assessment Practices in Scottish Higher Education [REAP]

1
Re-engineering Assessment Practices in Scottish
Higher Education REAP
Dr David Nicol, Project Director Centre for
Academic Practice and Learning Enhancement Univers
ity of Strathclyde (CAPLE) www.reap.ac.uk Univer
sity of Strathclyde, 25th April 2006
2
Scottish Funding Council Invitation

• 6m for HE/FE competitive bidding
• E-learning transformational projects
• Embedding and substitution
• New approaches to teaching/ learning
• Measurable benefits to institution/sector
• Partnerships with other HEIs/FEIs

3
Rationale for the REAP proposal

• Assessment key driver of student learning
• Assessment is a major cost in HE economies of
  scale limited
• Assessment influences a wide range of
  organisational, pedagogical and business
  processes in HE

4
Aims of REAP Project

• Re-engineer assessment practices in 3 HE
  institutions using supportive technologies.
• Enhanced approaches - self, peer, tutor, to
  enhance motivation learner self-regulation
• Integrate technologies online tests,
  simulations, e-voting, e-portfolios, VLEs, admin
  systems and online-offline interactions
• Focus is on large 1st year classes
• Improve learning quality show efficiencies
• Embed new processes in institution

5
Partners and roles

• Strathclyde University 5 departments one per
  faculty with large 1st year classes (Mechanical
  Engineering, Marketing, Pharmacy, Psychology,
  Primary Education)
• Glasgow Caledonian Business School (one faculty
  with range of implementations)
• Glasgow University (developing electronic voting
  technologies)
• Allows synergies and comparisons across
  disciplines /faculty/institution.

6
Research on Assessment in HE (1)

• Teaching/learning paradigm
• Transmission

Constructivist
student-centred
Assessment paradigm Transmission
teacher-centred
7
Some key research findings

• Students are always engaged in self-assessment/sel
  f-regulation of their own learning (Winne, 2005
  Black Wiliam, 2005). Logically entailed by
  constructivist ideas.
• The act of using teacher feedback implies that
  self-assessment must be present (Sadler, 1983,
  Nicol and Macfarlane-Dick, 2006)
• Feedback in HE is being significantly reduced so
  how are students still learning?
• The question is how we can scaffold students
  learning so they become better at self-regulation
  of learning (Lajoie, 2005)

8
References
Nicol, D. Macfarlane-Dick, D. (2006). Formative
assessment and self-regulated learning A model
and seven principles of good feedback practice.
Studies in Higher Education, 34 (1),
199-218 Nicol, D. Milligan, C. (2006),
Rethinking technology-supported assessment in
relation to the seven principles of good feedback
practice. In C. Bryan and K. Clegg, Innovations
in Assessment, Routledge.
9
Scaffolding self regulation the seven principles
of good feedback practice

• Clarify what good performance is (goals,
  criteria, standards).
• Facilitate reflection and self-assessment in
  learning
• Deliver high quality feedback to students
• Encourage peer and tutor dialogue around learning
• Encourage positive motivational beliefs self
  esteem through assessment
• Provide opportunities to close the feedback loop
• Use feedback information to shape teaching
• Source Nicol and Macfarlane-Dick (2006)
  Formative assessment and self-regulated learning
  A model and seven principles of good feedback
  practice, Studies in Higher Education

10
Black and Wiliam (2005)
As well as assessing and marking (through
discussion and clear guidance) their own work
they also assess and mark the work of others.
They do this in a very mature and sensible way..
They take pride in clear and well presented work
that one of their peers may be asked to mark.
Any disagreement about the answer is thoroughly
and openly discussed until agreement is reached.
Alice, Teacher at Watford School, UK
11
Questions to ask about assessment

• Do students actively engage with assessment
  criteria and standards?
• Are there formal/informal opportunities for self
  and peer assessment processes?
• What kind of feedback is provided does it help
  students to self-assess, self-correct?
• Does the feedback focus on learning goals rather
  than on marks?
• Is feedback acted upon?
• How is feedback used to shape teaching?

12
(No Transcript)
13
Educational basis of REAP

• Enhanced formative assessment resulted in
  learning gains that were among the largest ever
  reported (Black Wiliam, 1998)
• Assessment acts should establish the basis for
  students to undertake their own assessment
  activities in the future (Boud, 2000).
• Students are already self-regulating their own
  performance teachers must build on this capacity
  (Yorke, 2003 Nicol Macfarlane-Dick, 2004/05)
• Gibbs Simpson (2004). FAST project

14
Definition (1)

• What do we mean by assessment?
• Assessment of learning (summative)
  certification and progression.
• Assessment for learning (formative) to
  support learning (developmental)
• Assessment involves identifying appropriate
  standards and criteria and making judgements
  about quality. (Boud, 2000)

15
Definitions (2)

• Who is involved in formative assessment and
  feedback?
• Tutor
• Peer
• External (e.g. placement supervisor)
• Computer generated
• Self

16
Sadler (1989)
For feedback to benefit learning students must
know 1. What good performance is (goals,
criteria) 2. How current performance relates to
good performance (compare) 3. How to act to close
the gap No 2 means that students must already
possess some of the same evaluative skills as the
teacher (Royce Sadler, 1989).
17
The context of use of online tests (MCQs)

1. Students repeating MCQs (closing loop)
2. Case study with MCQs as open book
   (self-correcting)
3. Unique MCQs per student and sharing (peer
   dialogue)
4. Confidence ratings with MCQs (reflection/
   self-assessment)
5. Students construct MCQs (creating criteria)
6. MCQs before lecture (just in time teaching)
   testing shaping teaching

18
Questions and discussion