Increasing success in first year courses

1
Increasing success in first year courses
Dr David Nicol, Project Director, REAP
www.reap.ac.uk Deputy Director Centre for
Academic Practice and Learning Enhancement
(CAPLE) University of Strathclyde University of
Glasgow, 26th October 2006
2
The REAP Project

• 3 HEIs (Strathclyde, Glasgow Caledonian Business
  School, Glasgow University)
• Focus is on large 1st year classes
• Pedagogy assessment for learner self-regulation
• Range of technologies online tests, simulations,
  discussion boards, e-voting, e-portfolios,
  peer/feedack software, admin systems, VLEs,
  offline-online
• Goals Learning quality and teaching efficiencies
• Outputs case studies of redesign advice to
  support strategic change in institutions
  (transformation).
• Dissemination across HE/FE sector

3
My Plan

• Briefly discuss issues relating to first year
  experience and assessment
• Review literature on formative assessment and 1st
  year experience (synthesis)
• Provide case study of first year course redesign
  (drawn from REAP project)
• Analysis of case study in relation to assessment
  principles from literature
• Discussion of relevance to your context and ideas
  from you

4
First Year The academic experience

• Why students leave academic programmes early
• Flawed decision-making in initial choices
• Events that impact on students lives outside the
  classroom
• Students experiences of the programme and the
  institution
• Failure to cope with the academic demands of
  programmes
• Yorke and Logden (2004) Retention and student
  success in higher education.

5
First Year The academic experience

• What is important?
• Coping with transition
• Understanding what is required
• Engagement with academic programmes
• Support and feedback
• Experiences of success
• Agents in own learning
• Belief in self (ability) and motivation
• Social dynamics of learning (belonging)
• Yorke and Tintos research

6
Gibbs and Simpson (2004)
Gibbs, G. Simpson, C (2004) Conditions under
which assessment supports students learning,
Learning and Teaching in Higher Education, 1,
3-31. See Formative Assessment in Science
Teaching (FAST) project at http//www.open.ac.uk/
science/fdtl/
7
Gibbs and Simpson (2004)

• Assessment tasks Conditions 1-4
• Capture enough study time (in and out of class)
• Are spread out evenly across timeline of study
• Lead to productive activity (deep vs surface)
• Communicate clear and high expectations

8
Gibbs and Simpson (2004)

• Feedback Conditions 5-11
• Is sufficient (in frequency detail)
• Is provided quickly enough to be useful
• Focuses on learning rather than on marks
• Is linked to assessment criteria/expected
  outcomes
• Makes sense to students
• Is received by students and attended to
• Is acted upon, to improve work and/or learning

9
Nicol and Macfarlane-Dick (2006)

• Literature review
• Nicol, D. Macfarlane-Dick, D. (2006).
  Formative assessment and self-regulated learning
  A model and seven principles of good feedback
  practice. Studies in Higher Education, 34 (1),
  199-218
• See
• Student Enhanced Learning through Effective
  Feedback SENLEF project funded by HE Academy
• REAP project www.reap.ac.uk

10
Research on Assessment in HE

• Teaching/learning paradigm
• Transmission

Constructivist
student-centred
Assessment paradigm Transmission
teacher-centred
11
Some key research findings

• Students are always engaged in self-assessment/sel
  f-regulation of their own learning (Winne, 2005
  Black Wiliam, 2005). Logically entailed by
  constructivist thinking.
• The act of using teacher feedback implies that
  self-assessment must be present (Sadler, 1983,
  Nicol and Macfarlane-Dick, 2006)
• Feedback in HE is being significantly reduced so
  how are students still learning?
• The question is how can we scaffold students
  learning so they become better at self-regulation
  of learning? (Lajoie, 2005)

12
Scaffolding self regulation 7 principles of good
feedback

• Clarify what good performance is (goals,
  criteria, standards).
• Facilitate reflection and self-assessment in
  learning
• Deliver high quality feedback to students
  feedback that enables students to monitor and
  self-correct
• Encourage peer and tutor dialogue around learning
• Encourage positive motivational beliefs self
  esteem through assessment
• Provide opportunities to close the feedback loop
• Use feedback information to shape teaching
• Source Nicol and Macfarlane-Dick (2006)
  Formative assessment and self-regulated learning
  A model and seven principles of good feedback
  practice, Studies in Higher Education

13
Re-engineering Assessment Practices REAP

• Psychology Case Study
• Context
• 560 first year students
• Mixture of psychology majors (130) and those
  taking psychology only for one year (430)
• 6 topic areas, 48 lectures, 4 tutorials, 12
  practicals
• Assessment 2 x MCQs (25), tutorial attendance
  (4), taking part in experiment (5), essay exam
  (66)

14
Psychology Case Study

• Problems identified with the course
• Students got no practice in writing skills but
  required in the exam
• No feedback except on MCQs (percent correct)
• Didnt want to increase staff workload
• Wanted to improve overall exam marks
• And standard of entrant to second year

15
Psychology re-design
Stage 1 Question 1 moderate difficulty (50
words) Individual response post it discuss
answer in groups (of 6) Timed release model
answer to self-evaluate their response Stage 2
Question 2 difficult (100 words) Group response
discuss (online) agree post response Model
answer released for stage 2 Stage 3 Question 3
complex (300 word essay) Group response
discuss (online) agree post Model answer
released for stage 3
16
Psychology

• Task 1 Define and describe structural encoding,
  phonological encoding and semantic encoding.
  Provide an example of each construct. (50 words,
  individual)
• Task 2 Describe the serial position effect and
  its two separate components. Discuss the
  specific structural components of memory that are
  responsible. (100 word, group response)
• Task 3 Summarise the stage theory of memory.
  To what extent does it provide an adequate theory
  of memory? (300 word, group response)

17
Psychology re-design key features

• Writing tasks related to current lecture delivery
  (on human memory).
• 78 volunteers induction task
• Question of increasing difficulty and length of
  answer (50 words, 100 words and 300 words)
• Moving from individual to group responses.
• Scaffolding of learning through peer discussion
  (and cognitive growth thro conflict resolution)
• Comparing work against model answers
• Use of assignment/ discussion tools in (WebCT)

18
Planned developments

• Replace half the lectures with discussion tasks
• Tasks will become progressively more demanding
  within and across topics as the year progresses.
• Participation will be compulsory and monitored
• Individual responses will precede online
  discussion and response for all tasks
• Once a final group response has been agreed each
  student will submit a copy to VLE (WebCT)
• Purpose is to deal with free rider-effects
• Source Jim Baxter, Psychology, Strathclyde
  University

19
The seven feedback principles

• Standard format and model answers provide
  progressive clarification of expectations (clear
  goals, principle 1)
• Students encouraged to self-assess against model
  answer (self-assessment, principle 2)
• Online peer discussion aimed at reaching
  consensus about response (dialogue, principle 4)
• Staged complexity and focus on learning rather
  marks (motivation, principle 5)
• Repeated cycle of topics and tasks (closing gap,
  principle 6)
• Tutors can monitor progress and adapt (shaping
  teaching, principle 7)

20
Gibbs Simpsons four assessment conditions

• Assessment tasks require significant study out of
  class (capture sufficient study time, condition
  1)
• They are distributed across topics and weeks (are
  spread out evenly, condition 2)
• They move students progressively to deeper levels
  of understanding (productive/ deep learning,
  condition 3)
• There are explicit goals and progressive increase
  in challenge (communicates clear and high
  expectations, condition 4)

21
What can we learn from this case study?

• Use of a simple technology (discussion board)
• Considerable thought gone into the pedagogical
  design which is transferable.
• Supported by robust assessment/learning
  principles
• Coordinated approach within the department
• Evaluation showed a lot of supportive
  socialisation during induction task
• Intention to compare year on year examination
  performance.

22
Assessment and the the first year

• Two super principles that need to be balanced
  within the first year and across years
• Engagement Organised and structured academic
  activities (so students know they are making
  progress).
• Empowerment Opportunities for students to
  internalise academic standards and supervise
  themselves as they study in relation to these
  standards (self and peer assessment processes)

23
First Year The academic experience

• What is important?
• Coping with transition
• Understanding what is required
• Engagement with academic programmes
• Support and feedback
• Experiences of success
• Sense of agency in own learning
• Belief in self (ability) and motivation
• Social dynamics of learning (belonging)
• Yorke and Tintos research

24
Questions and discussion
25
Case Study 2

• Typical large class taught through lectures,
  tutorials
• Exam series of multiple choice questions to
  test breadth of understanding
• Sometimes essay or short answer questions in exam.

26
Redesign

• Students introduced to MCQs how they work
• After lectures but before tutorial/problem
  solving class students in pairs prepare MCQ
  including feedback for right and wrong answers.
• In tutorial pairs swap MCQs and get feedback
  revise - post in VLE.
• Refinement students identify which level of
  Blooms taxonomy they questions are testing
• Final exam teacher selects some students
  questions but has them provide reasons for
  answers.
• Producing questions compulsory

27
Benefits of Redesign

• Students develop questioning skills
• Creating feedback develops writing skills and
  critical thinking giving reasons for correct
  and wrong answers - deep learning
• Sharing in class encourages peer feedback
• Identifying Blooms level leads to further
  reflection
• Use in final exam encourages class to share work

28
Relation to feedback principles

• Students create MCQs (actively create criteria,
  principle 1)
• Evaluating content against criteria Bloom
  categories (self-assessment/reflection, principle
  2)
• Tutor monitoring and general feedback (principle
  3)
• Peer feedback during creation and in tutorials
  (dialogue, principle 4)
• MCQs used in exam and the creation process is
  engaging (motivation, principle 5)
• Cyclical development of MCQs (closing gap,
  principle 6)
• Teaching could be shaped by results (principle 7)

29
Gibbs Simpsons four assessment conditions

• Writing MCQs as preparation for tutorials
  (capture sufficient study time, condition 1)
• Task could be a regular requirement and built
  into tutorials (are spread out evenly, condition
  2)
• MCQs could move to deeper levels matching Blooms
  taxonomy (productive/ deep learning, condition 3)
• The goals are clear and there is progressive
  increase in challenge (communicates clear and
  high expectations, condition 4)

30
Task

• Do you have a learning design that you could
  share with the group or a specific method?
• Using the seven principles/questions and/or four
  conditions can you identify ways of improving
  some aspect of your own class teaching?

31
Other relevant papers
Nicol, D (1996), Increasing success in first year
courses assessment redesign, self-regulation and
learning technologies, Paper being presented at
ASCILITE Conference, Sydney, Dec 3-6,
2006. Nicol, D (in press), Laying the foundation
for lifelong learning cases studies of
technology supported assessment processes in
large first year classes, British Journal of
Educational Technology (to be published
2007). Nicol, D. Milligan, C. (2006),
Rethinking technology-supported assessment in
relation to the seven principles of good feedback
practice. In C. Bryan and K. Clegg, Innovations
in Assessment, Routledge.