TMA feedback: can we do better Mirabelle Walker

1
TMA feedback can we do better?Mirabelle Walker

• Centre for Open Learning of Mathematics, Science
  Computing and Technology (COLMSCT)

2
FAST project findings (1)

• Quantity and timing of feedback
• Sufficient feedback is provided, often enough and
  in enough detail
• The feedback is provided quickly enough to be
  useful to students
• Quality of feedback
• Feedback focuses on students performance and
  learning, and on actions under their control
• Feedback is appropriate to the purpose of the
  assignment and to its criteria for success
• Feedback is appropriate in relation to students
  understanding of what they are supposed to be
  doing
• Student response to feedback
• Feedback is received and attended to
• Feedback is acted upon by the student
• Gibbs Simpson (20045)

3
FAST project findings (2)

• What students said about their TMA feedback
• Received plenty of it
• Motivated by praise and encouragement
• Mainly received within three weeks
• Read feedback but rarely acted on it

4
FAST project findings (3)
5
FAST project findings (4)
6
Idea of depth of comment

• Depth 1 acknowledges e.g. more needed here
  good
• Depth 2 corrects / amplifies e.g. you needed
  to mention xxxx a good introduction
• Depth 3 explains e.g. you needed to mention
  xxxx because a good introduction because

7
FAST project findings (5)
8
FAST project findings (6)

• Significant weaknesses in current practice
• Too much emphasis on justifying the grade
• Lack of shared understanding of assessment
  criteria (students CT)
• ALs good at articulating students weaknesses
  explaining strengths problematic
• Lack of holistic assessment of students work

9
My project

• Replicate some of FAST investigations in
  Technology results similar?
• Courses chosen T173, T209 T224
• Analysis of feedback on sample TMAs
• Telephone interviews with students
• Follow-up action with ALs, monitors, CTs, STs as
  appropriate and review

10
Progress so far

• Feedback analysed on all three courses
• Some results circulated to T209 T224 ALs
• Some comparisons with Science made
• Telephone interviews conducted on T209 T224
  T173 next month
• Some analysis of telephone feedback done
  awaiting T173 to finalise

11
Category percentages compared Technology
Science
12
Category percentages compared Technology
courses Science
13
Category percentages compared Technology (not
T209) Science
14
Type of content comment compared Technology
Science
15
Type of content comment compared Technology
courses Science
16
Type of content commentcompared within T173
17
Depth percentages compared Technology Science
18
Depth percentages compared Technology courses
Science
19
Further comparison of depths
20
Number of comments per student
21
Action taken on T209 and T224

• Document Using TMA comments to good effect
  prepared for each course
• Explains ideas of depth and feed forward
• Contains course-specific examples of depth 2
  depth 3 comments
• Sent out in first tutor mailing (2006) with
  commendation from Course Chair
• Monitors briefed accordingly

22
Telephone surveys

• T209 and T224 complete carried out immediately
  after end of course
• Examined
• students perception of usefulness/helpfulness of
  feedback
• whether ( how) student had used the feedback in
  any future TMA / the ECA / the exam
• students preferences regarding placing of
  comments on PT3, script, (pro-forma)

23
Preliminary findings (1)

• Students are eager to receive their marked TMAs
  and do read the feedback
• but they do not necessarily use the feedback
  again in the rest of the course (approx 20 said
  they never used it)
• T209 students were likely to make more use of the
  feedback later in the course than T224 students
• T209 students particularly mentioned using skills
  development feedback

24
Preliminary findings (2)

• Not all students found that tackling the TMA, and
  the feedback they subsequently received,
  encouraged them to study the rest of the course
• A small number of students said they were
  disappointed with the quality of the feedback
  (but some were surprisingly accepting)

25
Preliminary findings (3)

• Overwhelming majority of students value comments
  on the script the most
• where lost marks made clear
• tells me exactly where the mistake is
• very specific
• more evidenced against actual text
• easier seeing my work with comments relating to
  it

26
Aside about feedback and eTMAs

• We cant assume that students read the PT3 first
  or even at all
• We cant assume that students find and read a
  separate marking document sent back with the
  marked TMA
• Some (most?) students find juggling documents on
  the screen awkward/difficult
• Its easy for tutors to place comments exactly
  where they apply
• Turn-around times longer symptom of a problem?

27
Action taken on T209 and T224

• T209 and T224Students reminded to look for and
  read the PT3
• T209Separate pro-forma dropped for eTMAs
  tutors asked to copy and paste grids at end of
  questions

28
FAST project findings in Technology?

• Significant weaknesses in current practice
• Too much emphasis on justifying the grade ?
• Lack of shared understanding of assessment
  criteria (students CT) varied
• ALs good at articulating students weaknesses
  explaining strengths problematic ?
• Lack of holistic assessment of students work ?

29
Good feedback in Technology?

• Quantity and timing of feedback
• Sufficient feedback is provided, often enough and
  in enough detail In general
• The feedback is provided quickly enough to be
  useful to students In general
• Quality of feedback
• Feedback focuses on students performance and
  learning Too much biased towards performance on
  this TMA
• Feedback is appropriate to the purpose of the
  assignment and to its criteria Purpose and
  criteria often implicit
• Feedback is appropriate in relation to students
  understanding of what they are supposed to be
  doing Sometimes, but more explanations would be
  helpful
• Student response to feedback
• Feedback is received by students, and attended to
  Yes
• Feedback is acted upon by the student Not
  sufficiently

30
Implications for Course Teams

• Different sorts of questions and criteria elicit
  different types of feedback
• (or) To elicit particular feedback, write the
  question and criteria accordingly maybe even
  write the course material accordingly

31
More implications for CTs

• Be explicit with ourselves and with ALs and
  students about what an assignments purpose and
  criteria are
• Dont assume that ALs will instinctively know
  what sort of feedback were hoping for be
  explicit in the marking guide (or elsewhere)

32
Implications for Associate Lecturers

• Need for shift towards emphasis on supporting
  students learning and progress through course,
  rather than just explaining what was wrong in
  this particular TMA
• That implies more student-centred feedback and
  more holistic feedback on PT3s
• It may also imply giving as much (more?) emphasis
  to feedback as to marks

33
Implications for monitors

• Need to shift emphasis from Was the mark OK?
  towards Was the feedback OK?
• Need to encourage appropriate forms of feedback
  (and discourage non-appropriate ones?)

34
References

• FAST presentations given at Open University, 10
  February 2005
• Gibbs, G Simpson, C (20045) Conditions under
  which assessment supports students learning
  Learning and teaching in higher education 1(1) pp
  331 available via http//www.open.ac.uk/science/
  fdtl/pub.htm

35
Centre for Open Learning of Mathematics, Science
Computing and Technology (COLMSCT) The Open
UniversityWalton HallMilton KeynesMK7 6AA

• http//cetl.open.ac.uk/colmsct