Dr Nadia Wager

1
Emotional, Attitudinal and Behavioural Responses
to Assignment Feedback

• Dr Nadia Wager
• Principal Lecturer in Criminological Psychology
• and
• Portfolio Leader in Psychology and Sociology

2
Why the interest in feedback??

• Gibbs (2005) - motivated to raise standards in
  Higher Education ???
• The we need to begin conceptualising assessment
  as a learning modality as opposed to a
  measurement tool.
• Ramsden (1991) the helpfulness of feedback is the
  single factor that distinguishes between
  students perceptions of their favourite and
  least favourite modules.

3

• Hattie (1987) meta-analysis of 87 studies of
  student achievement feedback is the most potent
  factor in enhancing performance
• BUT often not used by students Wager (2004)
  29 of the students surveyed admitted to only
  looking at the grade and ignoring the feedback

4
Why the reluctance to engage with the feedback?

• Strategic learning style
• Feedback lacks clarity and direction
• Leaves some students confused and feeling
  helpless
• Emotional inhibition
• Good grade motivated to maintain a positive
  mood
• Low grade unable to tolerate further assaults in
  self-esteem

5
Good practice according to the literature

• Language
• avoid final vocabulary, e.g. good, right or
  poor - does not permit change or repetition
• Use formative rather than summative language,
  e.g. do, dont and how rather than did, didnt
  and why - gives direction for improvement
• Focus of Feedback
• comment on the essay and not the author
• Boud power differences between lecturer and
  student

6
So my new study

• Two Parts
• Survey of student experiences of good and bad
  feedback
• Experimental study examining the emotional,
  attitudinal and anticipated behavioural responses
  to feedback

7
Participants

• Opportunistic sample of 47 psychology students
  from all three year groups
• Asked to reflect on their last piece of
  assignment feedback
• Questions on survey were open ended

8
Q(1) How did the feedback make you feel?

• 26 (55.3) reported a positive emotional
  response
• 4.9 a negative response
• 10.6 relative indifference
• 19 inappropriate or missing data

9
Positive emotions Three themes

• motivating and empowering experiences
• confirmation
• experience of conflicting emotions

10
Motivating and empowering experiences

• 42.3 of the positive experiences
• Made me feel positive for future assignments as
  it gave me something to work towards and improve
  upon.
• Really pleased that I received positive feedback
  chuffed and felt like Id got more confidence
  to do it again.
• It helped me understand better how to improve my
  grade and made me feel more confident.
• It made me feel great and excited.

11
Confirmation

• 27 of the positive experiences
• Reassured
• Relieved
• Very good. I knew exactly what my lecturer was
  looking for.
• Proud of the mark considering I had put the
  effort in
• Happy that it passed

12
The duality of emotions

• Good, but I was also annoyed (with myself)
  because some of the feedback related to things I
  should have done, but I just forgot to do.
• I was disappointed with the mark, but the
  feedback was good because it told me how to
  improve and where I went wrong
• I felt happy with the positive comments and low
  with the negative ones

13
Negative Emotions

• 3(43) referred to a sense of disappointment,
• 2 (28.6) related a sense of unjust accusations,
• 1 (14.3) describes a diminution in self-worth
• 1 (14.3) describes a loss in confidence.

14
Unjust accusations

• Down and like my work wasnt good enough,
  although I had worked hard. Also some wasnt
  accurate so the teacher hadnt read my work
  properly.
• Upset and frustrated, I was accused of sticking
  too close to the text when I feel that was not
  the case.
• Worthless - Very disappointed with the grade and
  the manner in which the lecturer addressed my
  work made me feel worthless.

15
Feeling okaybut confused

• A bit confused, but overall okay.
• It made me feel okay as it was very detailed but
  straight to the point.
• Okay, the content was positive and well written,
  but I would have liked to have known how to
  improve further.

16
What was good about the good feedback?

• All 47 reported having experienced good feedback
• 7 themes emerged from the responses
• positive encouragement highlighting areas for
  improvement 28.6
• offering guidance on how to improve - 25.7
• writing with a future rather than a past
  orientation -20
• clarity of both expectations expressed by
  lecturers and the wording of the feedback
• opportunity to discuss the feedback with the
  lecturer.
• confirmation of success

17
The importance of clear directive guidance

• It gave direction for the future, i.e. what was
  wrong and how to change it.
• Telling me my mistakes and how to improve on them
  for my next coursework.

18
But others highlight how we can point out what
but they really do need the how.

• Well explained, but I still feel the need to get
  together with someone to work out how to improve.
• Advice for the future and where it went wrong,
  however, I am unclear where to go with it next.

19
Less Favourable Experiences of Feedback

• 15 of the sample claimed that they had never
  received bad feedback!!!!
• 3 themes emerged
• the lack of direction for improvement -23
• the lack of detail 23
• the perception of being insulted 12.8

20
Lack of direction for improvement

• The feedback said that some points were wrong,
  but it didnt give an explanation for how to
  improve.
• Where I felt I had put a great deal of effort
  into the work but was only told that I didnt
  answer the question, with no indication of where
  I went wrong.

21
Perceived Insults

• It only contained criticism so I didnt benefit
  from it.
• The lecturer was rude to me. To the point of
  giving me no positive feedback for my work only,
  negative.
• Feedback that insinuates that because the level
  of work was good the student must have copied the
  text.
• It was bad because it didnt motivate me to work
  harder. Also because it put down my work.
• It was a bit humiliating, not enough
• detail for improvement.

22
How did the good feedback help you?

• 3 super-ordinate themes
• Empowerment 58.7
• reassurance 8.7
• confidence building 15.2

23
Empowerment and taking responsibility for own
improvement

• It motivated me to improve myself.
• Enabled me to see what I needed to do on future
  assignments.
• It means that I know what to look for in my work
  that could be improved.
• It encouraged me to do more independent research
  for my exams.
• Better understanding. It woke me up and made me
  want to work much harder.

24
Confidence Building

• It gave me confidence, as I wasnt sure how well
  my writing skills were due to the original
  feedback I had previously received. It made me
  feel that I wasnt as bad as I had originally
  thought.
• It improved my confidence because I can actually
  do the assignments.
• It boosted my self-esteem and made me what to
  work harder to keep up the standard.

25
Reassurance

• It meant that I am able to keep up with what I am
  doing right.
• It pointed out that what I am doing is right.
• It motivated me to continue with my style of
  writing and essay structure.

26
Bad feedback did it hinder you???

• 22 did not believe that their experience of bad
  feed back had any negative impact on them
• 5.5 felt it had made them work harder
• 72.3 felt it had hindered their progress in some
  way majority referred to the lack of guidance
  given and the consequent sense of helplessness

27
Lack of guidance

• If Id been given ideas on how to improve I could
  have understood where I had gone wrong.
• I couldnt see where mistakes were made
• Yes, because I only know where I got it right and
  where I went wrong, but not why and how to
  improve. I dont know what made the good areas
  good.

28
Others reported more extreme and enduring effects

• It made me feel really crap and I didnt want to
  continue. It affected me quite badly.
• Yes it reduced my confidence entirely. It made
  me reluctant to start my dissertation as I felt I
  could not write cohesively in an academic manner.
  This slowed the process of writing my
  dissertation and confidence that I could do it
  right.
• I was left feeling very negative about the tutor,
  my course and the experience.

29

• I wasnt confident or comfortable handing in work
  to this lecturer again.
• Felt low for days.
• Felt slightly de-motivated when it came to
  writing the next assignment.
• De-motivated me to work hard for this subject.
• It made me feel that I wasnt capable of
  achieving.

30
Students Advice to Lecturers

• Avoid providing feedback to student through
  Blackboard with our ID numbers which everyone can
  see.
• It has been good to see lecturers address
  feedback to the whole class if most of the class
  encountered the same problem with the assignment.
• Provide an example of a good paper so that we can
  compare and understand how to achieve this level.
• Be polite.

31
The experimental study

• An experimental study of student emotions,
  attitudes and anticipated behavioural responses
  on receiving fictitious feedback.
• 2 independent variables were manipulated in the
  study
• To whom the feedback is directed - 2 conditions
• addressed to the author and written in the past
  tense
• written to the assignment and written in the
  future tense.
• Whether or not guidance as to how to improve
  statements were provided.
• Thus four different variants of the stimulus
  material.
• Age was entered as a covariate

32
Example of feedback written addressing the
assignment with a future orientation

• All sources used were appropriate for this
  assignment however, remember to use original
  journal articles and some of the key texts
  highlighted on the reading list.
• Many of the key arguments have been identified,
  although inclusion of one or two more may have
  benefited this assignment.
• Clear referencing in the text, appropriate use of
  the Harvard system, just remember to be
  particular about the use of punctuation.

33
Now with how to improve suggestions

• All sources used were appropriate for this
  assignment however, remember to use original
  journal articles and some of the key texts
  highlighted on the reading list. The range can be
  broadened by looking at both theoretical and
  empirical material relating to the topic. There
  are some chapters or journals which are
  specifically theoretical (e.g. .) whilst other
  journals tend to focus on empirical studies (e.g.
  ..) try to use a selection of both. The
  electronic database collections can be
  particularly useful.
• Many of the key arguments have been identified,
  although inclusion of one or two more may have
  benefited this assignment. Remember that reading
  around the topic when planning the essay should
  begin by being quite broad and often a focus on
  review papers/ chapters can be quite helpful at
  the start. This will help the identification of
  all of the central arguments and important search
  terms.
• Clear referencing in the text, appropriate use of
  the Harvard system, just remember to be
  particular about the use of punctuation. The
  full stop goes after the reference in brackets at
  the end of the sentence rather than before the
  bracketed reference e.g. Cats are always
• clever (Smith, 1984). This means that we
  know
• that the reference relates to the preceding
  sentence.

34
Same comments directed at the author and in the
past tense

• You have selected appropriate sources, but the
  range is a little narrow.
• You have missed a few of the key arguments.
• Your referencing in the text was largely okay,
  however, your punctuation in relation to
  referencing was incorrect.

35
Now with how to improve suggestions

• You have selected appropriate sources, but the
  range is a little narrow. The range can be
  broadened by looking at both theoretical and
  empirical material relating to the topic. There
  are some chapters or journals which are
  specifically theoretical (e.g. .) whilst other
  journals tend to focus on empirical studies
  (e.g..) try to use a selection of both. Make
  sure you attempt to use the electronic database
  collections.
• However, you have missed a few of the key
  arguments. Remember that reading around the
  topic when planning the essay should begin by
  being quiet broad. Often focusing on review
  papers/ chapters can be quite helpful at the
  start of reading for the assignment. This will
  help you to identify all of the central arguments
  and important additional search terms.
• Your referencing in the text is largely okay,
  however, your punctuation in relation to
  referencing was incorrect. The full stop goes
  after the reference in brackets at the end of the
  sentence rather than before.

36
Assessing Emotions

• Please indicate what emotions were evoked when
  you read the feedback whilst imagining that it
  related to your own assignment?
• Pleased Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Angry Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Relieved Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Upset Not at all _/_/_/_/_/_/_/_/_/_ Extremely
• Proud Not at all _/_/_/_/_/_/_/_/_/_ Extremely
• Shamed Not at all _/_/_/_/_/_/_/_/_/_ Extremely
• Content Not at all _/_/_/_/_/_/_/_/_/_ Extremely
• Embarrassed Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Deflated Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Motivated Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Happy Not at all _/_/_/_/_/_/_/_/_/_ Extremely
• Depressed Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Inspired Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• Irritated Not at all _/_/_/_/_/_/_/_/_/_
  Extremely
• 2 scales Positive and Negative Emotions

37
Assessing Attitudes towards the feedback

• On reading the feedback, how likely you are to
  think the following, if this feedback related to
  your assignment?
• a) The comments were fair Not at all
  _/_/_/_/_/_/_/_/_/_ Extremely
• b) The comments were helpful Not at all
  _/_/_/_/_/_/_/_/_/_ Extremely
• c) The comments were insulting Not at all
  _/_/_/_/_/_/_/_/_/_ Extremely
• d) The comments were accusatory Not at all
  _/_/_/_/_/_/_/_/_/_ Extremely
• e) The comments were encouraging Not at all
  _/_/_/_/_/_/_/_/_/_ Extremely
• f) The comments were meaningless Not at all
  _/_/_/_/_/_/_/_/_/_ Extremely
• Two scales
• Favourable perceptions
• Unfavourable perceptions
• Meaningless removed from the analysis

38
Anticipated behavioural responses

• Having received this feedback, how likely do you
  think you would do each of the following in
  relation to the assignment and the feedback
  received?
• Approach the lecturer and ask for more guidance
  Not at All _ / _ / _/ _ / _ / _ / _ / Very
  Likely
• Think of the lecturer as incompetent
• Ignore the feedback
• Check over the assignment to see whether the
  comments were justified
• Ask someone else to read through your assignment
  to see if the comments are justified
• Decide on a course of action to rectify one or
  two of the weaknesses
• Decide to start working earlier on future
  assignments.
• Seek further guidance from the Academic Skills
  Support Unit
• File the feedback sheet safely so that you can
  refer to it when preparing for your next
  assignment
• Throw the feedback sheet away
• Discuss the feedback with a friend to help
• workout how to improve

39
Behavioural Responses (Cont.)

• Four factors/ subscales
• De-motivated inaction
• Seeking clarification
• Empowered action
• Disbelief in the feedback

40
Analysis

• A 2 X 2 MANCOVA was performed controlling for
  the age of the participant
• Preliminary assumptions testing was conducted to
  check for normality, linearity, univariate and
  multivariate outliers, homogeneity of
  variance-co-variance matrices and
  multicollinearity. The following two violations
  were noted. Boxs Test of Equality of Covariance
  Matrices produced a p .001, thus suggesting
  that this assumption had just been violated.
  Consequently, Pillais Trace statistic was used
  to determine significance (Tabachnick Fidell,
  1996) and the Bonferroni adjustment for multiple
  analyses was applied. Thus a significance level
  of p .006 is required on the between test of
  between-subjects effects. However, the Levenes
  test of Equality of Error variances, was also
  violated on one of the dependent variables
  (holding an unfavourable perception of the
  feedback) and thus a more conservative alpha
  level, (p .003) was utilised in determining
  significance in relation to this variable.

41
Results

• Both of the independent variables produced
  results that attained a level of significance or
  borderline significance.
• Where, the Pillais Trace for the presence or
  absence of how to improve suggestions revealed
  (F8,31 2.258, p .050, h2 .368)
• Whether the feedback was addressed to the
  assignment or the author revealed (F8,31 2.084,
  p .068, h2 .350).

42
Main effect of how to improve suggestions

• Only two of the dependent variables attained a
  borderline level of significance.
• Firstly, the inclusion of how to improve
  suggestions was associated with an almost
  statistically significant higher score on
  favourable attitudes towards the feedback (mean
  17.14, sd 7.42, F8,31 8.057, p .007, h2
  .175), in comparison to when these suggestions
  are omitted (mean 13.38, sd 9.70).
• And almost statistically significant in terms of
  the anticipation of engaging in de-motivated
  inaction. Thus, those who did not receive the
  how to improve suggestions were more inclined
  (mean 8.35, sd 4.52, F8,31 7.761, p .008,
  h2 .170) to report de-motivated inaction in
  comparison to student who were presented with
  such suggestions (mean 5.50, sd 3.03).

43
(No Transcript)
44
(No Transcript)
45
Main effects of to whom the feedback is
addressed (i.e. author or assignment)

• Addressing the feedback to the author and writing
  it in the past tense was associated with
• significantly less positive affect (mean 29.0,
  sd 16.19, F8,31 9.146, p .004, h2 .194)
• and higher levels of negative affect (mean
  36.07, sd 14.28, F8,31 7.959, p .008, h2
  .173) in comparison to when the feedback is
  written to the assignment and in the future tense
  (positive affect mean 40.93, sd 15.15
  negative affect mean 23.07, sd 11.68).
• No significant interaction effect was observed
  for the two independent variables.

46
(No Transcript)
47
(No Transcript)
48
Conclusions

• The how to improve suggestions appear to impact
  upon attitudes and intended behaviour and thus we
  can imagine would be directly related to whether
  or the students attempt to use the information.
  It is proposed that the how to improve statements
  may actually enhance self-efficacy.
• Addressing the feedback directly to the author,
  particularly in the context were either the mark
  is poor or there are significant areas for
  improvement can have a detrimental impact on the
  student which may erode self-esteem and
  confidence.

49
Constructing how to improve suggestions

• Small group activity
• Thank you