Can perceived reflective thinking be a successful predictor of grade mark

1
Can perceived reflective thinking be a successful
predictor of grade mark?

• Ruan Jones
• School of Sport and Exercise Science

2
How the question evolved

• HLST Pedagogic Research and Development Project
  (2005-06)
• Video-streaming as formative assessment (Erskine
  and Jones, 2004 Thornhill et al., 2001)
• Self and Peer Assessment opportunities within an
  e-learning context (Bloxham, 2004 Wiltshire,
  2005)
• Measure change with Reflection Questionnaire
  (Kember et al., 2000)
• Use of a blog to encourage discourse and
  reflection (2006-07)
• Measure change with Reflection Questionnaire
  (Kember et al., 2000) again!
• Triangulate with focus group

3
BUT

• no one volunteered for the focus group!!
• Frantic change of focus.
• Emphasis on the possible discriminating
  qualities of the questionnaire itself, rather
  than the blog.
• Problem with throwing psychometric instruments
  at participants???

4
Research Development

• Commonly accepted that many courses, including
  trainee teaching and health professionals, aim to
  develop and foster reflective thinking.
• Little regard given to assessment or measurement
  of reflective thinking and extent of (Kember et
  al., 2000)
• Modules should be structured and progressive in
  the development of thinking skills.
• Are students, therefore, exhibiting signs of the
  appropriate thinking skills dependent on level?

5
Research Development (2)

• Wellwe use formative and summative measures to
  gauge student progress
• Essentially, tutor oriented our perceptions
  (however much objectivity we strive for)
  determine grade
• How much meaning for students?
• Call for greater emphasis on peer assessment
  (Bloxham, 2004 Boud et al., 1999 Tara, 2002)
• Measure student perceptions of their cognitive
  development
• Marry grade (tutor) with perception (student)
• The greater the reflection the higher the
  mark????? Lets see!!!!!

6
Aim

• Investigate the degree to which Level 2 PE
  students engaged in perceived reflective
  practice.
• Identify whether perceived reflection scores
  would successfully predict grade scores on a
  piece of assessed work
• reflect on planning and delivery of a taught
  lesson.

7
Method

• Participants
• 53 Level 2 PE students studying Games 2
• 64 male
• Instrument
• Reflective Thinking Questionnaire (Kember et al.,
  2000)
• 4 scales that measure varying degrees of critical
  thinking (Mezirow, 1991) cited in Kember et al.,
  2000)

8
Questionnaire structure

• 4 items per scale
• Questions 9 13 are negatively worded
• Mean scale score of 3
• Can you distinguish the 4 levels of critical
  thinking which the questionnaire bases itself
  upon?
• Would you be able to identify these levels?

9
Reliability statistics

• Cronbachs Alpha (Pre Post)
• HABIT UNDERSTANDING (Terrible reliability
  reported)
• REFLECTION .36 (Pre) .69 (Post)
• CRITICAL REFLECTION .82 (Pre) .84 (Post)
• Alpha coefficient above .7 deemed acceptable

10
FACTOR ANALYSIS

• PCA Analysis carried out
• Pre
• Cross loading on all subscales
• 6 factors extracted
• Post
• 4 factors extracted cross-loading between
  Reflection (.61 to .87) and Critical Reflection
  (.71 to .82)

11
Paired Samples t-test

• Increase in Critical Reflection

T (25) -3.86, p lt 0.01. Large Effect Size (.37)
12
Spearman Rank Order

• There was a significantly moderate correlation
  between Grade mark and Reflection scoring
• (N 31, r .407, p lt 0.05)

13
Discussion/Conclusions

• Very tentative (sample size!) support for the
  proposition that perceived reflective thinking
  not critical reflective thinking is a
  successful predictor of grade mark.
• Remember Level 2 students nature of assignment
  and Learning Outcomes
• Greatly defined conception of what Critical
  Reflection is (compared to other factors) but
  no significant increase in score.
• Greater awareness of what reflection is and
  significant increase in perceived reflective
  practice student conceptions not ours!!!!

14
Implications/Future research directions

• End of Module Feedback. Quo Vadis?
• Is Reflection a product or a process?
• Measured against formative or summative assessed
  work, in light of pedagogical and andragogical
  research?
• What would we find at Level 1, Level 3, PGCE etc?

15
References

• Bloxham, S. (2004). Understanding the rules of
  the game marking peer assessment as a medium for
  developing students conceptions of assessment.
  Assessment and Evaluation in Higher Education,
  29, 721-733.
• Boud, D., Cohen, R. and Sampson, J. (1999). Peer
  Learning and Assessment. Assessment and
  Evaluation in Higher Education, 24,413-426.
• Erskine, L. and Jones, M. (2004).
  Video-Streaming in Sport and Exercise Science Is
  an image worth a thousand words? LINK 8.
  Available online from http//www.hlst.heacademy.ac
  .uk/resources/link/link8_contents.html
• Kember, D. and Leung, D.Y.P with Jones, A., Loke,
  A.Y., Mckay, J., Sinclair, K., Tse, H., Webb, C.,
  Wong, F.K.Y., Wong, M. and Yeung, E. (2000).
  Development of a Questionnaire to Measure the
  Level of Reflective Thinking.
• Tara, M. (2002). Using Assessment for Learning
  and Learning from Assessment. Assessment and
  Evaluation in Higher Education, 27, 501-510.
• Thornhill, S., Arsenio, M. and Young, C. (2002).
  Video streaming A guide for educational
  development. Retrieved October 14, 2005 from
  http//www.clickandgovideo.ac.uk/
• Wiltshire, H. (2005). Developing a Formative
  Assessment Culture within Undergraduate Sports
  Science Programmes redefining the base. LINK
  14. Available online from http//www.hlst.heacade
  my.ac.uk/resources/link14/link14.pdf