Tutors: Who Needs them

1
Tutors Who Needs them?

• Self assessment using Grade Related Criteria
• http//www.rgu.ac.uk/celt/learning/page.cfm?pge73
  47Tutor

2
Aims of this Session

• To Review the literature on Self assessment
• To outline the research design of an experiment
  with final honours year students
• To review the results
• To discuss the implications of the research for
  assessment

3
Aims of Assessment

• Assessment has traditionally four main roles
• Formative, to provide support for future
  learning
• Summative, to provide information about
  performance at the end of a course
• Certification, selecting by means of
  qualification and
• Evaluative, a means by which stakeholders can
  judge the effectiveness of the system as a whole.

4
Role of Assessment

• Assessment as an aid to learning?
• Assessment as a means of identifying ways of
  improving?
• Assessment as a skill to be acquired by students?

5
Aims of Self assessment

• An aid to student learning
• To help students understand more clearly the
  basis on which they are assessed
• To develop self assessment skills
• To encourage students to be more self critical
  about their work
• To give students more effective feedback
• To improve the efficiency of the assessment
  process

6
The Context

• Module Economics of Taxation and Corporate Taxes
  (BS4214)
• Part of Accounting Module on taxation
• Accounting students used to assessments which
  involve a known or correct answer
• How do you assess an economics answer?
• Session 2002/3
• 48 final year accounting honours students
• Using the Common Grading scheme with grade
  related criteria
• If GRC Scheme is as transparent as it claims then
  students should be able to use it to assess
  themselves accurately

7
Literature Review

• How Reliable is Self Assessment?
• What factors influence Self Assessment?

8
Literature Review

• Mabe and West (1982)
• Review of 55 studies from 1942-1977 involving 267
  correlations
• Findings
• Poor relationship between students and tutor
  ratings
• R 0.29 SD 0.25 (high degree of variability)

9
Literature Review

• Boud and Falchikov (1995)
• Total of 68 studies from 1932 -1994
• Do students over-rate themselves? ( 17 studies)
• Good students better at rating themselves than
  bad students? (11 studies)
• Advanced PG students better than freshmen (7
  studies found PG better at self assessment)
• More practice made self assessment better? (7
  studies found that it did not!)
• Gender differences? (6 studies 3 showed women
  more accurate than men the rest no evidence)

10
Literature Review

• Larres, Ballantine and Whittington (2003) Self
  Assessment with Accounting Students
• Computer Literacy in two UK universities with
  sample of accounting students
• Vast majority over-estimated their computer
  knowledge
• Conclusion
• Self Assessment is not an appropriate means of
  determining computer literacy
• But it did provide a useful adjunct into
  students attitudes to computing and stimulated
  reflection on their abilities.

11
Literature Review

• Self Assessment as an aid to learning
• Fitzgerald (1997) Found significant improvements
  in learning amongst medical students when self
  assessment introduced

12
Literature Review

• Self Assessment as an aid to learning
• Rust (2003)
• Developed the use of grade related criteria with
  a group of 290 second year undergraduate
  students
• 140 attended a workshop in which they used the
  criteria to assess work by students from previous
  years 150 did not attend Experimental v
  control
• Performance monitored and samples controlled for
  ability
• Results significant gains both short run and
  long-run

13
Methodology

• Set three topics for students to choose
• Set up iNET discussion forum using Salmons Five
  stage model Salmon (2003)
• Gave out the criteria and ran a one hour workshop
  on what was meant be each of the criteria
• Conducted an online Q A session
• Students handed in coursework and completed a
  Self assessment proforma using the same criteria
  as the tutor.
• Students completed an evaluation of the work.
  Ways in which it could be improved. Analysis of
  self evaluation feedback comments on iNET(the
  qualitative data)
• Proformas submitted but not read by
  tutor-assessor
• Coursework assessed internally, double marked
• Comparisons made student v tutors assessment

14
Criteria

• Presentation 10
• Research 10
• Knowledge and Understanding 20
• Analysis 30
• Evaluation 30
• Overall Grade obtained by averaging profile
  of grade

15
Accuracy of Self Assessment

• Actual Grades
• Degree of Match
• Combinations of both
• Tests used
• Correlations
• Kruskall Wallis, Mann Whitney
• Pearson Chi Square

16
Results
17
Results
18
Results
19
Hypotheses

• H1 There will be no statistically significant
  differences in the degree of match in grade tutor
  v student
• H2 There will be no statistically significant
  gender differences in the degree of match between
  tutor and students assessments.
• H3 There will be no statistically significant
  differences in the degree of match made by good
  and poor students and the tutor
• Three definitions of a good student
• 1. Grade 5 and above (Broad)
• 2. Grade 6 only (Narrow)
• 3. First (Honours Narrow)
• 4. First 21 (Honours Broad)

20
Hypothesis 1
21
Hypothesis 2
22
Hypothesis 3
23
Conclusions on Quantitative Data

• There were statistically significant differences
  between the grades by the tutor and grades by
  students. Students rated themselves significantly
  below the tutor on all dimensions except
  Evaluation
• There were no significant gender differences on
  any of the dimensions with the exception of
  Research where female students underscored
  themselves on this in comparison to male students
• No strong evidence that the best students rated
  themselves better than weaker students

24
Qualitative Evidence

• Used quotes from the self evaluation form iNET
  discussion forums to ascertain whether students
  were more aware of the criteria against which
  they were assessed
• Did a content analysis of responses

25
Qualitative Evidence

• I feel I have enhanced my ability to perform
  research and critical analysis through this
  assignment Female student Grade 4
• I feel the strengths of this report was (sic)
  the research conducted as well as the knowledge
  and understanding I gained from this Female
  student overall grade 5
• As a result of this work I have learned that tax
  can be interesting!. The strengths of this work
  is (sic) in its presentation, application of
  knowledge and analysis of the issues identified
  Male student overall grade 4

26
Feedback on the exercise

• Thank you! I cant believe I got that mark ( Im
  still shaking!) I honestly did think this was one
  of my poorer pieces of coursework, but Im very
  glad you didnt agree! Thanks also for such a
  detailed feedback, its not often we get this and
  I found it very useful Female student Grade 6
• Quote from Paper
• Biggest mismatches on Research and Evaluation

27
Conclusions

• Using a universitywide grade related criteria
  scheme improves the accuracy of self assessment
• Self Assessment
• helps students unpack the criteria by which
  they are assessed
• improves feedback that tutors can give to
  students
• identifies criteria that need to be made clearer
  in the future with more detailed briefings (or
  giving students actual coursework from previous
  years to practice assessing)
• has the potential to change the role of tutor
  from front-line assessor to moderator of the
  assessment process.
• has the potential to improve both effectiveness
  and efficiency in assessment