The role of teachers in the assessment of learning

1
The role of teachers in the assessment of learning

• Outcomes of the Assessment Systems for the Future
  project of the Assessment Reform Group
• Funded by the Nuffield Foundation

2
Assessment Systems of the Future Project

• Funding Nuffield Foundation
• Focus summative assessment of school pupils and
  the role that assessment by teachers can take in
  this
• Duration Sept 2003 to June 2006
• Method 5 expert seminars and 2 wider sets of
  consultation/dissemination conferences

3
Causes for concern

• High stakes testing causes frequent testing and
  teaching to the test
• Frequent testing affects childrens motivation
  for learning
• Teaching to the test restricts the curriculum and
  teaching methods
• External testing encourages more internal testing
  
• Reduces opportunities for formative assessment

4
Result

• Too much testing
• Validity is low (we do not get good information
  about learning, only about how good the children
  are at taking tests)
• High cost
• Negative impact on learning and teaching
• No evidence that testing drives up standards

5
Costs

• 165 hours per year spent by Year 6 teachers on
  tests in England
• 222 hours per year on other assessment and
  reporting activities
• 84 hours per year spent by Year 6 children
  on taking tests
• 240m cost of time spent on tests and
  examinations (primary sec)
• 610m total direct and indirect cost of tests
  and examinations (primary sec)
• (2003 estimates based on surveys)

6
Properties of summative assessment

• What learning outcomes are assessed (validity)
• How accurately and consistently they are assessed
  (reliability)
• Impact of the assessment (on teachers, learners,
  the curriculum and teaching methods)
• Cost (time for teaching and learning and other
  resources)

7
Properties depend upon

• How the assessment is carried out
• By testing
• By teachers
• Combination
• What the results are used for
• Information on individual students
• Evaluation/ accountability of teaching and
  teachers (groups)
• Monitoring the system (groups)

8
Uses of summative assessment in a national
assessment system

• Individual results
• Internal to the school/college (records, reports,
  guidance)
• External to the school (certification, selection,
  meeting statutory requirements)
• Group results
• Evaluation (teachers, schools and local
  authorities)
• Monitoring (year on year comparisons of averages
  at regional or national level)

9
What ought to be assessed?

• The full range of understanding, skills,
  competencies and attitudes that are the goals of
  a modern education
• In particular
• Learning with understanding, shown in ability to
  apply knowledge rather than recall facts
• Understanding learning, being able to reflect on
  the learning process and aware of how to learn

10
About reliability

• Tests and examinations are assumed to be
  reliable, but
• Strong research evidence that up to 30 pupils
  are given incorrect grades or levels
• Teachers own tests may be even less reliable and
  result in wrong decisions being made

11
Dominant impact of external assessment

• Reasons
• Often high stakes for individual students
• High stakes for teachers when results used for
  accountability
• Serious impact because of
• Influence on teachers own summative assessments
• Influence on the use of formative assessment
• Preference for using tests examinations

12
The need for change

• If current ways of assessing outcomes are not
  including important learning outcomes, are not as
  reliable as assumed and have negative impacts,
  then we must look for alternatives
• Most users of assessment want evidence of both
  academic and non-academic achievements

13
Some advantages of using assessment by teachers

• Potential for the full range of goals to be
  included when teachers collect evidence as part
  of normal work with students
• Can relieve the pressure of terminal tests and
  examinations
• Teachers can use information about students
  formatively as well as summatively
• Can release time and other costs for alternative
  use

14
Key points from experience in countries in
outside the UK

• Where teachers have become dependent upon
  external tests - at least two years of trial and
  evaluation needed for the value of new practices
  to be properly judged
•  Top-down approaches are not as effective as ones
  involving teachers in building up necessary
  procedures
•  Effective moderation and professional
  development are key factors in establishing
  confidence

15
What about the disadvantages?

• Teachers judgements often perceived as being
  unreliable
• Increase in work load for teachers
• Can lead to the same distortion of teaching as
  testing if used for high stakes accountability

16
What can be done - about reliability ?

• Evidence of unreliability of teachers judgements
  comes from studies where no guidance or training
  was given
• Research shows that, with appropriate pd and
  moderation procedures, teachers assessment can
  be highly reliable
• The training and moderation required have
  benefits beyond reliability of results they
  enhance the quality of teaching and learning
• Access to a bank of well designed tasks

17
What can be done about workload ?

• Teachers already spend a large proportion of
  their time on assessment
• In England, this amounts to
• nearly 400 hours/year in Y6, of which almost half
  is on internal or external tests or test
  preparation for pupils about 84 hours/year or
  almost four weeks
• in Y7, 8 9 the time is about 100 hours per
  class for subject teachers for pupils 20 hours
  per subject per year
• Saving half of this would more than compensate
  for extra time on moderation

18
What can be done about the high stakes ?

• Judging schools based on test results
• does not reflect all that a school strives for
• results in disproportional attention to
  borderline pupils
• encourages teaching to the test
• More valid and reliable methods for school
  evaluation taking context into account should be
  used
• High stakes should be transferred to how well
  school meets a range of goals

19
Conclusions

• We want a system capable of providing reliable
  information about a wide range of pupil
  competences
• Systems depending primarily on test results do
  not provide this
• No approach to summative assessment is without
  problems and some negative impacts on pupils and
  teachers
• Research evidence suggests that a system making
  appropriate use of assessment by teachers has far
  fewer negative consequences than one based on
  tests
• contd

20
continued

• Procedures are needed to help teacher understand
  and use criteria consistently also benefit
  teaching
• Assessment procedures need to be transparent to
  gain the confidence of users
• Summative assessment should be carried out only
  when needed to report progress, at other time
  assessment should have a formative function
• Procedures should enable evidence used for
  formative assessment to be reviewed against
  summative criteria Contd

21
continued

• To reduce the high stakes for schools that lead
  to distortion of the curriculum and teaching
• Systems for school accountability should not
  depend solely on pupils achievement results
• To provide more valid and useful information
  about national and regional standards
• Achievement should be monitored by assessing a
  sample of pupils using a wide evidence base

22
Implications for action policy

• Recognise the short-coming of current assessment
  policies in relation to validity, reliability,
  cost and impact on the curriculum, and teaching
• Consider replacing national testing, where it
  exists, by moderated teachers judgments
• Divert resources from tests into quality
  assurance and enhancement
• Review the role of teachers assessment in
  examinations for 16-19 year olds
• Promote openness in assessment procedures

23
Implications for action school management

• School assessment policy should require summative
  assessment only when necessary for reporting
  progress, not more frequently
• Establish, maintain and protect time for quality
  assurance procedures for internal summative
  assessment,
• Ensure parents understand the formative and
  summative use of assessment
• Resist pressure for test data and encourage
  positive discourse about assessment

24
Implications for action teachers

• Ensure that assessment is always used to help
  learning
• Only when a summative assessment report is
  needed, ensure that best evidence is reliably
  judged against relevant criteria
• Involve pupils in self-assessment and help them
  to understand the assessment criteria
• Take part in moderation of judgments and other
  quality assurance procedures.
• Use tests only when most appropriate, not as
  routine.

25
Implications inspectors and advisers

• Help schools to establish assessment policies
  that encourage formative use of assessment and
  moderation procedures for summative assessment
• For summative assessment encourage use of a range
  of pupils achievements
• Ensure that appropriate professional development
  in assessment is available
• Help schools set targets based on self-evaluation
  across a range of data not only on levels
  achieved by pupils

26
Implications teacher educators

• Ensure that courses allow adequate time for
• discussion of the different purposes of
  assessment and the uses made of assessment data
• trainees and participants to identify, sample and
  evaluate different ways of gathering evidence of
  pupils performance
• giving experience of generating assessment
  criteria linked to specific learning goals
• considering evidence of bias and other sources of
  error in assessment and how they can be minimised

27
Some references

• See the ARG website for information and reports
  from the ASF project
• www.assessment-reform-group.org
• Recent relevant ARG publications
• The Role of Teachers in the Assessment of
  Learning (2006) available on the ARG website and
  from the CPA Office, Institute of Education,
  London WC1H 0AL
• Gardner, J (Ed) Assessment and Learning (2006)
  London Sage

28
Formative and summative assessment working
together
Learning activities
Criteria for reporting levels
Lesson goals
a b c d etc
Moderation