Realising the Potential of System Reform Frameworks for Evaluation and Action Keynote Presentation a

1
Realising the Potential of System Reform
Frameworks for Evaluation and ActionKeynote
Presentation at a Seminar onEducational
Evaluationat the Bank of Italy, Rome,
ItalyWednesday 24th January 2007
Professor David HopkinsHSBC Chair of
International Leadership Institute of Education,
University of London
2
High Excellence High Equity Raising the Bar and
Narrowing the Gap
560
High excellence Low equity
High excellence High equity
Finland
540
U.K.
Canada
Korea
Japan
520
U.S.
Belgium
500
Switzerland
Spain
Germany
Mean performance in reading literacy
480
Poland
460
Low excellence Low equity
Low excellence High equity
440
420
60
80
100
120
140

• 200 Variance (variance OECD as a whole 100)

Source OECD (2001) Knowledge and Skills for Life
3
Brief History of Standards in Primary Schools
11 plus dominated
Standards and
Professional control
"Formal"
accountability
"Informal"
NLNS
2004
1950
1960
1970
1980
1990
2000
2010
4
(No Transcript)
5
(No Transcript)
6
4

7
Distribution of Reading Achievement in 9-10 year
olds in 2001
575
550
525
500
475
450
425
400
375
350
325
300
Italy
Israel
Latvia
Belize
Turkey
France
Greece
Iceland
Cyprus
Kuwait
Norway
Sweden
England
Hungary
Bulgaria
Germany
Scotland
Romania
Slovenia
Morocco
Lithuania
Colombia
Argentina
Singapore
Netherlands
New Zealand
United States
Czech Republic
Hong Kong SAR
Slovak Republic
Moldova, Rep of
International Avg.
Macedonia, Rep of
Russian Federation
Iran, Islamic Rep of
Canada (Ontario,Quebec)
Source PIRLS 2001 International Report IEAs
Study of Reading Literacy Achievement in Primary
Schools
8
The High Challenge High Support Framework
Intervention in inverse proportion to success
Ambitious Standards
High Challenge High Support
Devolved responsibility
Accountability
Access to best practice and quality professional
development
Good data and clear targets
9
(No Transcript)
10
Percentage of pupils achieving level 4 or above
in Key Stage 2 tests 1998-2003
English
Maths
80
75
70
Percentage
65
60
55
50
1998
1999
2000
2001
2002
2003

• Test changes in 2003
• Major changes to writing test/markscheme
• Significant changes to maths papers

11
The Key Question - how do we get there?

• Most agree that in England
• standards were too low and too varied in the
  1970s 80s
• some form of direct state intervention was
  necessary
• the impact of this top-down approach was to raise
  standards (particularly in primary schools).
• But now
• progress has plateaued - while a bit more might
  be squeezed out nationally, and perhaps a lot in
  underperforming schools, must question whether
  this is still the recipe for sustained reform
• there is a growing recognition that to ensure
  that every student reaches their potential,
  schools need to lead the next phase of reform.
• The 64k dollar question is how do we get there?

12
Towards large scale sustainable reform
Building Capacity
Professionalism
Prescription
National Prescription
Every School a Great School
Schools Leading Reform
System Leadership
13
4 drivers mould to context through system
leadership
Personalised Learning
SYSTEM LEADERSHIP
Networks Collaboration
Professional Teaching
Intelligent Accountability
14
System Leadership A Proposition

• System leaders care about and work for the
  success of other schools as well as their own.
  They measure their success in terms of improving
  student learning and increasing achievement, and
  strive to both raise the bar and narrow the
  gap(s). Crucially they are willing to shoulder
  system leadership roles in the belief that in
  order to change the larger system you have to
  engage with it in a meaningful way.

15
Every School a Great School Framework
Intelligent Accountability
Mobilisation and Agency
System Leadership
Professionalised Teaching
Personalised Learning
Innovation and Networking
Governance and Funding
16
Complementary Policy Framework for System Reform
17
This leads inevitably to a consideration of
Intelligent Accountability

• Accountability is in some ways the foundation
  of public services today. Without accountability
  there is no legitimacy without legitimacy there
  is no support without support there are no
  resources and without resources there are no
  services
• intelligent accountability serves two
  functions it helps the system learn from itself,
  and it shows the public that they are getting
  value for money
• David Miliband
• at the North of England Education Conference
• Belfast, 8 January 2004.

18
Which I define(d) as a moreResponsive
Accountability Framework

• Forms of accountability (national testing and
  inspection) and assessment (teacher and school
  initiated as well as examinations) need to evolve
  so that the system becomes increasingly
  responsive to the knowledge available to it. The
  process should then become increasingly
  formative, assessment literate, embrace self
  evaluation, be data rich and context specific and
  promote lateral responsibility.

19
And means Getting the Assessment Balance Right
Summative
Formative
a b
c
20
Between Internal External Accountability
21
What does this mean for theArchitecture of the
Accountability Framework?

• Inspection
• Targets
• Testing
• Performance Tables
• Performance Management

22
Inspection

• Centrality of framework as the standard of
  excellence and basis for self evaluation
• Principle of differentiation in proportion to
  effectiveness
• Increasing emphasis on moderated self evaluation
• Increasing involvement of peers and use of
  parental/student attitudes
• Raise status of Chief Inspectors report as
  national indicator of performance
• Lateral accountability through inspection of
  networks and federations

23
Targets

• Moral case for national targets
• De-couple national and local/school targets
• Stretch targets based on value added and
  benchmark data for Schools/Municipalities
• Trajectory to meet national target before 2???
• Peer support and pressure through local networks

24
Tests

• Limited number of high stake tests 7 (?), 11,
  14 (?), 16 and 18
• Increased emphasis on moderated teacher
  assessment and authentic testing
• Assessment for Learning more fully embraced

25
Performance Tables

• Increase use of (contextual) value added and
  benchmark data
• Rationalise reporting anomalies and broaden
  measures
• Inclusion of collective/network measures
• Introduction of school profile or charter

26
Performance Management

• Clear focus on teaching and learning
• Use of student performance data
• Emphasis on peer support and coaching
• Link to within school variation

27
Balancing internal and external accountability
Formative
School Self Evaluation
Published value-added data
Assessment For Learning
Pupil Tracker / FFT
External
Internal
APU style random sample tests
Moderated Teacher Assessment
Achievement targets in School Profile
National Benchmarks No Notice Inspection
Summative
28
Professor David Hopkins HSBC Chair in
International Leadership
29
Appendix Further Information on the English
Education System
30
The English Curriculum and Assessment of Learning

• England has a National Curriculum that is
  compulsory in all state schools.
• It is formulated and monitored by the
  Qualifications and Curriculum Authority in
  England (QCA)
• The national Curriculum introduced Key Stages
• Key stage tests which means that pupils are
  assessed at various stages throughout their
  education, known as SATs

31
The English Curriculum
Key Stages
Subjects studied
Pupil Age
32
Assessment - the English case

• Two types of assessment
• Internal usually relates to assessment
  undertaken by a students school, college or
  other provider, commonly referred to as teacher
  assessment
• External refers to a national standardised
  exam, externally marked.
• Both can be used in a formative and summative
  way
• Formative assessment is understood as Assessment
  for Learning (AfL). Its purpose is the
  improvement of learning, it is on-going and
  embedded component of effective teaching
• Summative assessment is understood as Assessment
  of Learning. It is used for certification,
  selection, standard-setting, and accountability

33
A Summary of student assessment in the system

• Foundation Stage 5 year olds are internally
  assessed by reception class teachers against the
  early learning goals in the Foundation Stage
  Profile
• Key Stage 1 7 year olds, are internally assessed
  by teachers who will use a nationally-set Key
  Stage 1 test as one piece of evidence. The
  results of the teacher assessment, not the tests,
  go into national statistics but are not published
  school by school
• Key Stages 2 3 11 and 14 year olds, are
  assessed by both teachers and tests. The results
  are collected and published in national
  statistics. Only test results, however, feature
  in comparative tables of achievement and
  attainment by school and local education
  authority.
• Most (academic) GCSEs and A levels commonly
  involve exam papers marked by awarding bodies
  with some minor internal elements, such as
  coursework or language orals assessed by class
  teachers.
• Vocational GCSEs and A Levels are on average one
  third external and two-thirds internal
  assessment. The latter is undertaken through a
  range of practical assignments which are
  moderated or sampled.
• Work-based qualifications are mainly internally
  assessed, with most comprising about 25 external.

34
National Assessment

• For children who choose to remain at school
  after 16, most study 3 or 4 subjects of their own
  choice at Advance Level (GCE A Level). This
  is usually a 2 year course and success in these
  examinations lead to a place at University or
  College of Higher Education.

35
Reporting and rationale

• The results of KS 2 and 3 are  reported to
  parents and nationally, and are used by schools
  for self-evaluation and improvement. Only test
  results are used for school level performance
  comparisons (in the School Achievement and
  Attainment Tables)
• The rationale for this arrangement is a clear
  priority on the Core subjects of the curriculum.
  This is created by objective external standards
  combined with accountability for schools based on
  student attainment. This focuses teaching time
  and resources on the Core.

36
Assessment for Learning (AfL)

• the process of seeking and interpreting evidence
  for use by learners and their teachers to decide
  where the learners are in their learning, where
  they need to go and how best to get there.
• Assessment Reform Group, 2002
• AfL implies
• clear evidence about how to drive up individual
  attainment
• clear feedback for and from pupils so there is
  clarity on what they need to improve and how best
  they can do so
• clarity for students on what grades / levels
  they are working at, with transparent criteria to
  enable peer coaching, and
• a clear link between student learning and lesson
  planning.

37
Forms of Assessment for Learning

• Sophisticated software used in order to
• CATs assess an individuals ability to reason
  with and manipulate different types of symbols.
  These symbols represent
• Words / verbal
• Quantitative
• spatial, geometric or figural patterns / non
  verbal