PLASC/NPD User Group

1
PLASC/NPD User Group

• Mick Quinlan (Mick.Quinlan_at_naa.org.uk)
• Change, or the lack of it
• The Players suppliers and users
• NC assessment process
• A wander through the book an A-Z of the
  collection of NC assessment data

2
Forward

• Id like to change the current process but
• there are many players in the game each with
  existing systems and many dependencies
• were not good at managing change!

3
Drivers for Change

• 1998 Service delivery failure of data collection
  exercise
• 1999 Rose report Weighing the Baby
• 2004 Beasley report service delivery failure of
  KS3 English test results
• 2005 Statistics Commission report 23 measuring
  standards in English Primary schools
• 2005 Statistics Commission report 26 school
  education statistics user perspectives
• 2006 NAO 3rd validation compendium report
  quality of data systems underpinning PSA targets
• Broadly appropriate but system needs strengthening

4
Metadata

• Statistics Commission 23, Feb 2005, 11
• We feel that public presentation of the KS scores
  in statistical releases should include a clear
  statement about the uses to which the data may be
  put, and the limitations of the data in respect
  of those uses
• Statistics Commission 26, June 2005, D
• Full and informative statistical commentary
  should be provided alongside the figures to guide
  users

5
Making Good Progress
6
Changes for 2006

• KS2/3 Electronic collection of TA, including
  pupil-level non-core results for KS3, via S2S
  rather than OMR forms
• A decision has been taken by the Departments
  Head of Profession for Statistics not to provide
  this data until the quality and robustness can be
  assured (SFR34-2006)
• KS3 English Richard III in, Henry V out
• KS3 ICT onscreen test 2nd pilot (170,000 pupils)

7
The Players DfES

• DfES Paymaster/customer (TRD)
• School Census/NPD and FSP/KS1 data collection
• EduBase
• School Self-Evaluation/Improvement
• Autumn Package (benchmarks, progress, target
  setting)
• PAT/RAISEonline)
• School Accountability - Performance Tables
• SERAP, FORVUS, FFT
• National Statistics
• Statistical First Releases
• PSA Targets

8
The Players NAA

• A division of QCA (the Regulator)
• Managers of the process
• Test development (NFER, Edexcel, UCLES, RM)
• Pupil registration (DfES, schools)
• Test Printing/Distribution (COI/Granby)
• Test marking (Edexcel)
• Teacher Assessments (schools, LAs)
• Data capture (Pearson)
• Non-standard processes (in-house)

9
The Process
Schools
Borderlined scripts borderlined marksheets
school report (inc. marks/levels)
Script Revised mark and level
TeacherAssessments
Request forReview
Request forProcess review
MarkerStandardisation
7.9
Jul/Aug
7.15
Jul
7.13
Oct
7.2
May-Jun
Samplescripts
Guidance
Mark and level
Investigatereview
Marked scripts
Mark and level
Upheldreview
Dataentry
MarkScripts
Borderlining
7.14
Oct/Nov
7.8
Jun-Aug
7.1
7.7
May/Jun
Jun
Re-review
R1 Clericalcheck
Samplescripts
Cut scores
Returnedscripts
7.12
Jul/Aug
DC resultsdatabase
Script pool collection
Level-settingdata collection
Re-markedscript
R2 IndividualRe-marking
7.3
7.5
May/Jun
May
Selected scripts
7.10
Jul/Aug
Malpractice
Scriptscrutiny
Level- setting
R3/GR Group review
7.14
May/Jul
Recom-mend-ation
7.4
Jun
Special consideration
7.6
Jun
7.11
Jul/Sep
Pretest 2 Draft Level thresholds
May/Jun
7.13
10
A is for Assessment

• National Curriculum
• key stages Foundation, KS1, KS2, KS3
• Subjects
• core English, maths science
• Attainment Targets
• non-core Art Design, DT, Geography, History,
  ICT, MFL, Music and PE
• Test/Task Teacher Assessment
• Levels

11
Levels

• Q Whats the difference between Level 4 and
  Level 5?
• A 1 mark
• Making Good Progress (KS2-KS3,page 9)
• This shows that 15 pupils did not progress by
  a full level since finishing KS2 and three
  have actually regressed

12
A

• A pupil Absent from the test
• Aegrotat
• Achievement Attainment Tables
• Age Standardised Scores (ASS)
• Additional time
• Appeals (process reviews)

13
BCD

• B Working Below the level of the test
• Borderlining
• Braille
• Compensatory award
• Classes/Teaching Groups collected in 1997
• D pupil Disapplied from the curriculum
• Date of Birth

14
E

• E Exceptional Performance
• Extension Papers
• Early takers
• Errors

15
FGH

• F Future sitting (subjects early takers not
  taking)
• Fine grade
• Foot and mouth
• Gender
• Handwriting

16
IJKL

• Independent schools
• L pupil Left school (KS2/3)
• L below Level 2 in the reading comprehension
  test (KS1)

17
MNO

• M Missing scripts
• Maladministration/malpractice
• Missing pages
• Mental Maths test
• Modified large print (MLP)
• N No test level (too few marks gained)
• Names, of pupils
• Overseas/Offshore schools
• Optional tests (Y3-5 Y7/8)

18
PQR

• P Past sitting (subjects early takers took last
  year)
• Progress tests (Year 7)
• P Scales
• Q result annulled - malpractice (DfES)
• Reading
• Reviews
• Re-sits

19
ST

• Service Childrens Education (SCE) Schools
• Special Consideration
• Shakespeare
• Sub-Levels
• Spelling test
• T Pupil unable to access the tests
• Tier

20
UVW

• U No TA subject Level calculated (KS1)
• Unique Pupil Number (UPN)
• V mixed tiers
• Versions
• W Working towards Level 1
• Writing

21
XYZ

• X Marks pending (KS2/3)
• X pupil not entered for reading comprehension
  test (KS1)
• Year groups
• Z ineligible pupil