Mike Treadaway

1
Using FFT Live Secondary Schools

• Mike Treadaway
• Director of Research
• Fischer Family Trust

2
Contents
3
Data Landscape
FFT
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
DfE
Value-Added
Contextual Value-Added
3 Year Summaries
Estimates
4
Whether or not to Contextualise?
Type of Analysis Advantages Disadvantages
Raw Outcomes e.g. L4 Provides a view as to whether pupils are attaining the knowledge and skills needed for their next stage of education? When used to compare schools, depends strongly upon attainment of the schools intake.
Value-Added Enables a focus upon progress irrespective of context e.g. How well are our FSM pupils doing compared to our non-FSM pupils? Comparing value-added outcomes for schools with different contexts and make-up of intakes can be misleading.
Contextual Value-Added Aims to take account of factors which are outside the schools control and which, from analysing national data, can be shown to influence progress. If used to look ahead (estimates) then danger of self-fulfilling prophecy for pupils in groups who currently under-perform nationally.
5
Differences VA and CVA

• Compare PA (Value-added) and SX (Contextual
  Value-Added) outcomes for KS2-gt4 in 2009/10.
• Use KS4 Capped Points Score as outcome measure.
• Look at how many change significant state.
• Analyses by grouping schools in different
  socio-economic contexts.

School VA compared to CVA VA compared to CVA
Context Better Worse
Q1 30 2
Q2 9 11
Q3 7 17
Q4 14 13
School Change in VA Score Change in VA Score Change in VA Score
Context Avg Min Max
Q1 7 -19 25
Q2 -1 -53 38
Q3 -3 -62 32
Q4 0 -54 34
In Lambeth, most schools have VA better than CVA
this is mainly related to the progress of
ethnic minority pupils.
To get some idea of the likely change for your
school, compare PA and SX value-added outcomes on
FFT Live or in FFT Self-Evaluation reports.
6
Recommendations
7
Expected Progress

• DfE Approach
• Whole Levels, Subject Specific Input
• So, for example, 68 L4 in Mathematics at KS2
  attained grade C or higher at KS4 in 2011
• FFT Approach
• Differentiate within levels (using Fine Grades)
• Take into account prior attainment in English and
  Mathematics

KS2 English C KS4 Maths
3 50
4C 61
4B 69
4A 75
5 80
KS2 Maths KS4 C Maths
4C 47
4B 69
4A 86
8
FFT Live

• Includes
• Latest updates
• Future releases
• Help Files

9
SETS

• Key Changes for 2011/12
• Default changed to PA (Prior Attainment)
• SX (Contextual VA) continues to be available
• Options to SORT and FILTER (Pupil Groups)
  available on range of reports
• Default layout for KS4 and KS5 Subject Summary VA
  Simplified

10
Self-Evaluation

• Significant Areas Grid
• Range of pupil groups and indicators
• KS2-gt3, KS2-gt4, KS3-gt4
• 3 Year summary, Overall Trends

• Significant Areas Detail
• Shows actual attainment and VA/CVA scores for
  each year
• Can be accessed by clicking on Significant Areas
  grid

• Subject Summary Reports
• KS4 and KS5 Individual Subjects
• Summary Report (most users find this is
  sufficient)
• Detail Report

• Pupil Value-Added
• Overall Indicators KS2-gt3, KS2-gt4, KS3-gt4
• Individual Subjects KS2-gt4, KS3-gt4, KS4-gt5
• VA or CVA (KS3 and KS4 outcomes)

11
Estimates

• School Summary Estimates
• Range of Indicators and Pupil Groups
• A PA 50th Percentile B SE 50th Percentile,
• D SE 25th Percentile S School Previous

• Pupil Summary Estimates
• Types A, B, D
• Range of indicators (e.g. 5AC, 5ACEM, Points)

• School Subject Estimates
• Core Subjects including Expected Progress for
  English and Mathematics
• Outcomes for all grades (A to G)
• Some pupil groupings available

• Pupil Subject Estimates
• Show likelihood of attaining all grades (A to G
  or A to E)
• Also provides option to show most likely grade
  (GM)

12
Using Estimates
Two basic approaches

• Start with pupil data
• Take into account other factors
• Add up and compare with overall estimates
• Revise as necessary to arrive at overall targets

• Use overall estimates and evaluation of past
  performance to draft overall targets
• Use RANK option in FFT School Subject Estimates
  to set overall figure at your overall targets
• Click on link to pupil estimates

• Estimate -gt Prediction -gt Target
• Consider option of range - realistic and
  ambitious
• Dont add up estimated levels / grades

13
Adding up estimated levels why not?

• Consider a group of 50 identical pupils
• They all have the same estimates

G F E D C B A A GM HGM GA
0 0 10 20 40 20 10 0 C 30 B
If we add up the GM grades
E D C B A
0 0 50 0 0
If we average the chances for each level and
multiply by number of pupils
E D C B A
10 20 40 20 10
5 10 20 10 5
Note Adding up predicted or target grades is
fine assuming that they are not simply just
using unmoderated estimates!
14
KS4 Estimates
Default
Extended

• GM Grade likely to be achieved by 50 of more
  similar students nationally
• HGM likelihood of attaining above GM
• GA one grade above GM
• EP Expected Progress from KS2-gtKS4 (English and
  Mathematics only)

15
Other Reports

• Export Section
• Provides additional data (e.g. UPN, FFT_ID) which
  is not available on SETS reports
• Removes formatting and colours, so easier to
  import into other systems

• Student Explorer
• Select students by range of characteristics
• Look at individual pupil history
• Developments
• FSM as pre-set group (can currently select using
  drop-down menus)
• Pilot estimate report which takes into account
  factors such as EAL and when arrived in UK

16
Questions