Severe Grading in MFL - the people

1
Severe Grading in MFL- the people
draft Sat 18th

• Helen Myers (Past President, ALL Asst Head, The
  Ashcombe School)
• David Blow (Head, The Ashcombe School)
• Duncan Byrne (Chair ISMLA)
• Geoffrey Plow (Exams, ISMLA)
• John Dunford (Gen. Secretary ASCL)
• Peter Downes (former Chief ML examiner, Head,
  President ALL SHA)

ALL Assoc. for Language Learning
ISMLA Independent Schools ML Assoc
2
Severe Grading in MFL- brief history

• Long-standing concern - research by Nuttall in
  1974 Peter Downes raised over time, leading to
  recommendation in the Nuffield Report
• Series of presentations by Helen Myers and David
  Blow using DfES and other data
• Continuing concern in ML community, esp as
  Languages is no longer compulsory and numbers go
  into freefall
• Series of meetings by ALL, ISMLA etc with QCA and
  exam boards (06)

3
GCSE MFL grading

• Our outline principle is that students studying a
  particular GCSE subject should have a reasonable
  expectation that they will get comparable grades
  across a range of subjects.
• The PANDA Relative Performance Indicator measures
  the performance of students in their other
  subjects compared with the particular one. By
  the principle above, these should be negligible.
  They are not.

4
Severe grading
thanks to Peter Tymms for this phrase

• Terminology is important. Words like hard,
  standards, demand carry multiple
  connotations.
• Restate principle that pupils of comparable prior
  attainment should attain comparable grades across
  a range of subjects whether Business Studies
  which will be only started at GCSE, Maths, Drama,
  etc all of which have very varying situations for
  different pupils

5
Depressing exams (NB separate issue from grading)

• As professionals, we know (or should know) that a
  difficult paper with low scores does not need
  to lead to low grades as the grade boundaries
  /conversion to UMS will be adjusted to
  compensate...
• BUT pupils leaving that exam will perceive that
  they have performed badly, which compounds the
  severe grading problem

6
Issues for pupils

• Perceive that they are performing worse in
  languages in other subjects
• Reduces the take-up at KS4 and A-level
• In each case, pupils are doing relative
  comparisons on grades (note key issue at AS in
  French as compare with other subjs - at A2 Univs
  can make subj. by subj. adjustments)

7
Issues for staff managers

• Heads and Governors may think that languages are
  under-performing
• Mixed messages re school targets and DCFS targets
• Ritual debate each year about whether standards
  are slipping / pupils are choosing soft
  subjects........

8
Dearing Review

• National outcry at drop in numbers GCSE Aug 06 -
  Alan Johnson sets up Dearing Review
• Mr Johnson wants to see what more can be done to
  encourage14-16 year olds to study GCSE or other
  language courses leading to a recognised
  qualification. DfES Press Notice 2006/0144
  Oct 06
• Meeting Nov 06 with Lord Dearing
• Idea of broad comparability with Maths grading
• Awareness of public and political sensitivity

9
Numbers - GCSE French
German - similar pattern but 40 as many, e.g
120,000 (French 300,000)
02
06
10
Grades
02

• drop of 30,000 in number gaining A-C from 02 to
  06
• drop of 40,000 in number gaining A-C from 02 to
  08

08
German - similar pattern but 10,000 (02 to 06)
and 18,000 (02 to 08)
11
Joint Proposal to Dearing (1)

• The mandate of QCA and the examination boards is
  to ensure that "standards are maintained over
  time". This is an enormously challenging task
  and one which has a high public profile.
• Comparability of grading
• It is now accepted by QCA and the exam boards
  that pupils of similar prior/concurrent
  attainment will gain a lower grade in languages
  than most other subjects.
• The presented documentation is available at
  http//www.all-london.org.uk/severe_grading.htm
• Because this difference is embedded historically,
  and their mandate is to maintain standards over
  time, a decision from the DfES is needed to
  change this.

12
Joint Proposal (2)

• Opportunity for change
• In the light of the crisis at KS4, there is now
  the opportunity for a change.
• The very strong experience of modern language
  teachers, advisers and senior managers is that
  the discrepancy in grading is one significant
  negative factor in the decline.
• Making changes here will not in itself solve the
  overall problem, but, to use a metaphor, will
  remove having a hand tied behind one's back.

13
Joint Proposal

• Proposal
• That QCA, and thus the exam boards are mandated
  to ensure that the grades awarded in Modern
  Languages move to become comparable, on average,
  to those awarded in Mathematics. Different
  methodologies exist to measure the comparability
  (described in the papers), and these should all
  be used, so that the pupils of similar
  prior/concurrent attainment have a similar
  average grade in Modern Languages to that in
  Mathematics.
• This proposal should be treated independently of
  the many other ideas and proposals to deal with
  the issue of KS4 M.L.

14
Dearing Review

• Led to recommendation in Consultation Report (Jan
  07) confirmed in Final Report (Mar 07) to have
  definitive study published on perception of
  severe grading (with implicit corollary that
  there should be action...) - the consultation had
  found strong confirmation of the view that award
  of grades is more demanding than for most other
  subjects

15
Dearing Review - press

• Dr Coe, of Durham's curriculum, evaluation and
  management centre, believed such trends were
  repeated in other years, but insisted "the
  question of difficulty is not about content of
  the subject ... It is purely about the
  examination and grading process."
• David Willetts, Conservative education spokesman,
  said "If there is evidence modern languages is
  tougher than other GCSEs, then that is something
  that has to be corrected. They should be the same
  level of challenge as traditional academic
  GCSEs." http//education.guardian.co.uk/gcses/stor
  y/0,,2031769,00.html
• James Meikle, education correspondent
• Monday March 12, 2007

16
Political context

• I would like to wait until I have this advice
  before making any decisions about changing grade
  boundaries Alan Johnson, Secretary of State, May
  08 referring to the QCA study set up in response
  to Dearing recommendation when replying to joint
  letter from ASCL, ALL, ISMLA
• Meetings with Jim Knights, David Laws (LibDem)
  and Nick Gibb (Con) to ensure cross-party
  awareness

17
Sensitivity

• We do not underestimate the sensitivity of this
  topic
• Remember Feb 08
• get GCSE ML without speaking
• ML Orals abolished!
• Through all of our involvement, have sought to
  make politically realistic proposals - e.g Lord
  Dearing liking the idea of comparability with
  Maths no-one will say that Maths is easy

18
QCA report - Feb 08

• Introduced two definitions of comparability
• 1. Attainmentbased comparability - 8
• each boundary must be set using professional
  judgement. The judgement must reflect the quality
  of candidates work, informed by relevant
  technical and statistical evidence.' - 8
• Fundamentally, awarding meetings are required to
  ensure that the quality of work describing a
  particular grade this year is the same as that of
  the previous year. - 9

19
QCA report - Feb 08

• 2. Aptitudebased comparability
• The aptitude-based conception of comparability
  holds that two examinations may be seen as
  comparable if students of a certain calibre
  have an equal chance of achieving a particular
  grade in any examination (either within subject
  areas or between them).
• One examination is harder than another if the
  results of a (representative) group of students
  taking both examinations are worse in it. This is
  the approach to comparability on which the ALL
  proposal is based.

20
statistical case of severe grading now accepted
- see joint press release
QCA report - Feb 08

• What if French grades based on candidates key
  stage 3 test scores and their results in GCSE
  mathematics? The analyses indicated that the
  changes would be marked at the higher grades.
• About half the candidates presently awarded a
  grade B would gain a grade A as the threshold
  mark or performance standard for a grade A would
  have to move down by about half a grade width.
  There would be a similar effect at grade C.

evidence suggests that for D grade in French 70
gain higher grade in Maths
QCA report had said that only 123 schools
(excluding grammar schools) had maintained a high
rate of GCSE languages take-up (over 90 per cent
entered in each year from 2003 to2006).
21
QCA report - Feb 08

• Conclusion The present examination system is
  not based on an aptitude-based conception of
  comparability and its adoption would create a
  major threat to public confidence in students
  results. We have similar concerns about applying
  a new performance standard to languages but not
  to any other subjects, some of which might also
  want to claim special status. We do not have
  evidence that there have been significant changes
  to grade standards in GCSE languages in recent
  years. In conclusion, we do not therefore,
  recommend any adjustment to national grade
  standards in GCSE languages.
• unfortunate implication in conclusion of report
  that TL special problem with MFL Instead we
  should focus on improving levels of teaching and
  learning in modern languages in order to gain
  students' commitment and raise performance.

22
Brigshaw School Language College
presented at Language College Conference May 08
and Oct 08

• 2007 GCSE results
• 53 5A-C 40 5A-C inc En Ma
• 24 A-C in MFL i.e. 59 out of 243
• comparing MFL results with Maths
• 70 lower grade than Maths
• 23 same grade in Maths
• 7 higher grade than Maths
• so in Year 11 those not in line for C switched
  to Asset Languages (138 of 240) of which expect
  46 to get equiv grade C or higher. So (46 102)
  / 240 60 A-C in MFL for Jun 08

but is it TL issue?
23
Brigshaw School Language College

• What is most striking about the school is the
  pervasive sense that it is a place where learning
  languages matters. The school has extensive
  provision for languages, with good links across
  subjects so that pupils can see how and why they
  can apply their language skills and knowledge.
  Pupils appreciate the very good opportunities to
  get involved in visits abroad. New courses are
  ensuring that, whilst everyone has to study a
  language throughout their time at school, there
  are courses to suit different needs. Extensive
  use is made of the expertise in the language
  college to support other schools and the
  community.
• Ofsted Nov 06
• The school had also called in the examiners to
  look at the papers, and they agreed that the
  school had properly prepared the candidates

24
Way forward

• We believe it is unrealistic to move to
  comparable grading for all subjects
• Currently 4 main bands
• ML
• Maths, Sci, His, Geo
• English
• Art, Drama, PE,
• We are proposing simply that ML moves into the
  3rd band - tiny changes in grade boundaries
  involved, similar to those which have taken place
  - one-off step change

data from CEM SCORE paper
25
Situation of MFL - A/L
To asses the relative performance of subjects in
the context of uneven grading at a national
level, there is a calculation in the PANDA of 1)
the average pts score of candidates in the
subject 2) the average points score of those
candidates in the OTHER subjects they are
taken. The difference between the two figures
shows how the grading for the subject compares to
other subjects. Because the second figure is
calculated ONLY for the candidates doing the
first subject, it automatically takes account of
the ability profile of the candidates doing the
subject This establishes the national context,
and if the same calculation is done for the
school, it removes the "severe grading" issue
from the consideration of performance
nearly 1 grade on average
from post-16 PANDA national data for Relative
Performance Indicator (RPI)
20 points 1 grade
26
Situation of MFL - A/L

• French - blue
• German - black

nearly 1 grade on average
Col. 3 "difference" as in previous graph
from post-16 PANDA national data for Relative
Performance Indicator (RPI)
20 points 1 grade
27
Situation of MFL - A/L

• French - blue
• German - black

Col. 3 "difference" as in previous graph
B
C
89.0
0
-7.2
Col. 1 - average score in subject
20 points 1 grade
28
Situation of MFL - A/L

• French - blue
• German - black

Col. 3 "difference" as in previous graph
C
B
7.2
0
96.2
-7.2
20 points 1 grade
Col. 2 - average score in OTHER subjects (related
to ability profile)
29
Issues for MFL - A/L

• Note that French German have high severe
  grading factor at A/L.
• French German have the highest average grades
  in OTHER subjects
• So
• will suffer in comparison with other subjects,
• are only left with the most able
• Compounded by situation at AS for these 2004
  A/L grades,in 2003 at AS French had subject
  ave 22.9 OTHER subject ave
  29.9 (diff - 7.7)

30
A virtuous circle???

• Language grading is fair, so...
• Pupils choose MFL at KS4, feel good after their
  exams and get grades in line with other subjects
  so...
• More choose at AS and get grades in line with
  other subjects so...
• More continue to A2 and so...
• Everyone is happy!