What research tells us about childrens progress in schools

1
What research tells us about childrens progress
in schools

• Peter Tymms
• CEM, Durham University

www.cemcentre.org
2
Outline

• Introduction
• Worldwide Movement
• Distinguishing Knowledge
• Campbell and Policies
• The Gold Standard
• Systematic Reviews and Meta-analysis
• Three Teaching Examples

3
The Failure of Educational Policies

• Reading standards in the USA have remained
  constant for decades despite the expenditure of
  trillions of dollars.
• In England The National Literacy Strategy cost
  500 million.
• Its impact on reading levels has not been
  detected.

4
A Cartoon
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
9
7
9
1
14
The Newcastle Commission

• A political promise

15
Which matters most?

• LEA
• School
• Teacher
• Pupil
• Home
• etc

16
Data Sources

• Several national datasets including
• ASPECTS, PIPS, MidYIS YELLIS
• plus KS1, KS2, KS3 GCSE
• Looked a value-added using 3 level multilevel
  models

17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Willms Diagram
25
PIPS year on year data

• The teacher effect

26
(No Transcript)
27
Which matters most?

• LEA
• School ?
• Teacher ? ? ?
• Pupil ? ? ? ? ?
• Home ?
• etc

28
A General Rule
Proximate factors dominate
29
Evidence Based Education
30
Worldwide Movement

• Campbell Collaboration
• USA
• Laws
• AERA
• What Works Clearing House
• The Netherlands
• UK
• NZ John Hattie

31
Distinguishing Knowledge

• Distinguish types of knowledge
• Distinguish layers
• Distinguish Certainties

Tacit
Explicit
Schools
Policy
Facts
Best at the moment
32
Campbell and policies

• Campbell, D. T. (1969). Reforms as experiments
  American Psychologist, 24, 409-429.

33
Hierarchy of Evidence

• Opinion
• From expert committee
• From non experimental studies
• From one quasi experimental study
• From at least one RCT
• Systematic reviews or RCTs

34
The Gold Standard

• RCTs
• Randomised Control Trials

35
Research into what works

• An example from outside education

36
Question

• If the police are called to a house where the man
  has been hitting his partner what should they do?
• A) Arrest him.
• B) Mediate.
• C) Separate the pair temporarily.

37
How to find out .

• Arrest some, mediate others and separate others.
• Then see what happens later.
• But which ones to arrest, which to caution and
  which to separate?
• Do it at random

38
Randomisation
39
Result

• Arrest him

40
Systematic Review Example

• ADHD and NICE
• Background
• Characteristics
• Prevalence
• Impact
• Nature of NICE
• Methodology
• Clinical Treatment
• Psychologists
• Education

41
First Line Treatment

• Parent-training/education programmes should be
  offered as first line treatment of ADHD.

42
Three Positive Education Examples

• Cross-age peer tutoring

43
Second

• Assessment for learning

44
Third

• Feedback

45
Effect Sizes

• Progress in one year 0.6
• Teacher Effects 1.0
• School Effects 0.3
• Assessment for learning 0.7
• Cross age peer tutoring
• small scale
  0.7
  large scale 0.2

46
Bridging the Gap between Pupil Level and District
Level Interventions
47
Little difference between Authorities
48
Peer Learning

• For mathematics and reading (and other
  attainment).
• For social outcomes
• For tutors and tutees
• Across many ages groups

49
Scotland

• Well organised Authorities
• No high stakes test in primaries
• No National Curriculum
• Alignment with the aims of peer learning

50
Fife

• Next to Dundee and the Centre for Peer Learning
• About 150 primary schools
• Already doing PIPS in P1, P3, P5 and P7
• Enthusiastic Authority

51
The Overall Plan

• Run Peer Learning in schools for two years in
  reading and mathematics.
• Use PIPS to check impact

52
We know that ..

• Peer Learning is effective
• But
• Can a whole Authority change together?
• Which is best
• Cross-age or Same-age?
• Mix or separate subjects?
• Intensive or light?

53
Cross-age versus Same-age

• Research suggests Cross-age works best
• But
• It is hard to sustain and organise

54
Mixing versus Separation

• Learning skills in one context may help in
  another
• But
• Research suggests that transfer is problematic

55
Intensive or Light

• Harder more intensive work might get better
  results
• But
• Intense work can confuse and be onerous

56
What data will we collect?

• What Heads and Teachers say
• What schools have done
• Pupils
• Progress
• Attitudes
• Home background, sex, age etc
• Behaviour
• From Fife primary schools and many others in
  Scotland

57
Allocation for forms of peer learning

• 120 schools agreed to be randomly assigned

58
Strategic Authority Decisions

• Commitment
• Funding
• Ethics
• School Support
• Parental support
• Councillors
• Timing

59
Conclusions

• Cross age peer tutoring was shown to be effective

60
Limits
61
(No Transcript)
62
(No Transcript)
63
References

• Tymms, P.B., Merrell, C, Heron, T., Jones,
  Albone, S. and Henderson, B. (2008) The
  Importance of Districts School Effectiveness and
  School Improvement 19, 3 261-274
• Tymms, P.B., Merrell, C. and Coe, R.J. (in press)
  Educational Policies and Randomised Controlled
  Trials Psychology of Education Review
• Merrell, C. and P. Tymms (2007). "Identifying
  reading problems with computer adaptive
  assessments." Journal of Computer Assisted
  Learning 23 27-35.
• The First seven Years at School (with Jones, P,
  Albone, S and Henderson. B Paper presented at
  BERA London 2007
• Bridging the Gap Between Pupil Level and District
  Level Interventions (with merrell, C., Topping,
  K. and Conlin, N.) Seventh Annual International
  Campbell Collobaraition Colloquium London 2007
• Coe, R., Tymms, P. (2008). Summary of research
  on changes in educational standards in the UK. In
  M. Harris (Ed.), Education Briefing Book 2008
  IoD Policy Paper. London Institute of Directors.
• Dickens, W. T., Flynn, J. R. (2001).
  Heritability Estimates Versus Large Environmental
  Effects The IQ Paradox Resolved. Psychological
  Review, 108(2), 346-369.
• Fitz-Gibbon, C. T. (1992). Peer and Cross-Age
  Tutoring. In M. C. Alkin (Ed.), Encyclopedia of
  Educational Research (Sixth ed., pp. 980-984).
  New York Macmillan Publishing Company.
• Petrosino, A., Turpin-Petrosino, C.,
  Finckenauer, J. (2003). Well-Meaning Programs Can
  Have Harmful Effects Lessons from Experiments of
  Programs Such as Scared Straight. Crime and
  Delinquency, 46(3), 355-371.