The characteristics of highperforming education systems: Lessons from international comparisons

1
The characteristics of high-performing education
systems Lessons from international comparisons

• Barry McGawMelbourne Graduate School of
  Education, University of Melbourne

Gold Coast, 7 August 2009
Revolutions, Revelations RealityPDN School
Leaders Conference 2009
2
There is a rising demand for high-level skills
3
Changed demand for skills in the US
The dilemma for schools The skills that are
easiest to teach and test are also the ones that
are easiest to digitise, automate and outsource.
Autor, D., Levy, F. and Murnane, R. J., (2003)
The skill content of recent technical change,
Quarterly Journal of Economics 118, M.I.T. Press,
Cambridge, pp.1279-1334. Levy, F. and
Murnane, R.J. (2006), How Computerized Work and
Globalization Shape Human Skill Demands, working
paper, available at http//web.mit.edu/flev
y/www/computers_offshoring_and_skills.pdf.
4
The storyline so far
There is a growing labour market demand for
higher level skills.
5
How good is Australian school education?
6
Using international comparisons
7
Making international comparisons of achievement
requires decisions about...
what to assess,
whom to assess.
8
Deciding what to assess...
looking back at what they were expected to have
learned
OR looking ahead to what they can do with what
they have learned.
OECD/PISA chose the latter. IEA studies have
chosen the former.
NAEP intention was the latter (as for PISA)but
it became more curriculum-oriented (as for IEA)
9
PISA defines science performancein terms of a
students
For example When reading about a health issue,
can students separate scientific from
non-scientific aspects of the text, apply
knowledge and justify personal decisions?

• Scientific knowledge and use/extrapolation of
  that knowledge to
• identify scientific issues,
• explain scientific phenomena, and
• draw evidence-based conclusions about
  science-related issues
• Understanding of the characteristic features of
  science as a form of human knowledge and enquiry
• Awareness of how science and technology shape our
  material, intellectual and cultural environments
• Willingness to engage with science-related issues

10
PISA defines science performancein terms of a
students

• Scientific knowledge and use/extrapolation of
  that knowledge to
• identify scientific issues,
• explain scientific phenomena, and
• draw evidence-based conclusions about
  science-related issues
• Understanding of the characteristic features of
  science as a form of human knowledge and enquiry
• Awareness of how science and technology shape our
  material, intellectual and cultural environments
• Willingness to engage with science-related issues

For example Can students distinguish between
evidence-based explanations and personal opinions?
11
PISA defines science performancein terms of a
students

• Scientific knowledge and use/extrapolation of
  that knowledge to
• identify scientific issues,
• explain scientific phenomena, and
• draw evidence-based conclusions about
  science-related issues
• Understanding of the characteristic features of
  science as a form of human knowledge and enquiry
• Awareness of how science and technology shape our
  material, intellectual and cultural environments
• Willingness to engage with science-related issues

For example Can individuals recognise and explain
the role of technologies as they influence a
nations economy? Or are they aware of
environmental changes and the effects of those
changes on economic/social stability?
12
PISA defines science performancein terms of a
students

• Scientific knowledge and use/extrapolation of
  that knowledge to
• identify scientific issues,
• explain scientific phenomena, and
• draw evidence-based conclusions about
  science-related issues
• Understanding of the characteristic features of
  science as a form of human knowledge and enquiry
• Awareness of how science and technology shape our
  material, intellectual and cultural environments
• Willingness to engage with science-related issues

Interest in science, support for scientific
enquiry, responsibility for the environment This
addresses the value students place on science,
both in terms of topics and in terms of the
scientific approach to understanding the world
and solving problems.
13
Example question Sunscreens

• Mimi and Dean wondered which sunscreen product
  provides the best protection for their skin.
  Sunscreen products have a Sun Protection Factor
  (SPF) that shows how well each product absorbs
  the ultraviolet radiation component of sunlight.
  A high SPF sunscreen protects skin for longer
  than a low SPF sunscreen.
• Mimi thought of a way to compare some different
  sunscreen products. She and Dean collected the
  following
• two sheets of clear plastic that do not absorb
  sunlight
• one sheet of light-sensitive paper
• mineral oil (M) and a cream containing zinc
  oxide (ZnO) and
• four different sunscreens that they called S1,
  S2, S3, and S4.

14
Example question Sunscreens

• Mimi and Dean included mineral oil because it
  lets most of the sunlight through, and zinc oxide
  because it almost completely blocks sunlight.
• Dean placed a drop of each substance inside a
  circle marked on one sheet of plastic, then put
  the second plastic sheet over the top. He placed
  a large book on top of both sheets and pressed
  down.
• Mimi then put the plastic sheets on top of the
  sheet of light-sensitive paper. Light-sensitive
  paper changes from dark grey to white (or very
  light grey), depending on how long it is exposed
  to sunlight.
• Finally, Dean placed the sheets in a sunny place.

15

• Question
• Which one of these statements is a scientific
  description of the role of the mineral oil and
  the zinc oxide in comparing the effectiveness of
  the sunscreens?
• A. Mineral oil and zinc oxide are both factors
  being tested.
• B. Mineral oil is a factor being tested and zinc
  oxide is a reference substance.
• C. Mineral oil is a reference substance and zinc
  oxide is a factor being tested.
• D. Mineral oil and zinc oxide are both reference
  substances.

16

• Question
• The light-sensitive paper is a dark grey and
  fades to a lighter grey when it is exposed to
  some sunlight, and to white when exposed to a lot
  of sunlight. Which one of these diagrams shows a
  pattern that might occur? Explain why you chose
  it.
• Full Credit A. With explanation that the ZnO
  spot has stayed dark grey (because it blocks
  sunlight) and the M spot has gone white (because
  mineral oil absorbs very little sunlight).
• A. ZnO has blocked the sunlight as it should and
  M has let it through.
• I chose A because the mineral oil needs to be the
  lightest shade while the zinc oxide is the
  darkest.
• Partial Credit A. Gives a correct explanation
  for either the ZnO spot or the M spot, but not
  both.
• A. Mineral oil provides the lowest resistance
  against UVL. So with other substances the paper
  would not be white.
• A. Zinc oxide absorbs practically all rays and
  the diagram shows this.
• A because ZnO blocks the light and M absorbs it.

A
B
C
D
17
Deciding whom to assess...
grade-based sample
OR age-based sample
OECD/PISA chose the age-baseddifferences in
school starting agedifferences in grade
retention/promotion policies.IEA studies have
chosen grade-based.
NAEP started as age-based (as for PISA)became
grade-based in 1980s (as for IEA)
18
Problem of age-based sample (IEA/PIRLS)
Countries that perform best are those that test
older students.
R 0.55R2 0.30
19
OECDs PISA assessment of the knowledge and
skills of 15-year-olds
Coverage of world economy
83
77
81
85
86
87
20
What do international comparisons tell us?
21
Mean reading results (PISA 2000)
Australia tied for 2nd with 8 othersamong 42
countries.
OECD (2003), Literacy skills for the world of
tomorrow Further results from PISA 2000, Fig.
2.5, p.76.
22
Australias ranking in OECD/PISA Reading

• Reading ranks
• PISA 2000 4th but tied for 2nd
• PISA 2003 4th but tied for 2nd
• PISA 2006 7th but tied for 6th

FinlandKoreaCanadaNZHong Kong
23
Trends in reading performance
Higher performers in Korea improved.
Korea
Finland
Lower performers in HK improved.
Hong KongChina
Canada
New Zealand
Australia
Changes for Finland, Canada New Zealand are not
significant.
OECD (2007), PISA 2006 science competencies for
tomorrows world, Vol. 1 - analysis, Fig. 6.21,
p.319.
24
Trends in Australian reading performances
95th ile
90th ile
75th ile
Mean
25th ile
10th ile
5th ile
OECD (2007), PISA 2006 science competencies for
tomorrows world, Vol. 1 - analysis, Fig. 6.21,
p.319.
25
Mean mathematics results (PISA 2003)
Australia tied for 5th with 8 othersamong 40
countries.
OECD (2004), Learning for tomorrows world first
results from PISA 2003, Fig. 2.16b, p.92.
26
Australias ranking in OECD/PISA Mathematics

• Mathematics ranks
• PISA 2000 6th but tied for 3rd
• PISA 2003 11th but tied for 5th
• PISA 2006 13th but tied for 9th

TaiwanFinlandHong KongKoreaNetherlandsSwitzer
landCanadaMacao
27
None of those that moved ahead of Australia in
mathematics from 2003 to 2006 improved
significantly.
Australia did not move significantly from 2003 to
2006.
Australias significant shift in ranking is a
consequence of cumulative non-significant shifts
in different directions.
28
Trends in Australian mathematics performances
95th ile
90th ile
75th ile
Mean
25th ile
10th ile
5th ile
OECD (2007), PISA 2006 science competencies for
tomorrows world, Vol. 1 - analysis, Fig. 6.21,
p.319.
29
Mean science results (PISA 2006)
Australia tied for 4th with 7 othersamong 57
countries.
OECD (2007), PISA 2006 science competencies for
tomorrows world, Vol. 1 - analysis, Fig. 2.11b,
pp.56-57.
30
Australias ranking in OECD/PISA Science

• Science ranks
• PISA 2000 8th but tied for 3rd
• PISA 2003 6th but tied for 4th
• PISA 2006 8th but tied for 4th

FinlandHong KongCanada
31
The storyline so far
There is a growing labour market demand for
higher level skills.
International comparisons on quality in education
show that

• Australian students are relatively high
  performing.

• The competition is not standing still.

32
The impact of raising expectations of low
performers
33
Variation in reading performance (PISA 2000)
Variation of performance within schools
Variation of performance between schools
OECD, UNESCO (2003), Literacy skills for
tomorrows world further results from PISA 2000,
Table 7.1a, p.357.
34
Variation in mathematics performance (PISA 2003)
Variation of performance within schools
Variation of performance between schools
OECD (2004), Learning for tomorrows world, Table
4.1a, p.383.
35
Trends in reading performance
Korea
Finland
Hong KongChina
Canada
New Zealand
Australia
Poland
Lower and higher performers in Poland improved.
Lower performers in Poland improved.
Changes for Finland, Canada New Zealand are not
significant.
OECD (2007), PISA 2006 science competencies for
tomorrows world, Vol. 1 - analysis, Fig. 6.21,
p.319.
36
The storyline so far
There is a growing labour market demand for
higher level skills.
International comparisons on quality in education
show that

• Australian students are relatively high
  performing.

• The competition is not standing still.

International comparisons on equity in education
show that

• Setting high expectations for all can improve low
  performers.

37
Elite student achievement is important too.
38
Scientific excellence of 15-year-olds and
countries research intensity
R2 0.70
OECD (2007), PISA 2006 Science Competencies for
Tomorrows World, Vol. 1, p.51.
39
Equity matters too.
40
at each reading proficiency level PISA 2000
Korea has relatively high mean but with few very
high performers and very few low performers.
Level 5
Australias mean is high because of its
relatively high percentage of very
high-performing students.
Level 4
Level 3
Level 2
Level 1
Australia has somewhat more low performing
students than some high-performing countries
around it.
BelowLevel 1
Source OECD, UNESCO (2003) Literacy skills for
the world of tomorrow, Table 2.1a, p.274
41
at each science proficiency level PISA 2006
Level 6
Level 5
Level 4
Level 3
Level 2
Level 1
Australias percentage of low performing students
is similar to those in other relatively high
performing countries around it.
BelowLevel 1
OECD (2007), PISA 2006 science competencies for
tomorrows world, Vol. 1 - analysis, Fig. 2.11a,
p.49.
42
Socioeconomic status reading literacy (PISA
2000)
Two indices of relationship
High
Social gradient
Correlation or variance accounted for
Social gradient Magnitude of increment in
achievement associated with an increment in
social background (on average)
Reading literacy
Correlation How well the regression line
summarises the relationship
SocialAdvantage
Low
PISA Index of social background
Source OECD (2001) Knowledge and skills for
life, Appendix B1, Table 8.1, p.308
43
Social gradients for reading literacy (PISA 2000)
High
Finland
Canada
Reading literacy
Australia
Germany
Steeper slope less equitable results
SocialAdvantage
Low
PISA Index of social background
Source OECD (2001) Knowledge and skills for
life, Appendix B1, Table 8.1, p.308
44
Social gradients for reading literacy (PISA 2000)
High quality High equity
High quality Low equity
Low quality Low equity
Low quality High equity
Source OECD (2001) Knowledge and skills for
life, Table 2.3a, p.253.
45
Social gradients for science literacy (PISA 2006)
High quality High equity
High quality Low equity
Low quality Low equity
Low quality High equity
OECD (2007) PISA 2006 science competencies for
tomorrows world, Vol 1 analysis, Figure 4.6,
p.184.
46
SES-science literacy correlations (PISA 2006)
High quality High equity
High quality Low equity
Low quality Low equity
Low quality High equity
OECD (2007) PISA 2006 science competencies for
tomorrows world, Vol 1 analysis, Figure 4.6,
p.184.
47
The storyline so far
There is a growing labour market demand for
higher level skills.
International comparisons on quality in education
show that

• Australian students are relatively high
  performing.

• The competition is not standing still.

International comparisons on equity in education
show that

• Setting high expectations for all can improve low
  performers.

• Low performers are not left further behind than
  in other high-performing countries except to some
  extent in reading.

• The disadvantaged are over-represented among low
  performers.

48
What do international comparisons tell us about
Australian students engagement?
49
Australian students engagement with science
learning
OECD (2007) PISA 2006 science competencies for
tomorrows world, Vol 1 analysis, Chapter 3,
pp.122-163.
50
The storyline so far
There is a growing labour market demand for
higher level skills.
International comparisons on quality in education
show that

• Australian students are relatively high
  performing.

• The competition is not standing still.

International comparisons on equity in education
show that

• Setting high expectations for all can improve low
  performers.

• Low performers are not left further behind than
  in other high-performing countries except to some
  extent in reading.

• The disadvantaged are over-represented among low
  performers.

51
Characteristics of the best performing systems
52
Autonomy and standards
PISA science
Standards-based external examinations
School autonomy (in teacher selection)
53
Characteristics of Finland

• High-quality teachers
• More difficult to enrol in teacher education than
  medicine
• All 6-year trained with Masters on entry
• Schools responsible for all their students
• No grade repetition
• No school differentiation (before age 15)
• Early intervention for students needing it
• Largest of students given extra support is in
  Grade 1

54
bmcgaw_at_unimelb.edu.au