PISA International Conference

1
PISA International Conference
2
Reading Performance of Hong Kongs15-Year-Old
Students in PISA
3
Contents

• I. Design of Reading Assessment
• The construct of reading literacy
• Design of PISA assessment tasks
• II. The findings
• The overall and specific performance of HK
  students on reading literacy
• III. Some Observations

4
The Construct of Reading Literacy in PISA
5
The Construct of Reading Literacy in PISA

• the ability to understand, use and reflect on
  written texts in order to achieve ones goals, to
  develop ones knowledge and potential, and to
  participate effectively in society (OECD, 1999)

6
Reading Literacy in PISA
7
The Process - Five aspects of reading

• Retrieving information
• Broad understanding
• Developing an interpretation
• Reflecting on content
• Reflecting on form

Retrieving information
Interpreting
Reflecting and Evaluating content and form
8
The Content 2 categories, 11 types
Text
Continuous
Non-Continuous
Descriptive Narrative Expositive Argumentative Inj
unctive
Charts and Graphs Tables Diagrams Maps Forms Adver
tisements
9
The Context

• Personal
• Public
• Occupational
• Educational

10
The Construct of Reading Literacy for PISA
Process (retrieving, interpreting, reflecting)
Content (Format, Type)
Context (Personal, Public, Occupation, Education)
11
The Design of PISA Literacy Assessment
12
The Hong Kong Sample

• PISA sampling procedure closely followed
• 140 schools
• 4405 15-year-old students

13
The assessment materials

• 37 stimulus texts
• 141 reading tasks
• 9 equal sets
• Chinese version verified by PISA

14
Formats of Response

• Multiple choice questions
• Complex multiple-choice items
• Closed-constructed responses
• Short responses
• Open-ended responses

15
Reporting of Reading Literacy Performance

• Four different proficiency scales
• Retrieving information sub-scale
• Interpreting text sub-scale
• Reflecting on and evaluating text sub-scale
• Combined scale
• These scores then mapped on to a five-level
  proficiency scale

16
Descriptors of skills demonstrated at each level
of the three reading proficiency sub-scales
(Appendix 3)
17
Table 5 Reading literacy score range of the
reading proficiency levels
18
The testing material A sample item

• Graffiti
• Question 1
• The purpose of each of these letters is to
• A explain what graffiti is.
• B present an opinion about graffiti.
• C demonstrate the popularity of graffiti.
• D tell people how much is spent removing
  graffiti.
• Answer B
• Situation Public
• Text format Continuous
• Aspect Interpreting texts
• Level 2
• PISA scale score 421

19
HK Students Performance

• Overall

20
(No Transcript)
21
Fig. 2 Percentage of students performing at each
of the proficiency levels on the combined reading
literacy scale
22
(No Transcript)
23
Fig. 3 A comparison of students performance on
the combined reading literacy scale
24
Descriptors of performance by students at Level 1
and Below Level 1

• Level 1 capable of completing only the least
  complex reading tasks, such as locating a single
  piece of information, identifying the main theme
  of a text or making a simple connection with
  everyday knowledge.
• Below Level 1 not capable of the most basic
  type of reading that PISA seeks to measureSuch
  students have serious difficulties in using
  reading literacy as an effective tool to advance
  and extend their skills in other areas.

25
(No Transcript)
26
(No Transcript)
27
Summary of HK students overall performance on
reading literacy

• HK has a relatively high performance (525,
  Proficiency Level 3, 6th) in reading literacy
  comparing with other countries
• Comparing with other countries, HK has a
  relatively small percentage (10) of very
  proficient readers and poor readers
• The high rank of HK is largely due to the large
  percentage of students achieving Level 3 (33)
  4 (31)

28
HK Students Performance

• Specific

29
Table 7a HK students performance at Retrieving
Information
Note There may be rounding errors.
30
Table 7b HK students performance at Interpreting
Note There may be rounding errors.
31
Table 7c HK Students performance at Reflecting
and Evaluation
Note There may be rounding errors.
32
Table 7 Proportion of students at each
proficiency level of the reading sub-scales
Note There may be rounding errors.
33
Summary of HK students performance on the
reading sub-scales

• HK students generally do better than OECD
  countries on the sub-scales
• HK students maintain their good performance on
  reflecting and evaluation
• HK high-achieving students gradually improve
  their performance from retrieving to evaluating,
  while low achievers showed a reversed pattern of
  results
• HK students performances show a greater
  disparity across the sub-scales than that of the
  OECD countries

34
Table 8 Mean percentage scores across different
text formats
2 items were deleted from the Hong Kong data set
for subsequent analysis. This OECD average is
calculated based on 127 corresponding items.
35
Table 9 Mean percentage scores across different
text types
36
Summary of HK students performance on text types

• HK students outperformed OECD countries on all
  types of texts except in the case of
  advertisement
• HK students show an advantage in argumentative
  text in the order of performance
• HK students score exceptionally better on
  Forms, Diagrams and Tables than that of
  OECD
• HK has a 7 variation across different types of
  continuous texts

37
Table 10 Mean percentage scores across different
contexts of use
38
Summary of HK students Performance across
contexts of use

• HK students are better at texts written for
  educational (68) and occupational Purposes (69)
  than for personal (65) and public uses (63)
• HK students perform exceptionally better than
  OECD average on Occupational texts

39
Conclusion and Discussion
40

• Comparatively speaking, HK lags behind in the
  number of very proficient readers.
• HK students are relatively good at high order
  reading skills.
• HK students are particularly good at
  argumentative text, but weak at advertisement.
• HK students perform better on educational,
  occupational than on personal and public
  context of reading.
• HK education is successful in producing a
  majority of students with an above-average
  reading literacy.
• Are the results contradictory to our perception?
• How good do we want our students to be?