Benchmarking to International Standards TIMSS

1
Benchmarking to International StandardsTIMSS
PIRLS A Bridge to School Improvement
Ina V.S. Mullis and Michael O. Martin,
DirectorsTIMSS PIRLS International Study
Center Boston College Institute of Education
SciencesMay 30, 2008
This paper is intended to promote the exchange of
ideas among researchers and policy makers.  The
views expressed in it are part of ongoing
research and analysis and do not necessarily
reflect the position of the National Center for
Education Statistics, the Institute of Education
Sciences, or the U.S. Department of Education.
2
What Are TIMSS and PIRLS?

• International Assessments of Educational
  Achievement
• TIMSS
• Mathematics and science
• Fourth and eighth grades
• PIRLS
• Reading
• Fourth grade

3
Why TIMSS and PIRLS?

• Provide comparative information about educational
  achievement across countries in relation to the
  approaches used to improve teaching and learning
  in mathematics, science, and reading
• Monitor curricular implementation and
  effectiveness
• Identify most promising instructional practices
• Measure progress in achievement over time and
  across grades

4
Trends Over Time

• Grade 4
• TIMSS in 1995, 2003, 2007, 2011
• PIRLS in 2001, 2006, 2011
• Grade 8
• TIMSS in 1995, 1999, 2003, 2007, 2011
• Grade 12
• TIMSS Advanced in 1995, 2008

5
TIMSS Benchmarking U.S. States and Districts

• 1995 First in the World Consortium, Colorado,
  Illinois, Minnesota
• 1997 Missouri and Oregon
• 1999 13 states and 14 districts and consortia
• 2003 Indiana
• 2007 Massachusetts and Minnesota

6
IEAs Curriculum Model An
Educational Perspective on Educational Issues
7
What Data Are Collected?

• Intended Curriculum
• Participating countries, curriculum experts
• Routinely published in Encyclopedia
• Implemented Curriculum
• Teachers and principals of participating students
• The students themselves
• Attained Curriculum
• The mathematics, science, and reading tests
• Routinely published in International Reports

8

• 13 above average, U.S.
• 3 about average
• 9 below average
• Wide range of achievement
  across and within countries

9

• 13 above average7 about average, U.S.25 below
  average
• Wide range of achievement
  across and within countries
• Range between 5th and 95th percentiles
  approximately 270-300
• Average in Singapore same as 95th percentile in
  lowest countries

10
Widespread Participation

• TIMSS 2007 Grade 8
• 50 Countries
• 242,000 Students
• TIMSS 2007 Grade 4
• 37 Countries
• 183,000 Students
• PIRLS 2006 Grade 4
• 40 Countries
• 215,000 Students

11
Comprehensive Content Coverage

• Mathematics Framework
• Grade 4 179 items (192 score points)
• Number (50)
• Geometric shapes and measures (35)
• Data display (15)
• Grade 8 215 items (238 score points)
• Number (30)
• Algebra (30)
• Geometry (20)
• Data and chance (20)

12
Comprehensive Content Coverage

• Science Framework
• Grade 4 174 items (194 score points)
• Life science (45)
• Physical science (35)
• Earth science (20)
• Grade 8 212 items (240 score points)
• Biology (35)
• Chemistry (20)
• Physics (25)
• Earth science (20)

13
Mathematics and Science Cognitive Domains
Grades 4 8

• Reasoning (20-30)
• Analyze, integrate/synthesize, generalize,
  hypothesize/predict, draw conclusions, justify,
  solve non-routine problems and conduct
  investigations
• Applying (35-40)
• Compare/contrast, model, represent, use
  relationships and concepts to solve problems
• Knowing (30-40)
• Recall, recognize, classify, define, measure

14
Comprehensive Coverage

• PIRLS Framework Grade 4
• Purposes for Reading
• Reading for Literary Experience (50)
• Reading to Acquire and Use Information (50)
• Processes of Comprehension
• Retrieval and Straightforward Inferencing (50)
• Interpreting, Integrating Evaluating (50)
• Assessment
• 10 passages 126 items (167 score points)

15
Comprehensive Background Information About
Contexts for Teaching and Learning

• Questionnaires
• Country intended curriculum
• Student home and classroom experience
• Teacher implemented curriculum, education and
  preparation, instructional practices
• School climate, resources, composition
• Parent (PIRLS) early literacy activities, home
  resources, parental attitudes toward reading

16
Process for Updating Frameworks

• Goal Include topics regarded as important in a
  significant number of countries
• Recommendations by subject matter experts
• e.g., committees, NSF initiative for TIMSS 2003
• Questionnaires completed by participating
  countries
• Topics included in their curricula
• Desirability of assessing particular topics
• Iterative process of reviewing and revising
  successive drafts

17
Aspects of Comparative Validity

• Are target populations comparable?
• Was sampling conducted properly?
• Are translations comparable?
• Were the tests administered appropriately?
• Was scoring done correctly?
• Are the data comparable?

18
In Conclusion

• Much has been learned about effective education
  since the pioneering days
• Different countries use different approaches but
  an effective educational system always requires
  enormous effort
• High percentages of students completing high
  school, and taking advanced courses
• A rigorous and progressive curriculum
• Well-prepared teachers
• Resources for facilities and materials
• Students ready to learn and encouraged by society

19
(No Transcript)
20
Benchmarking to International StandardsTIMSS
PIRLS A Bridge to School Improvement
Ina V.S. Mullis and Michael O. Martin,
DirectorsTIMSS PIRLS International Study
Center Boston College Institute of Education
SciencesMay 30, 2008