TIMSS 2003

1
TIMSS 2003

• Ina V.S. Mullis and Michael O. Martin
• 46th IEA General Assembly
• Helsinki, 10-13 October, 2005

2
TIMSS 2003

• At Last General Assembly
• Previewed Achievement Results
• Mathematics Grades 4 and 8
• Science Grades 4 and 8
• Trends 1995, 1999, 2003
• Since then...

3
Published Three International Reports

• Mathematics Eighth and Fourth Grades
• Science Eighth and Fourth Grades
• Technical Report

4
Completed International Release

• Nov 23, 2004
• Mailed International Mathematics and Science
  Reports, Technical Report, and Press Packet to
  NRCs
• Dec 14, 2004
• Released International Reports at Boston College
• IEAs Chair and Executive Director Presiding

5
Completed International Database

• Feb 2005
• Conducted Training for Participating Countries
• Cairo, Feb 7-9
• May 2005
• Released International Database and User Guide

6
Completed Developmental Project on Mathematics
Cognitive Domains

• History
• Increased Interest in Reporting Achievement in
  Cognitive Domains
• PIRLS, PISA, NAEP
• TIMSS 2003 Countries Supported a Developmental
  Project in Mathematics
• United States, New Zealand, Singapore, Norway,
  Ontario, Quebec, Chinese Taipei, Sweden, Cyprus

7
Cognitive Domains Project (Cont.)

• Purpose
• Enhance Information from TIMSS 2003
• Facilitate Planning for TIMSS 2007
• Framework
• Reporting
• Serve as a Roadmap for Science

8
Cognitive Domains Project (Cont.)

• Mapping Items to Cognitive Domains
• Expert Meeting February 2005
• Iterative Discussion and Item Classification
• Three Cognitive Domains
• Knowing Facts, Procedures, and Concepts
• Applying Knowledge and Understanding
• Reasoning

9
Cognitive Domains Project (Cont.)

• Knowing Facts, Procedures, and Concepts
• What Students Need to Know
• Applying Knowledge and Understanding
• Applying Knowledge to Solve Routine Problems
• Reasoning
• Unfamiliar Situations, Complex Contexts, and
  Multi-step Problems

10
Cognitive Domains

• Review
• SMIRC Meeting April 2005
• Refined Classifications and Their Descriptions
• TIMSS PIRLS ISC Examined Item
  Distribution

11
Cognitive Domains Project (Cont.)

• Review (Cont.)
• TIMSS PIRLS ISC Found Good Spread of Items
  Within Cognitive Domains
• Item Type
• Content Domain
• Difficulty
• Constructed Three Scales for Reporting
• Achievement Overall and by Gender

12
Cognitive Domains Project (Cont.)

• Scaling and Reporting
• Applied TIMSS 2003 Scaling Methodology
• IRT
• Same as Used for International Reports for
  Achievement Overall and Content Domains
• Published International Report
• Web and Print

13
Knowing Cognitive Domain
8th Grade Above International Average
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
14
Knowing Cognitive Domain
8th Grade Below International Average
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
15
Knowing Cognitive Domain
4th Grade
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
16
Applying Cognitive Domain
8th Grade Above International Average
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
17
Applying Cognitive Domain
8th Grade Below International Average
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
18
Applying Cognitive Domain
4th Grade
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
19
Reasoning Cognitive Domain
8th Grade Above International Average
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
20
Reasoning Cognitive Domain
8th Grade Below International Average
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
21
Reasoning Cognitive Domain
4th Grade
Country average significantly higher than
international average
p
Country average significantly lower than
international average
s
22
TIMSS 2003 Now Complete!
23
TIMSS 2003

• Ina V.S. Mullis and Michael O. Martin
• 46th IEA General Assembly
• Helsinki, 10-13 October, 2005