National Assessment of Educational Achievement NAEA

1
National Assessment of Educational Achievement
(NAEA)

• Jimin Cho

2
Purposes of the NAEA (1)

• To diagnose the educational achievements of
  elementary, middle and high school students and
  the trends of the achievements systematically and
  scientifically.
• To provide basic reference data of improving a
  curriculum by analyzing the degrees of students'
  achieving educational goals of the curriculum and
  checking the problems of the embodiments of the
  curriculum in the classroom

3
Purposes of the NAEA (2)

• To improve the teaching and learning methods
  and produce basic data used to set up a learning
  encouragement policy by analyzing assessment
  items and the relationships between achievements
  and background variables
• To guide the schools toward better assessment
  methods by developing and utilizing new and
  appropriate assessment methods

4
A Brief History of the NAEA (1)

• NAEA (19982000)
• - Basic plan of educational achievements studied
  in 1998
• - Field test conducted in social studies and
  math in 1999
• - Field test and main test for different subject
  in 2000
• NAEA (20012002)
• - 1 of the population of each grade
• - Five subjects assessed, which was
  followed by a results
• analysis and report.

5
A Brief History of the NAEA (2)

• NAEA (2003)
• - 1 of the population of each grade
• - Five subjects major (Korean, math) and
  basic (social
• studies, science, and English)
• NAEA (2004, 2005)
• - 1 and 3 of the population of each grade
• - Major and basic subjects are exchanged.
• - In 2005, all five subjects are basic subjects
  where each test forms are composed of multiple
  choice items and constructed-response items
  together.

6
Assessment Framework

• achievement standards
• - Statements specifying the objectives and
• contents of the national curriculum enough
  to
• guideline in teaching-learning activities
• achievement levels
• - Advanced, Proficient, Basic and
  Below-level
• instructions for item development

7
Standard Setting

• Modified Angoff Method used in NAEA
• - 12 experts are allocated for each subject.
• - Three of the cut off scores are decided for
  the three levels
• after reviewing the item contents and item
  difficulties.
• - The characteristics of each achievement
  level are described
• based on the representative items of each
  level.
• - NAEP, NCA

8
Cut-off Score

• Cut-off scores and achievement levels

Advanced Proficient Basic Below-level
Advanced cut-of score
Proficient cut-off score
Basic cut-off score
9
Achievement Level

• Advanced denotes superior performance
• ( more than 80)
• Proficient represents solid academic
• performance (between 5080
  )
• Basic denotes partial mastery of prerequisite
• knowledge and skills that are
  fundamental
• for proficient work ( between 20
  50 )

10
Areas on students background variables
11
Teachers Background Variables
12
School Background Variables
13
Development of test forms and questionnaires
The conduction of a field test
Analyzing the results of field test
Revising and Finalizing the test forms and
questionnaires
14
The Role of the Institutes for the NAEA
Korean Institute of Curriculum and Evaluation

• Deliver
• The test sheets
• OMR cards
• Answer sheets
• Audiotapes of listening test
• Administration guidelines

• - Provide the boxes of test tools
• The test sheets for five
• subjects
• OMR cards
• Answer sheets
• Questionnaires
• Audiotapes of listening
• test
• Administration
• guidelines
• Answers and guides
• Markers
• Return boxes
• Disk of students list

Provincial Offices of Education
Check the test preparation and its progress
Schools
15
The sample size of the NAEA_ 2004
16
of high school students in each POE
17
The Main Test_2004
18
Test Item_2004
19
Final Report

• Produce scale score, achievement level
• Report the achievement level for each
  subject in national level
• NO report of any of rank information such as
  total score, rank, percentile, etc

20
Summary of main results of 2004

• NAEA Research Report
• NAEA Education Index

• Gender differences
• Urbanization
• Performance Trends

21
Gender difference_2004
22
Regional differences_ 2004
23
(No Transcript)
24
Average scale scores in mathematics
25
Trends in the educational achievement
(2003 and 2004)
26
Trends in the educational achievement
(2003 and 2004)
27
Future Challenges for NAEA

• Reliability and validity study- quality control
• Trend Analysis educational progress
• Scoring constructed response items
• Secondary analysis of the NAEA results
• Consistency with revision of national
• curriculum and textbook, etc.