Florida Association of Mathematics Supervisors

1

Florida Association of Mathematics Supervisors
NAEP Mathematics October 2006 Michele Sonnenfeld NAEP State Coordinator Florida Department of Education
2
National Assessment of Educational Progress
(NAEP)

• The Nations Report Card
• NAEPs two major goals are to
• Measure and compare student achievement in states
  and other jurisdictions
• 2. Report changes in achievement of 4th, 8th, and
  12th graders over time

3
NAEP

• Has produced National results for Grades 4, 8,
  and 12 since 1969
• Has produced state-level results at Grades 4 and
  8 since 1990
• Has produced results for the Trial Urban District
  Assessment (TUDA) since 2002
• Does not produce student-, school-, or
  district-level results, except for the 10 TUDAs

4
Current OrganizationalStructure of NAEP
5
Federal Legislative Reference to NAEP

• No Child Left Behind Act of 2001 (NCLB)
• Section 111(c)(2) of NCLB requires
  states/districts who receive Title I funding to
  participate in biennial State NAEP assessments of
  4th and 8th graders in Reading and Mathematics,
  beginning with the 2002-2003 school year.

6
State Legislative Reference to NAEP

• Florida State Statute 1008.22 (2)
• The Commissioner of Education shall direct
  Florida school districts to participate in the
  administration of NAEP . . . both for the
  national sample and for any state-by-state
  comparison programs .
• State legislation requires all selected schools
  to participate in all NAEP assessments.

7
NAEP Components
STATE (Public) (Grades 4 8)
8
NAEP Schedule of Assessments
NAEP Schedule of Assessments NAEP Schedule of Assessments NAEP Schedule of Assessments
Year National State
1996   Mathematics (4,8) Mathematics (4,8)
1996   Science (8) Science (8)
1998     Reading (4,8) Reading (4,8)
1998     Writing (8) Writing (8)
1998     Civics (4,8,12)  
2000     Mathematics (4,8) Mathematics (4,8)
2000     Science (4,8) Science (4,8)
2000     Reading (4)  
2002   Reading (4,8) Reading (4,8)
2002   Writing (4,8) Writing (4,8)
2003   Reading (4,8) Reading (4,8)
2003   Mathematics (4,8) Mathematics (4,8)
2005     Reading (4,8,12) Reading (4,8)
2005     Mathematics (4,8,12) Mathematics (4,8)
2005     Science (4,8,12) Science (4,8)
2006     U.S. History (4,8)  
2006     Civics (4,8)  
2006     Economics (12)  
2007     Reading (4,8,12) Reading (4,8)
2007     Mathematics (4,8,12) Mathematics (4,8)
2007     Writing (8,12) Writing (8)
2009     Reading (4,8,12) Reading (4,8)
2009     Mathematics (4,8,12) Mathematics (4,8)
2009     Science (4,8,12) Science (4,8)
9
NAEP Frameworks

• The American Institute of Research (AIR), with
  the help of teachers, develops the NAEP
  frameworks.
• The National Assessment Governing Board (NAGB)
  approves and adopts the frameworks after they
  have been reviewed and critiqued by teachers.
• Each framework describes the content and format
  of the 4th, 8th, and 12th grade assessments.
• There are frameworks for each of the content
  areas assessed by NAEP.

10
NAEP Mathematics Framework

• Current mathematics framework requires assessment
  in
• Number Properties and Operations
• Measurement
• Geometry
• Data Analysis and Probability
• Algebra

11
NAEP Design Item Development

• Items are based on the subject-area frameworks
  approved by the National Assessment Governing
  Board (NAGB).
• Items are developed by teachers, subject area
  specialists, and assessment experts.
• Items meet professional standards for testing.
• Experts review items for technical/statistical
  quality, content quality, bias, and sensitivity.

12
How is Floridas NAEP School Sample Selected?

• Step 1 Lists of schools are obtained from the
  National Center for Education Statistics Common
  Core of Data (CCD).
• Step 2 Schools are grouped into strata by two
  primary characteristics
• Type of location (central city, urban fringe,
  rural)
• Minority enrollment (white, African American,
  Hispanic)
• Step 3 Within strata, schools are ordered by a
  measure related to achievement.
• Step 4 A proportional sample is selected
  systematically from the ordered list.

13
Measure of Achievement Used for Floridas NAEP
School Sample Selection

• Grade 4
• Mathematics average scale scores for all Grade 4
  students in 2005
• Grade 8
• Median household income (because the match rate
  for achievement scores for 8th grade was less
  than 70)

14
How is Floridas NAEP Student Sample Selected?

• A list of all 4th or 8th grade students enrolled
  at the sampled schools is submitted
  electronically to Westat from the FLDOE, based on
  the October FTE data. Westat randomly selects
  students chosen to participate in NAEP.
• 90 of selected students at each school must
  participate in the assessment or make-up sessions
  are scheduled at schools not meeting this
  benchmark.
• Students exempted by parental request are not
  replaced and are not used to calculate the
  assessment participation rate.

15
NAEP 2007

• NAEP assessments will be administered in Florida
  from January 22 through February 21, 2007.
• Each student takes only a portion of the
  questions, answering two 25-minute sections of
  subject-area questions.
• Students also answer questions related to their
  background and their home or school experiences
  in the subject of their assessment.
• Assessment is administered by a team from Westat,
  the NAEP administration contractor for the US
  DOE, and the National Center for Education
  Statistics (NCES).
• Parents must be notified prior to the assessment.
  Refusals must be in writing.

16
NAEP Sampling School and Student Selection

• 350 Florida public schools in 49 districts will
  participate in some type of NAEP Assessment in
  2007.
• Grade 4 Reading and Mathematics
• Floridas sample will include 167 schools.
• Schools can elect to take all students (up to
  120).
• If there are more than 120 4th graders, NAEP
  will select a sample of 62 students to assess.
• Grade 8 Reading, Mathematics, and Writing
• NAEP will select a sample of 93 students at each
  of the 161 Florida schools included in the
  sample.
• Grade 12 Reading and Writing
• Floridas sample will include 31 high schools
  and
• 75 students from each school will be selected to
  participate.

17
NAEP Inclusions and Accommodations

• NAEP has included students needing accommodations
  since 1998.
• Students with disabilities (SD) and English
  language learners (ELL) are to be included in
  NAEP based on an IEP/504 Accommodations Plan/LEP
  Committee decision.
• Most frequent accommodations with NAEP
• Extra testing time
• Individual or small group administrations
• Large print booklets
• Assessment must be completed in one day.

18
NAEP Analysis and Reporting

• National, State, and 10 Trial Urban District
  Assessment (TUDA) results.
• NAEP scores are reported in two ways by Scale
  Scores and by Achievement Levels
• No student, classroom, or school results are
  provided
• No district results are provided except for the
  10 TUDAs
• Subgroup, state, and national comparisons can be
  made
• Relationships between student proficiency and
  certain background variables are available

19
NAEP Scale Scores

• Scale scores provide a comparison of student
  performance with regard to a group (e.g., the
  nation) or a subgroup (e.g., ELL) on a set of
  items.
• Scale scores
• Reading and Mathematics scale score range is
  0-500
• Science and Writing scale score range
• is 0-300
• Scale scores are reported as an average for
  groups of students and may obscure progress or
  problems related to student performance across a
  distribution of scores.

20
National NAEP Average Scale Scores Grades 4 and 8
Mathematics 1990-2005

• National NAEP mathematics average scale scores
  are higher in 2005 than in 1990 by 25 points at
  Grade 4 and by 16 points at Grade 8.

21
Florida Compared to the Other 49 States, District
of Columbia, and Department of Defense
SchoolsNAEP 2005 Average Scale Scores Grade 4
Mathematics
Of the 50 states and two jurisdictions that
participated in the NAEP 2005 Grade 4
mathematics assessment, students average scale
scores in Florida were higher than that of 19,
not significantly different from that of 17, and
lower than that of 15.
22
Florida Compared to the NationNAEP Average Scale
Scores 1996-2005 Grade 4 Mathematics
Florida improved from scoring below the National
average in 1996 to above the National average in
2005 on NAEP Grade 4 mathematics.
23
Florida Compared to the Other 49 States, District
of Columbia, and Department of Defense
SchoolsNAEP 2005 Average Scale Scores Grade 8
Mathematics
Of the 50 states and two jurisdictions that
participated in the NAEP 2005 Grade 8
mathematics assessment, students average scale
scores in Florida were higher than that of 10,
not significantly different from that of 8, and
lower than that of 33.
24
Florida Compared to the NationNAEP Average Scale
Scores 1996-2005 Grade 8 Mathematics
Floridas Grade 8 NAEP mathematics performance
parallels and continues to score below National
performance.
25
NAEP Grade 4 Mathematics 1996-2005Florida and
the Nation Average Scale ScoresRace/Ethnicity
26
NAEP - Florida and the Nation Grade 4
Mathematics 1996-2005

• Performance Gaps Between
• Average Scale Scores
• by Race/Ethnicity

27
NAEP Grade 8 Mathematics 1996-2005Florida and
the Nation Average Scale ScoresRace/Ethnicity
28
NAEP - Florida and the Nation Grade 8
Mathematics 1996-2005

• Performance Gaps Between
• Average Scale Scores
• by Race/Ethnicity

29
NAEP Achievement Levels

• NAEP Achievement Levels (ALs) try to identify
  what students should know and be able to do at
  various achievement levels.
• Advanced, Proficient, Basic, and below Basic
  indicate NAEP achievement levels. Each is
  determined by cutpoints established by
  professional educators and NAGB.
• The number or percentage of students in the
  population at large or in subpopulations can be
  reported by the four achievement levels.
• Results show how different groups are performing
  in relation to each other and over time.

30
Florida Compared to the Other 49 States, District
of Columbia, and Department of Defense
SchoolsNAEP 2005 at Basic and above Grade 4
Mathematics
Of the 50 states and two jurisdictions that
participated in the NAEP 2005 Grade 4
mathematics assessment, the percentage of
students scoring at Basic and above in Florida
was higher than that of 20, not significantly
different from that of 16, and lower than that of
14.
31
NAEP Achievement Levels Florida and the
NationGrade 4 Mathematics 1992-2005
Florida has a greater percentage of Grade 4
students scoring at Basic and above in NAEP 2005
mathematics than does the Nation.
32
Florida Compared to the Nationat Basic and above
Grade 4 NAEP Mathematics 1996-2005
Florida went from scoring below the National
average in 1996 to scoring above the National
average in 2005 at the Basic and above
achievement level in NAEP Grade 4 mathematics.
33
Florida NAEP Grade 4 Mathematics Summary of
Achievement Level Results

• The percentage of students in Florida who
  performed at or above the NAEP Basic level was 82
  percent in 2005. This percentage was greater
  than in 2003 (76 percent), and greater than that
  in 1992 (52 percent).

34
Florida Compared to the Other 49 States, District
of Columbia, and Department of Defense
SchoolsNAEP 2005 at Basic and above Grade 8
Mathematics
Of the 50 states and two jurisdictions that
participated in the 2005 Grade 8 mathematics
assessment, the percentage of students scoring at
Basic and above in Florida was higher than that
of 9, not significantly different from that of
12, and lower than that of 30.
35
NAEP Achievement Levels Florida and the
NationGrade 8 Mathematics 1990-2005
Florida has a smaller percentage of students
scoring at Basic and above than does the Nation
on Grade 8 mathematics.
36
Florida Compared to the Nationat Basic and above
Grade 8 NAEP Mathematics 1996-2005
Florida Grade 8 mathematics performance parallels
and continues to score below the National
performance.
37
Florida NAEP Grade 8 Mathematics Summary of
Achievement Level Results

• The percentage of students in Florida who
  performed at or above the NAEP Basic level was 65
  percent in 2005. This percentage was not
  significantly different from that in 2003 (62
  percent) and was greater than in 1990 (43
  percent).

38
FCAT and NAEP

• FCAT measures student performance on selected
  benchmarks, as defined by Floridas Sunshine
  State Standards (SSS)
• NAEP reports on the performance of groups of
  students at a given time - and across time -
  without specifying how a subject should be taught

39
FCAT and NAEP ParticipationGrade 4 Mathematics
2005
All Students White Students African American Students Hispanic Students
FCAT 195,868 93,779 44,899 45,934
FL NAEP 4,347 2,130 1,000 1,000
National NAEP 162,942 92,877 27,700 32,588
40
FCAT and NAEP ParticipationGrade 8 Mathematics
2005
All Students White Students African American Students Hispanic Students
FCAT 201,490 100,760 46,291 45,521
FL NAEP 3,927 2,042 864 864
National NAEP 152,750 91,650 25,968 25,968
41
FCAT, Florida NAEP, and National NAEPGrade 4
Mathematics 2003 and 2005

• Achievement Levels
• FCAT Level 3 and above
• NAEP Percent Basic and above

42
FCAT, Florida NAEP, and National NAEPGrade 8
Mathematics 2003 and 2005

• Achievement Levels
• FCAT Level 3 and above
• NAEP Percent Basic and above

43
FCAT and Florida NAEP Grade 4 Mathematics by
Achievement Level2003 and 2005
Percentage of students scoring at Level 3 and
above on the FCAT and Basic and above on NAEP on
Grade 4 mathematics increased between 2003 and
2005. There is a 10-point gain on the FCAT and a
6-point gain on Florida NAEP.
44
FCAT and Florida NAEP Grade 8 Mathematics by
Achievement Level 2003 and 2005
Performance of Grade 8 students in mathematics
has been flat on both the FCAT and NAEP. There
is a 3-point gain on both the FCAT and Florida
NAEP between 2003 and 2005.
45
FCAT and Florida NAEP by Achievement LevelGrade
4 Mathematics 2003 and 2005 Race/Ethnicity
White, African American, and Hispanic students
improved on the Grade 4 mathematics FCAT and
NAEP between 2003 and 2005.
46
FCAT and Florida NAEP by Achievement LevelGrade
8 Mathematics 2003 and 2005 Race/Ethnicity
White, African American, and Hispanic students
continued to score higher on Grade 8 mathematics
NAEP than on the FCAT.
47
The Nations Report Cardhttp//www.nces.ed.gov/na
tionsreportcard/
48
NAEP Websites

• Floridas NAEP website
• http//firn.edu/doe/sas/naephome.htm
• NAEP website
• http//nces.ed.gov/nationsreportcard/
• NAEP Data Explorer
• http//nces.ed.gov/nationsreportcard/nde/
• NAEP Questions Tool
• http//nces.ed.gov/nationsreportcard/itmrls

49
NAEP Coordinator Contact Information

• Michele Sonnenfeld
• NAEP State Coordinator
• Florida Department of Education
• [phone]
• SC 205-0787
• FAX [phone]
• SC 205-0781
• Michele.Sonnenfeld_at_fldoe.org