Understanding the New AYP

1
Understanding the New AYP

• Tim Kurtz, NH DoE
• 271-3846 TKurtz_at_ed.state.nh.us
• Deb Wiswell, NHDoE
• 271-3828, DWiswell_at_ed.state.nh.us

2
2006 AYP Results(impacts school year 2006-2007)
AYP Definition Grades Testing Data with dates Report Issued
School District State New definition Elementary and Middle Grades Gr. 3-8 NECAP (Oct 2005) Gr. 2-7 NH-Alt (2004-2005) Aug. 28 2006 1000 AM
District Old definition High School Gr. 10 NHEIAP (May 2005) Gr. 10 NH-Alt (2004-2005) Aug. 28 2006 1000 AM
School State Old definition High School Gr. 10 NHEIAP (May 2006) Gr. 10 NH-Alt (2005-2006) Aug. 28 2006 1000 AM
3
Basic AYP Decisions

• Calculate Other Indicator for the school
• Calculate Participation rate for the school and
  each subgroup
• Calculate Performance for the school and each
  subgroup
• Calculate performance
• If not OK, then check confidence interval
• If still not OK, then check safe harbor

4
Old/New AYP Comparison
Old Definition New Definition
Participation 95 (assigned to school that gave the test) No change (note that NECAP and NH-Alt were administered in different years)
Performance Basic calc. Percent Basic or above Index full credit for levels 3 and 4, partial credit for levels 1 and 2
Performance Assigned to school where student took the test Assigned to school where student was taught the tested material
AMO (Annual Measurable Objectives) Starting points based on May 2002 NHEIAP, three year intervals to 100 Starting points based on October 2005 NECAP, two year intervals to 100
Confidence Interval 99 CI based on Binomial distribution (AMO, size) 99 CI based on average within school variance (AMO, size)
Safe Harbor 10 reduction in percent of students at Novice Equi-percentile proxy this year for 10 in addition, other indicator will be evaluated when safe harbor is calculated
Cell Size Performance 11 per group 11 per group
5
Old/New AYP Comparison
Old Definition New Definition
Other Indicator 90 Attendance (EM Schools) 75 Graduation rate (HS) Same, but now attendance rate based on EOY (End Of Year) district submissions
Withdrew after Oct 2 n/a Excused from testing for affected content areas (must be coded on SRB)
ELL and in the US less than 10 months Counted as participating in reading even if they didnt. Not included in performance for reading or math. No change (must be coded on Student Response Booklet - SRB)
Medical Emergencies n/a Excused from testing for affected content areas (must be state approved)
Enrolled after Oct 2 Not included in any way same
Placement w Private Providers Affects only state report Same (for now)
Out-of-Level Testing n/a Not allowed
6
Grades 3-8 NECAPAchievement Levels

• NECAP Cut points
• Commissioners from VT, RI, and NH adopted cut
  points on January 20
• Proficiency at the student level includes
  performance at achievement levels 3 and 4.
• Achievement Levels
• Level 1 Substantially Below Proficient
• Level 2 Partially Proficient
• Level 3 Proficient
• Level 4 Proficient with Distinction
• Scaled Scores will be reported as a 3-digit
  number where the first digit is the grade level
  and the other part will be a score 00 to 80
  (reported as whole numbers at the student level).
  X40 is the reported cut point for Proficient.
• Grade 3 300-380
• Grade 4 400-480
• Grade 5 500-580
• Grade 6 600-680
• Grade 7 700-780
• Grade 8 800-880

7
New AYP Definition some details

• An External AYP Task Force met to make
  recommendations to the Department. After review
  by the Internal AYP Task Force, we have arrived
  at the following decisions
• Racial categories (no change)
• Defined by BOY (Beginning of Year) submissions
• Other Indicator (no change)
• Attendance Rate (90 or improvement over previous
  year)
• Calculated from EOY submissions
• Participation assigned to testing school
• Performance assigned to teaching school
• Full Academic Year (FAY) for performance
  calculations
• In the future, October 1 of teaching year
• 310 half days of enrollment is the proxy this year

8
New AYP Definition Minimum n(minimum number of
students in group required in order to perform
calculation)

• 95 Participation rate 40 for each group within
  testing grades in school or district
• 75 Graduation rate 40 within school or district
• 90 Attendance rate 40 within school or district
• Includes all grades 1-8 in the school or district
• Performance targets11 for each group within
  testing grades in school or district

9
New AYP Def Index System

• Modified Index System provides partial credit for
  scores below Proficient. A schools index score
  will be the average of all student index points
  assigned to the school.

Proficiency Level Proficiency Level Index Points
Level 1 Substantially below Proficient X00 0
Level 1 Substantially below Proficient 1a 20
Level 1 Substantially below Proficient 1b 40
Level 2 Partially Proficient 2a 60
Level 2 Partially Proficient 2b 80
Level 3 Proficient Level 4 Proficient with Distinction Level 3 Proficient Level 4 Proficient with Distinction 100
10
Scaled Scores for each IndexNECAP
Reading Reading Reading Reading Reading Reading
0 1a 1b 2a 2b 3 and 4
300 301 to 321 322 to 330 331 to 335 336 to 339 340 to 380
400 401 to 421 422 to 430 431 to 435 436 to 439 440 to 480
500 501 to 519 520 to 529 530 to 534 535 to 539 540 to 580
600 601 to 617 618 to 628 629 to 634 635 to 639 640 to 680
700 701 to 717 718 to 728 729 to 734 735 to 739 740 to 780
800 801 to 815 816 to 827 828 to 833 834 to 839 840 to 880
Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics
0 1a 1b 2a 2b 3 and 4
300 301 to 323 324 to 331 332 to 335 336 to 339 340 to 380
400 401 to 421 422 to 430 431 to 435 436 to 439 440 to 480
500 501 to 525 526 to 532 533 to 536 537 to 539 540 to 580
600 601 to 625 626 to 632 633 to 636 637 to 639 640 to 680
700 701 to 727 728 to 733 734 to 736 737 to 739 740 to 780
800 801 to 827 828 to 833 834 to 836 837 to 839 840 to 880
11
Raw Score to Performance IndexConversion table
for NH-Alt
Portfolio Raw Score Points Earned Proficiency Level Performance Index Level Performance Index Points Assigned
47-52 Proficient with Distinction 4 100
38-46 Proficient 3 100
34-37 Partially Proficient 2b 80
29-33 Partially Proficient 2a 60
21-28 Substantially Below Proficient 1b 40
13-20 Substantially Below Proficient 1a 20
0 Un-scoreable (SBP) 0 0

• The lowest score any scoreable portfolio can
  earn is 13 raw score points. 
• A raw score of 0 is only possible if the
  portfolio submitted was judged to be unscoreable
  by two trained and independent scorers.

12
New AYP Def Starting Points

• Starting Points For each content area
  separately, create a new baseline via the 20
  method outlined in NCLB
• Determine index for each school
• Rank schools by each index
• Identify at 20th percentile school
• 20th percentile school the school where 20 of
  the students in the whole list attend that school
  or a school with a lower index.
• That schools index is the starting point
• Starting points Reading 82, Mathematics 76

13
New AYP Def AMOs

• Annual Measurable Objectives
• 2 year intervals, equally spaced to 100

Grades 3-8 Index Index
ELA Mathematics
Starting Point (2005-2006) 82 76
2006 2007 82 76
2007 2008 86 82
2008 2009 86 82
2009 2010 91 88
2010 2011 91 88
2011 2012 95 94
2012 2013 95 94
2013 2014 100 100
14
An Index Example Two schools

• Two schools, both with 180 students tested and
  61.1 of their students scoring proficient or
  better in reading.

READING READING Maple Street CS Maple Street CS Pine Street MS Pine Street MS
Level points of students total of students total
X00 0 30 0 5 0
Level 1a 20 20 400 5 100
Level 1b 40 10 400 10 400
Level 2a 60 5 300 20 1200
Level 2b 80 5 400 30 2400
Level 3 100 60 6000 60 6000
Level 4 100 50 5000 50 5000
Sum Sum 180 12500 180 15100
Index Index 69.4 69.4 83.9 83.9
15
New AYP Def Confidence Intervals
For each subject separately

1. Calculate the within-school variance for each
   school.
2. Calculate the average within-school variance for
   the state.
3. Calculate the standard error for each group
4. Calculate the 99 confidence interval for each
   group

For NH Reading 603.38615 Math 800.72625
The smallest index allowed with the CI is AMO
CIgroup
Where indexstudent index score for the
student mean_indexschool average index score
for the school to which the student
belongs nindexstudent the number of student
index scores for the school of interest nschools
the number of schools, and ngroup number of
students in the group
16
(No Transcript)
17
(No Transcript)
18
New AYP Def Safe Harbor

• Equi-percentile comparison

19
Safe Harbor an example

• For reading, the percent of students scoring
  below 240 on the May 2004 NHEIAP will be compared
  to the percent of students scoring below X37 on
  the October 2005 NECAP.

May 2004 NHEIAP May 2004 NHEIAP May 2004 NHEIAP October 2005 NECAP October 2005 NECAP October 2005 NECAP SH Goal (04 below) x (.90) Safe Harbor? Is (05 below) lt or (SH Goal)?
Tested lt 240 04 below Tested lt X37 05 below SH Goal (04 below) x (.90) Safe Harbor? Is (05 below) lt or (SH Goal)?
Park Ave. ES 200 160 80 500 380 76 80 x .9 72 Is 76 lt 72 No
Gould School 100 70 70 400 250 62.5 70 x .9 63 Is 62.5 lt 63 Yes
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Special Note on Data

• Reports based on student demographic and program
  participation data reported by districts
• EOY (End of Year) files
• BOY (Beginning of Year) files
• SPEDIS, ESL, FR systems
• Known Issues
• SPEDIS /SASID Match
• FAY (Full Academic Year) see next page
• Review AYP Reports carefully
• Check FAY
• Check Educational Disability

24
Schools with Known FAY issuesReports for these
schools and for their districts are based on
assuming all student included these schools EOY
files were enrolled for a full academic year. In
fact, not all students in these schools were
enrolled for a full academic year.
Woodsville Elementary School Memorial Middle
School Hanover Street School Lebanon Junior High
School Mt. Lebanon School Sacred Heart Public
School School Street School Seminary Hill
School McDonough School New Boston Central
School Iber Holmes Gove Middle School Lamprey
River Elementary School
A. Crosby Kennett Middle School Epping Middle
School Henry Wilson Memorial School Edward Fenn
School Carpenter Elementary School Crescent Lake
School Effingham Elementary School Kingswood
Regional Middle School New Durham Elementary
School Ossipee Central School Haverhill
Cooperative Middle School
25
AYP Results and Resources

• www.ed.state.nh.us
• AYP Reports at grades 3-8
• AYP Data Explained
• AYP Look-up Table Explained
• AYP Lookup Table (pdf and Excel formats)
• Csv file of all results
• Understanding the New AYP (ppt)
• Separate HS District Reports (old AYP def)

26
AYP Work for Next year

• Growth
• Measures of Growth at the student level to
  provide support for the Commissioners Follow the
  Child initiative
• NH DoE submitted a letter of interest to USED
  February 17
• Taskforce met on February 22, March 28, April 18,
  May 9, Aug. 22
• NH DoE intends to submit a proposal to USED by
  November 1, 2006
• Out of District Students in Private Placements
• How to include students placed in out-of-district
  private placements?
• Writing and Science
• Neither Writing nor Science is scheduled to be a
  part of AYP at this time