The State of States NCLB Accountability Plans: June 2006

1
The State of States NCLB Accountability
PlansJune 2006
William J. Erpenbach, WJE Consulting, Ltd. Ellen
Forte, edCount, LLC 36th Annual National
Conference on Large-Scale Assessment June 26,
2006San Francisco, CA
2
What happened last year with respect to
accountability workbook amendments?

• You have our Year 3 paper summarizing all of
  this.
• But, lets take a quick look back before we
  looking at what happening now.

3
Headlines from the 2004-05 Amendment Season

• Escalating calls to fix the law.
• Conflicting evaluation resultsmany different
  groups weighing in creating a mixed picturethe
  law is working vs. it isnt.
• Funding challengesStates claiming the law is
  under-funded and LEAs similarly arguing more
  fiscal support needed to help low-achieving
  schools a few law suits.
• Local challenges to States NCLB implementation.
• SEA capacity issuesability to find funds for
  sufficient staffing and to provide technical
  assistance/support to LEAs and schools.
• Newly-appointed ED Secretary Spellings promises
  chiefs increased flexibility and a common sense
  approach to administering NCLB.
• Renewed interest in growth models.

4
2004-05 Accountability Workbook Amendment Outcomes

• October 2005 PaperStatewide Educational
  Accountability Under NCLB A Report of 2005
  Amendments to State Plans
• Most common requests/approvals
• Modify requirements for identifying LEAs for
  improvement
• Modify graduation rate
• Modify method of calculating Participation Rate
• Add confidence intervals and indexing to AYP
  determinations
• Change minimum n including differentiated minimum
  ns for some subgroups
• Modifications to safe harbor review
  criteria/processes
• One-year use of modified achievement standards
  for SWDs (2 proxy)
• Surprise decisions
• Switch order of school choice and supplemental
  educational services
• Even larger minimum ns including more
  proportional minimums involving large upper
  limits
• Inclusion of SWDs and LEP students taking more
  than 4 years in the calculation of Graduation
  Rate
• Use of up to three years data in making safe
  harbor determinations

5
So, whats happening now?

• The game has changed (again).
• Amendments approved in previous years for other
  States are being denied this year.
• Lets take a closer look.

6
Headlines from the 2005-06 Amendment Season

• Hurricanes Katrina and Rita dislocate thousands
  of students to other States.
• Controversy about the rigor of States
  achievement standards again surfaces with new
  round of NAEP scores.
• NCLB Standards and Assessments Peer Reviews
  continue.
• Growth Model Pilot Program launchedconcept of
  universal proficiency introduced.
• NPRM released for 2 modified achievement
  standards.
• Presidents statement on American Competitive
  Initiative includes reference to requiring
  testing in two additional high school grades and
  adding science to AYP requirements.
• Assistant Secretary Johnsons March 7, 2006,
  letter announces deadline for 2006 accountability
  amendments and includes transition requirements
  and clarification of parameters related to
  identifying LEAs for improvement.
• Appointment of the Commission on No Child Left
  Behind.
• AP study contends 1.9M students excluded in AYP
  determinations.
• Accountability workbook approval Take-Backs
  loom.

7
Hurricanes Katrina and Rita

• States with large hurricane-refugee student
  populations allowed to create separate subgroup
  of these students for AYP performance does not
  count at all.
• So farAlabama, Arkansas, Georgia, and Tennessee

8
NCLB Standards and Assessments Peer Reviews

• All states have been reviewed.
• South Carolina, Delaware, and Tennessee have been
  granted full approval.
• Five additional states expected to get full
  approval by June 30.
• States in the Approval Pending category may lose
  10 of administrative funds for 2006-07.
• States in the Unapproved category may lose 25 of
  administrative funds for 2006-07.
• Big remaining issues standard setting,
  alignment, inclusion (especially, alternate
  assessments for SWDs).

9
Growth Models2006 Applications

• Deferred to 2006-07 Hawaii, Maryland, Nevada,
  New Hampshire, Ohio, Pennsylvania, and South
  Dakota
• Rejected Colorado, Indiana, Iowa, South
  Carolina, and Utah
• Peer Reviewed Alaska, Arkansas, Arizona,
  Delaware, Florida, North Carolina, Oregon, and
  Tennessee

10
Growth ModelsReview Outcomes and Developments

• Tennessee and North Carolina approved for
  2005-06 other six States sent letters with
  reasons for non-approval and invitation to
  submit a revised proposal by September 15 to be
  peer-reviewed in mid-October.
• North Carolina asked to modify several aspects of
  its model and awaiting result of its standards
  and assessments review.
• ED apparently negotiating with a third State.
• May 17Peer Review Panel issues paper on
  cross-cutting themes suggesting additional
  criteria for growth models.
• Peer Review processes, criteria, and extent to
  which panels recommendations followed not
  transparent.
• No one seems to be raising same subject, same
  subgroup issue in connection with growth models.

11
Growth ModelsNext Steps

• According to letter to non-approved States from
  Assistant Secretary Johnson (mid-May 2006)
• Non-approved States (5 or 6) may submit revised
  proposals by September 15 for a second peer
  review in mid-October. These will have priority
  over new submittals in the review/approval
  process.
• These States and others may submit new proposals
  by November 1, 2006. States submitting proposals
  this year but deferring review unclear will
  likely be considered as new submittals under
  this plan.
• ED still intends to limit to ten the number of
  approved plans through the pilot project.

12
Appointment of the Commission on No Child Left
Behind

• Private, bi-partisan panel formed to study the
  federal school accountability law and recommend
  to Congress changes for the laws 2007
  reauthorization.
• Tommy Thompson, former U. S. HHS Secretary and
  Wisconsin Governor, and Roy Barnes, former
  Georgia Governor, co-chairs. Thirteen additional
  members comprise the full panel.
• On other fronts, for example, CCSSO has
  established its own panel under the leadership of
  Libby Burmaster, WIs chief in January, a
  coalition of school, civil rights and
  child-advocacy groups submitted a list of 14
  recommendations for changing NCLB to
  Congressional staffers.

13
Accountability Workbook Approval Take-Backs Loom

• Repeal of larger minimum ns for subgroups
  included in NPRM on 2 options.
• One State required to drop either SEM or CI for
  proficiency determinations.
• Another State loses battle to omit additional
  grades (in 3-8) for AYP in 2005-06.
• In general, ED not approving increases in minimum
  n, confidence intervals, or use of indexing for
  2005-06.
• Secretarys June 13, 2006, letter to
  Representative McKeon signals fall 2006
  conference for States to review/justify minimum
  ns and confidence intervals used in AYP
  determinations.

14
Secretarys June 13, 2006, Letter to Rep. McKeon

• The Department, through its Assessment and
  Comprehensive Assistance Centerwill invite
  States to participate in a national technical
  assistance conference to be held this fall to
  help States improve their systems for ensuring
  the validity and reliability of their
  accountability decisions. With full testing under
  NCLB now underway, we will work with States to
  acquire new impact data on school and student
  inclusion rates and discuss with them a process
  for justifying how their specific n-size is
  necessary for valid and reliable results.
  Additionally, this forum will look at how States
  use of statistical tools affects the flow of
  interventions to students who most need academic
  assistance. (p.4)

15
2006-06 Accountability Workbook Amendment Requests

• Over 40 States submitted accountability workbook
  amendments by April 1, 2006.
• Number of requests meet or exceed those submitted
  last year a bit of a surprise.
• Virtually every State requesting an increase in
  minimum n or the application of a confidence
  interval has been denied (via a phone call).
  Pretty much the same for indexing in proficiency
  determinations.
• It appears that minor amendments are being
  approvedvery few written responses yet.
• No response (written or verbal) generally means
  not approved.

16
2005-06 Amendment RequestsSelected Examples

• Replace high school assessments with SAT or ACT
  (actually a standards and assessment issue).
• Delay use of results from additional grades in
  3-8 for use in AYP for one to two years
  (Wellstone amendment).
• Modify how LEAs are identified for improvement
• Base AYP on missing same subject by same
  subgroup.
• Modify FAY and graduation rate definitions.
• Modify or reset AMOs and IGs (typically linked to
  bringing on line new or additional assessments).
• Modify or add indexing to proficiency
  determinations
• Modify safe harbor calculations.
• Continue use of 2 proxy for SWDs against
  modified achievement standards.
• Modify manner in which LEP students are included
  in State assessments.

17
So, whats next?

• Look for our Year 4 paper in late summer/early
  fall.
• Stayed tuned. Decisions are forthcoming quickly
  on standards and assessments.
• Phone calls appear to be into many States from ED
  regarding accountability workbook decisions.
• Try to get reasons for nos in writing
  including the legal basis for same.
• Dont assume that because something in your
  accountability workbook is approved that it
  will stay that way unless you seek to amend it at
  some future point!
• Share information.