Children Left Behind AYP Schools: Validation of AYP Focusing on Student Progress and the Distributio

1
Children Left Behind AYP Schools?!Validation
of AYP Focusing on Student Progress and the
Distribution of Student Gains

• Kilchan Choi
• Michael Seltzer
• Joan Herman
• Kyo Yamashiro

UCLA Graduate School of Education Information
StudiesNational Center for Research on
Evaluation,Standards, and Student Testing
(CRESST)
2
Research Questions

• Are there schools that meet AYP yet still have
  children who are not making substantial progress?
  i.e., leaving some children behind?
• Are there schools that do not meet AYP yet still
  enable students to make substantial progress?
• Do AYP schools achieve a more equitable
  distribution of student growth? Are students at
  all ability levels making progress in AYP
  schools?
• Are there non-AYP schools that are reducing the
  achievement gap?

3
Sample

• Large, Urban District in WA
• 2,543 Students
• 2 time-point ITBS reading scores (Grade 3 in 2001
  Grade 5 in 2003)
• Standard Errors of Measurement (SE) on ITBS
  reading scores (Bryk, et.al., 1998)
• 73 Schools
• Average students/school 35
• Average qualifying for FRPL 36.2
• Average Minority (African American, Native
  American, or Latino) 68.6

4
AYP vs. Non-AYP schools In WA

• School AYP decision made based on 4th grade
  performance on WA Assessment of Student Learning
  (WASL)
• 52 schools made AYP 21 did not make AYP in
  baseline year (2002), according to WA State Dept
  of Ed
• Our study re-evaluates AYP and non-AYP schools
  with an advanced Hierarchical Modeling technique

5
Latent Variable Hierarchical Model (LVR-HM)

• Level 1 Time Series within Student
• Estimating initial status and gain for each
  student i with standard errors
• Level 2 Student Level
• Gain for student i is modeled as function of his
  or her initial status

6
Different Levels of Initial Status

• Examined Gains for 3 Performance Subgroups within
  Each School
• Defined by initial status
• Hi Performers 2 SDs above the estimated school
  mean initial status
• Mean School mean initial status
• Low Performers 2 SDs below the estimated school
  mean initial status
• Possible to define performance subgroups based on
  a set of absolute values (e.g., 10 pts
  above, mean, 10 pts below)

7
Estimating Expected Gains for Different Levels of
Initial Status

• Model-based estimation, not separate group
  analysis
• Point estimate of gain its 95 confidence
  interval (statistical inferences)
• Possible to estimate expected gains after
  controlling for factors that lie beyond schools
  control (e.g., student SES, school compositional
  factors)

8
Expected mean gain in ITBS reading scores for
AYP schools

• Only 9 of 52 AYP schools have 95 interval above
  the district avg.
• 1 AYP schools 95 interval includes 0
• 2 Non-AYP schools have 95 interval above
  district avg.

9
Expected mean gain in ITBS reading scores for
non-AYP schools
10
Expected gain for low-performing students (AYP
schools)

• 3 AYP schools 95 interval includes 0 (low
  performers make
• no gains)
• 7 AYP schools have gains of lt10
• 2 Non-AYP schools have gains for low performers
  gt20

11
Expected gain for low-performing students
(non-AYP schools)
12
Expected gain for high-performing students (AYP
schools)

• Many AYP and Non-AYP experience large gains for
  high
• performers
• 3 AYP and 3 Non-AYP schools 95 interval
  includes 0
• (high performers make no gain)

13
Expected gain for high-performing students
(non-AYP schools)
14
Distribution of Gains Within A School

• Type I Substantial gain across all performance
  subgroups (e.g., no child left behind ex AYP
  school 8, non-AYP school 26)
• Type II No adequate gain for high performers
  substantial gain for low performers (ex AYP
  schools 35 43, non-AYP school 27)
• Type III No adequate gain for low performers
  substantial gain for high performers (ex AYP
  schools 14 28)

15
Distribution of student gain for 3 AYP schools
16
Distribution of student gain for 2 AYP schools
17
Distribution of student gain for 2 Non-AYP schools
18
Comparing Features AYP the CRESST Approach
19
Mean AYP and Subgroup AYP
20
Conclusions

• Analyses using our alternative approach
• More informative picture of growth using
  individual, longitudinal student gains
• More complete picture of how student growth is
  distributed within a school
• Stimulate discussion among teachers and
  administrator to identify students in need
  earlier (Seltzer, Choi Thum, 2003)
• Encourage educators to think about achievement
  levels rather than (or in addition to) current
  subgroup categories - may be more productive and
  actionable