The ABCs of Public Education

1
The ABCs of Public Education

• Update for the 2005-2006 School Year
• The New Growth Model

2
Notes Regarding EOG and EOC Tests

• The EOG and EOC tests are aligned with the
  Standard Course of Study, and items are written
  by NC teachers. However they are multiple-choice,
  relatively short, and test only part of the
  curriculum, all of which lowers reliability for
  individual student scores.

3
Notes Regarding EOG and EOC Tests

• Multiple forms of each test are used in each
  classroom and forms are statistically comparable
  in difficulty level according to statewide
  field-testing. Multiple forms make group scores
  much more reliable than individual student scores.

4
The Three Major Components

• ABCs Performance Composite
• AYP Proficiency
• ABCs Growth (the new formula)
• Remember All numbers and labels begin with
  individual student scale scores!

5
Conversion of Scale Scores to Achievement Levels
Example Grade 5 Reading

• Scale Scores
• 259-277
• 247-258
• 239-246
• 228-238

• Achievement Levels
• Level IV
• Level III
• Level II
• Level I

6
Conversion of Scale Scores to Achievement Levels
Example Grade 5 Reading

• Scale Scores
• 259-277
• 247-258
• 239-246
• 228-238

• Achievement Levels
• Level IV
• Level III
• Level II
• Level I

Proficiency Demarcation
7
Review of Two ABCs Components

• The two components that are not changing
• ABCs Performance Composite
• AYP Proficiency

8
Performance Composite

• Elementary Schools
• EOG regular administration
• EOG with accommodations
• Extend2
• NCCLAS
• NCAAP
• Writing

• Middle Schools
• Same as the elementary
• school with the following
• additions
• Computer Skills
• EOCs regular administration
• EOCs with accommodations

9
Performance Composite Calculation

• Elementary School Example
• Number of Level III IV Scores for Reading
  Number of Level III IV Scores for Math Number
  of Level III IV Scores for Writing
• Number of Reading Scores Number of Math Scores
  Number of Writing Scores

10
Performance Composite Calculation

• Middle School Example
• Number of Level III IV Scores in Reading
  Number of Level III IV Scores in Math Number
  of Level III IV Scores in Writing Number of
  III IV scores in Algebra I and Geometry
  Number of Students Passing Computer Skills
• Number of Reading Scores Number of Math Scores
  Number of Writing Scores Number of Algebra I
  Scores and Geometry Scores Number of 8th
  graders in membership on first day of EOG testing
  (applied to Computer Skills)

11
Performance Composite

• High School
• EOCs (Algebra I, Algebra II, Biology, Chemistry,
  Civics and Economics, English I, Geometry,
  Physics, Physical Science, and U.S. History)
• EOCs with accommodations
• NCCLAS

12
Performance Composite Calculation

• High School Example
• Number of Level III IV Scores for Ten EOCs
• Number of Scores for Ten EOCs

13
Adequately Yearly Progress (AYP) The Ten
Subgroups

• All students
• American Indian
• Asian
• African American
• Hispanic
• Multi-racial
• White

• Economically Disadvantaged
• Limited English Proficient
• Students with Disabilities

14
Adequately Yearly Progress (AYP) Percent Tested
for Ten Subgroups

• In addition to the subgroup proficiency
  requirements, 95 of each subgroup must be tested.

15
Subgroup Requirements

• Test scores, including Extend2 and NCCLAS
• 140 days in membership (70 days for block
  schedule high schools)
• Minimum of 40 students per subgroup

16
Target Proficiencies for AYP (Adequate Yearly
Progress)
Math
Math
High School Reading
Gr. 3- 8 Reading
Year
54.9
52.0
74.6
68.9
2003-04
70.8
35.4
81.0
2004-05
76.7
80.5
56.9
87.3
84.4
2007-08
90.2
78.4
93.7
92.2
2010-11
100
100
100
100
2013-14
17
OAI Other Academic Indicator

• Attendance (for schools without grade 12)
• Graduation Rate (for schools with grade 12)

18
41 Possible Targets

• 10 subgroups testing 95 for reading
• 10 subgroups testing 95 for mathematics
• 10 subgroups for reading proficiency
• 10 subgroups for mathematics proficiency
• OAI (either attendance or graduation rate

19
Safeguards

• Confidence Interval
• Safe Harbor
• Title I Targeted Assistance Schools

20
Safeguards for Testing Error
21
Safe Harbor

• Applied to any subgroup not meeting proficiency
  target
• 10 reduction in non-proficient students from the
  previous year
• Additionally, either attendance or
• graduation rate must be met for the subgroup.

22
Safe Harbor Example

• 2006 Subgroup proficiency54.6
• 2005 Proficiency (same subgroup)49.0
• The 2005 NON-proficiency was 100-49.051.0
• 51.0 divided by 105.1
• 2006 Safe Harbor Goal
• 49.0 (previous proficiency) 5.154.1
• 2006 proficiency is higher than needed for Safe
  Harbor
• SAFE HARBOR IS MET!

23
Title I Targeted Assistance

• This only applies to schools with Title I
  Targeted Assistance programs.
• The re-analysis uses only the scores of students
  served in the Title I program.
• If served students do not meet AYP, the final
  analysis uses scores of those eligible to be
  served.

24
Review of Proficiency Components

• Performance Composite looks at the percentage of
  the test scores at or above Achievement Level
  III.
• AYP looks at whether the students in the school
  as a whole and in each identified subgroup met
  the performance standards set by the state
  following federal guidelines.

25
Problems with the OLD Model

• Fixed formulas based on 1993-96 data
• New editions of tests had to be linked to the old
  in order to use the old formulas
• Impact of new curricula and new tests could not
  be consistently predicted

26
Problems with the OLD Model (continued)

• Statewide, as a cohort of students moved from
  grade to grade, a zig-zag pattern of success was
  seen

27
(No Transcript)
28
Problems with the OLD Model (continued)

• The use of school averages instead of individual
  scores limited disaggregation and other analyses
• NOTE THESE WERE PROBLEMS WITH THE ACCOUNTABILITY
  MODEL-NOT THE TEST SCORES!

29
The New Growth Component

30
The New Growth Component

• The new growth component uses a different way to
  measure success across levels and courses.
• A student is expected to do at least as well
  this year as she/he has done in the past,
  compared to other NC students who took the test
  in the standard-setting year.

31
Normal Curve
32
Normal Curve and Percentiles
33
Scale Scores and Standard Scale
34
Z-Score Example (Gr. 5 Reading)

• State average256.9, SD8.03
• Sample Score258
• Z 258 256.9 / 8.03
• Z 1.1 / 8.03
• Z 0.14
• In this example, the z-score is slightly above
  average

35
The New Growth Component

• Growth comparison of current performance to
  past performance, looking at a students position
  compared to statewide performance
• Determined by calculations that use
• End-of-Grade developmental scale scores
• End-of-Course scale scores
• Other components at the high school level

36
The New Growth Component

• Predicts expected achievement based on past
  achievement at the individual student level
• Separates reading from math scores,
• Adjusts for differences in relative difficulty
• Works with new editions of tests

37
Regression to the Mean

• Students with extreme scores, either high or low,
  will tend to score closer to the average on the
  next test.
• The state used an adjustment in the old model for
  regression to the mean, and will also use it in
  the new model.

38
Regression to the Mean

• Students with scores below the state average will
  have expected scores that are higher, or closer
  to average.
• -2.30 .92 (regression factor)-2.12

39
(No Transcript)
40
Regression to the Mean

• Students with scores above the state average will
  have expected scores that are lower, or closer
  to average.
• 0.40 .92 0.36.

41
(No Transcript)
42
Regression to the Mean

• A student score with a score at the state average
  of 0 will not change as a result of regression.
• 0. 0 .92 0.

43
(No Transcript)
44
The New Growth Component Academic Change

• Z-scores are determined the first year a new test
  is given, based upon statewide results.
• The statewide SD (standard deviation) and mean
  scale score from the first year are used for the
  life of the test to calculate a C-score
  (change-scale score).

45
The New Growth Component Academic Change
46
The New Growth Component Academic Change
47
Academic Change

• Academic Change (AC) for EOG. For each student,
  AC will be calculated by
• Averaging c-scores for the past two years (if the
  data are available, otherwise just the most
  recent score will be used)
• Adjusting the average for regression to the
  mean (using a larger regression factor if only
  one year of data is available), and
• Comparing the expected c score to the
  students actual c-score.

48
Academic Change Grade 5 Reading Example

• Grade Scale State Diff Standard
  Actual
• level score mean deviation
  c-score
• Reading 5 268.0 256.9 11.1 8.03
  1.382
• Reading 4 263.0 252.3 10.7 8.68
  1.233
• Reading 3 157.0 146.9 10.1 9.29
  1.087

49
Academic Change Grade 5 Reading Example/PREDICT

• Grade Pretest Average
  Expected
• level c-scores (ATPA) Coeff.
  C-score
• Reading 4 1.233
• Reading 3 1.087 1.16 0.92
  1.067
• ATPAAverage of Two Previous Assessments
• CoefRegression coefficient

50
Academic Change Grade 5 Reading Example/COMPARE

• Actual Expected Difference
• 1.382 1.067 0.315

51
School Expected Growth Average All Scores

• To determine Expected Growth for a school, all of
  the AC scores are averaged.
• If the average is 0.00 or greater, Expected
  Growth is met.

52
School Expected Growth Average All Scores

• AC will be calculated only for students who have
  the necessary test scores, and who have been in
  membership 140 days (or 70 days for block high
  schools)
• This is a change for high schools. The EOC AC
  components will only be calculated for students
  who have been in membership for 140/70 days

53
Academic Change Differences for High School

• For high schools, the model uses EOC scores for
  the current year, with previous EOC or EOG scores
  as the pre-test(s).
• In addition, as in previous years, growth uses
  changes in drop out rate, competency pass rate,
  and percentages of students meeting the College
  University Pre/College Tech Prep standards.

54
Academic Change The High School ModelAlgebra I
Example

• .
• Scale
  C-scale
• Score
  Score
• Grade 8 EOG Math 280
  0.913
• Algebra I EOC 72
  1.171
• Expected 0.913 0.82 0.749 (after
  regression)
• AC 1.171 0.749 0.422.

55
Academic Change The High School ModelPrevious
Assessments

• Biology- EOG Reading Grade 8 and English I (taken
  the previous semester or before), if available
• Physical Science- EOG Math Grade 8
• Physics- Chemistry and Geometry
• Chemistry- Biology
• Algebra II- Algebra I
• Algebra I- EOG Math Grade 8
• Geometry- Algebra I and EOG Math Grade 8 (if
  available)
• English I- EOG Reading Grade 8
• Underlined assessments are necessary for
  inclusion in the model

56
High Growth

• The school must make expected growth
• 60 of students must make positive AC, (0 or
  above), a ratio of 1.5
• 6/4 3/2 1.5

57
Implications

• Fewer schools will achieve expected growth
• Fewer schools will achieve high growth
• Better comparability will be achieved across
  grade levels and EOC subjects

58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
What does this mean for my school?

• My school may not make Expected or High Growth
  this year even if we made it last year.
• The AC of every student counts so we need to be
  addressing the needs of every student.
• The new formula will give a better comparison of
  AC across grade levels and subjects

62
Acknowledgements

• This presentation and the accompanying script
  (Notes) was developed by Winston-Salem/Forsyth
  County Schools, using information provided by the
  NC Department of Public Instruction,
  Accountability Services. Wake County Schools
  provided significant contributions to the
  contents of the presentation.
• Displays were also contributed by Gaston County
  Schools.
• The following school systems provided invaluable
  feedback and improvements to the displays and
  text
• Cumberland County, Durham County, Gaston County,
  Johnston County, Wake County, Wilson County