Revised Achievement Profiles

1
Revised Achievement Profiles

• Research and Policy Division
• Arizona Department of Education
• September 2001

2
Existing Achievement Profiles as Outlined in
Proposition 301

• 3 Components
• AIMS
• Measure of Academic Progress (MAP)
• Dropout Rate
• 2 School Designations
• Underperforming
• Failing

3
Existing Achievement Profiles as Outlined in
Proposition 301

• 90 of Students meet the Standards on AIMS or a
  greater percentage than the previous year
• 90 of Students make One Year Growth on MAP or a
  greater percentage than previous year
• 6 Dropout rate or lower than the previous year

4
Flaws with Existing Achievement Profiles

• Relies on one model for both elementary and high
  school
• Cut-scores are arbitrary
• Potentially mislabels schools and provides only a
  limited view of school performance
• Vulnerable to normal fluctuations in student
  groups (cohort effects)
• Conjunctive, all or nothing approach
• Only goal for schools is to avoid failing label

5
Revised Achievement Profiles
6
Revised Achievement Profiles Working Parameters

• Existing Achievement Profiles will become
  effective in October 2001 unless changes are made
  this legislative session
• The purpose is to develop a fair profile for all
  schools
• The results will be released publicly
• One of the classifications will be a failing
  designation or equivalent
• Achievement Profiles must include MAP, AIMS and
  Dropout Rates

7
Revised Achievement Profiles
A separate achievement system for elementary and
high schools
8
Elementary Model

• AIMS
• benchmark to the Arizona Academic Standards
• MAP
• provides longitudinal view of individual student
  progress across all grade levels

9
Elementary Model
AIMS
AIMS
AIMS
3-Year
3-Year
3-Year
Average
Average
Average
1
2
3
4
5
6
7
8
Grade
MAP
(SAT9)

• AIMS
• 3-Year rolling average mitigates cohort effects
• Requires growth by all student groups
• Captures incremental, meaningful progress beyond
  existing performance levels
• Analyzes data by student mobility to confirm
  findings

10
3-Year Rolling Average on AIMS

• Minimizes cohort effects
• Increases reliability
• Identifies trends in school performance

11
Growth for ALL Student Groups AIMS
Existing Achievement Profiles focus on only one
group of students
Revised Achievement Profiles require the
advancement of students at all performance levels
12
Closing Achievement Gaps between Student Groups
Falls Far Below the Standard
This group is composed of various student
sub-groups based on

• Race
• Economic status
• Gender

Improving the achievement of the entire
performance level Closing achievement gaps
between student sub-groups
13
Capturing School Progress on AIMS

• Subdivides existing performance levels to capture
  meaningful progress within and between
  performance levels
• Determines whether schools are improving at
  teaching 3rd, 5th and 8th grade
• Difficult standard yields incremental progress in
  instructionand student achievement

Elementary
14
Elementary Model Student Mobility

• Analyze data on student mobility to confirm
  results
• Reveal any extreme differences in performance

15
Elementary Model
AIMS
AIMS
AIMS
3-Year
3-Year
3-Year
Average
Average
Average
1
2
3
4
5
6
7
8
Grade
MAP
(SAT9)

• MAP
• Holds all grades in a school accountable for
  achievement
• Provides longitudinal view of performance
• Takes into account student mobility

16
Growth for ALL Student Groups MAP

• A student will achieve Adequate Yearly Progress
  on MAP if the student remains in the same stanine
  or progresses to a higher stanine from one grade
  to the next
• Stanines will be collapsed in order to report
  results by achievement level
• Insightful analysis that schools can perform
  themselves

17
High School Model

• AIMS
• Graduation Rate
• Dropout Rate
• 9th Grade Indicator

18
High School Model

• Establishes supercohort based on entry into 9th
  grade
• Schools are held responsible for all students
  regardless of when they enroll in the school

19
The Super Cohort
9th Grade
Multiple sub-cohorts enter high school at the
same time.
Elementary
20
2-Year Rolling Average on AIMS

• Minimizes cohort effects
• Increases reliability
• Identifies trends in school performance

Average
(CY-1) CY
2
CY Current Year
Elementary
21
High School Model Graduation Rate

• 4-Year longitudinal measure of how many students
  graduate from high school

Potential changes to Graduation Rate include a
5-year graduation rate
22
High School Model Dropout Rate

• Proportion of students who dropped out during a
  12-month period
• Annual snapshot that detects more immediate
  changes in school performance than the graduation
  rate

23
9th Grade Indicator

• MAP and AIMS are not available for 9th grade
• Local CRT scores will be reported
• Focuses instruction on Arizona Academic Standards
• Local CRT will be public information only and not
  part of the criteria for school classification

24
Overall Achievement Profile Features

• Data-driven cut scores
• Compensatory Model
• Modified Gap-Reduction Model
• Multiple school classifications
• Multi-level reporting format
• Parallel system for unique schools
• Comprehensive state system

25
Data-driven Cut Scores

• Establishes accurate measures to ensure every
  school is evaluated fairly
• Lets data, not politics, drive decisions
• Potential legislative obstacle

26
No Two Schools Are Alike
27
Data-driven Methodology Yields Accurate School
Classifications for Low Performing Schools ...
FOCUS
Improving
Underperforming/ Failing
28
and High Performing Schools
FOCUS
Maintaining
Improving
29
Compensatory Model

• Compensatory model allows the flexibility
• to consider the full picture of
• school performance.

30
Pitfalls of a TraditionalGap-Reduction Model

• Arbitrary deadlines and progress rates
• Attractive to policymakers
• Absolute standard and progress rates areeasy to
  calculate
• Inconsistent expectations for each school

EXAMPLE
31
Modified Gap-Reduction Model

• Data-driven deadlines and progress rates
• Challenging expectations for all schools
• School focus shifts from a deadline tostudent
  progress

EXAMPLE
32
Multiple School Classifications

• A spectrum of school classifications allows for
  the identification of diverse school outcomes.

• Improving
• Maintaining
• Underperforming
• Failing

Each school will be given a single school
classification.
33
Multi-level Reporting Format

• Multiple levels of reporting allows information
  and level of detail to be tailored to various
  audiences.
• Each report will build on information from other
  reports.

School Reports District Reports Technical
Reports Media level Reports
34
Parallel System for Unique Schools
A parallel achievement profile for unique schools
(i.e., accommodation, extremely small) avoids the
pitfalls of a one-size-fits-all approach.
35
Comprehensive State System
One comprehensive system to satisfyall state and
federal statute accountability requirements
36
Revised Achievement Profiles

• Create fair and accurate school classifications
• Capture broad spectrum of school performance
• Provide meaningful information to all
  stakeholders
• Promote the achievement of all students

37
Under Consideration...

• Site visit to confirm any Underperforming or
  Failing classification prior to public label
• Classify as Underperforming only as many
  schools as the state has the resources to assist