The Intended and Unintended Consequences of the Use of Alternate Assessments

1
The Intended and Unintended Consequences of the
Use of Alternate Assessments

• Claudia Flower, Lynn Ahlgrim-Delzell, and Diane
  Browder
• University of North Carolina at Charlotte
• Meagan Karvonen
• Western Carolina University

• http//education.uncc.edu/cpflower

2
Access to the General Education Curriculum

• Individuals with Disabilities Education Act
  (IDEA) 1997 2004
• NCLB
• alternate assessments for students with
  disabilities unable to participate in statewide
  assessments (1)

3
Students with Significant Cognitive Disabilities

• Traditionally the content focus was nonacademic
• Excluded
• Preschool (developmental)
• Functional

4
Inclusion in Accountability

• Positive consequences
• access to the general curriculum
• improved instructional methods
• additional resources
• higher expectations for student learning

5
Consequences of Test Use

• Standards for Educational and Psychological
  Testing
• It is the responsibility of those who mandate
  the use of tests to monitor their impact and to
  identify and minimize potential negative
  consequences. Consequences resulting from the
  uses of the test, both intended and unintended,
  should also be examined by the test user (p.
  145).

6
Intended Consequences

• Unlimited number of intended unintended
  consequences
• Gathering evidence of the intended effects is
  first necessary (Lane, Parke, Stone, 1998).

7
From the Literature

• Results of accountability systems are intended to
  impact the following
• (a) implemented curriculum
• (b) the instructional content and strategies
• (c) the content and format of classroom
  assessments
• (d) student, teacher and administrator motivation
  and effort

8
From the Literature (cont)

• (e) learning experienced by all students
• (f) the nature of professional development
  support
• (g) teacher participation in administration,
  development, and scoring of assessments
• (h) student, teacher, administrator, and public
  awareness and beliefs about the assessment,
  criteria for judging performance, and the use of
  the assessment results and
• (i) the use and nature of test preparation
  materials

9
From the Literature (cont)

• see Cizek, 2001 Frederiksen Collins, 1989
  Koretz, Barron, Mitchell Stecher, 1996 Koretz,
  Stecher, Klein, McCaffrey, 1994 Linn, 1993
  Linn, Baker Dunbar, 2001 Messick, 1992 and
  many others

10
Inclusion of Students with Significant Cognitive
Disabilities

• What do we know?
• Several studies out of Kentucky Massachusetts
  (Kleinert Colleagues Zatta et al.) and others
• Both benefits challenges

11
Benefits

• inclusion in state accountability system raised
  expectations for students with significant
  disabilities
• increased focus on the general education
  curriculum
• increase in professional development activities

12
Challenges

• competed with teaching time and meeting
  individual student needs
• Whats important?
• increased the paperwork burden

13
Purpose

• Previous studies used teachers as the data source
• This study
• Perspective of special education administrators
• Hypothesized potential intended unintended
  consequences

14
Survey Research Method

• Procedures
• Post card to 4500 special education
  administrators
• Responded via the internet
• Participants
• 708 surveys from 49 states and the District of
  Columbia
• Most district-level administrators (65)

15
Instrument (28 Items)

• Six domains served to guide in item development
• (a) access to the general curriculum Access
• (b) improved instruction Instruct
• (c) increase student expectations Expect
• (d) increased educational resources and training
  Resources,
• (e) support from principal Principal, and
• (f) increased workload and stress Workload.

16
Results

• Top Rated Items
• More paperwork
• IEP reflects students needs vs. AA
• Increased teacher stress level
• Increased amt of required training

17
Lowest Rated Items

• Principals have thorough understanding of AA
  process
• Raised general education teachers expectations
• Changed views about SWSCD
• Increased principals involvement in exceptional
  children programs

18
Exploratory Factor Analysis (EFA)

• Split the sample in half
• Principal factors extraction
• Six factors were extracted with eigenvalues
  greater than 1.0
• Skee plot leveled off
• Used loadings of .4 for interpretation

19
EFA Results

• (a) instruction,
• (b) principal,
• (c) training,
• (d) workload,
• (e) access, and
• (f) support

20
Confirmatory Factor Analysis (CFA)

• Using the other half of the sample
• Six factor model was hypothesized with items
  identified in the EFA as indictors of the factors
  
• LISREL

21
Results of CFA

• Marginal support
• ?2 (df237, N356) 401.06, plt.000001
• root mean square error of approximation (RMSEA)
  .05, and
• comparative fix index (CFI) .93

22
Post hoc Model Modifications

• basis of the Lagrange multiplier test and
  theoretical relevance
• error variances between the observable variables
  were allowed to correlate
• ?2 (df233, N356) 320.56, p.00012,
• RMSEA.037,
• comparative fix index (CFI) .96

23
Correlations between Factors
24
Summary

• Regardless of the accuracy and consistency of
  identifying proficient students and schools, it
  is meaningless if nothing happens to improve the
  work of schools (Marion Gong, 2003).

25
Continuous Improvement

• Alternate assessments are relatively new
• Using results to improve educational systems for
  students
• Starting point for looking at consequences of
  alternate assessments