Equity and Assessment The Opportunity to Learn Challenge

1
Equity and AssessmentThe Opportunity to Learn
Challenge
Math/Science Partnerships Workshop Assessment Of
Student Learning
February 1 - 3, 2004
William T. Trent University of Illinois at
Urbana-Champaign
2
Introduction
The focus of this workshop on assessment is
timely and occurs at a critical point as the
results from the NCLB required assessments become
more clear. Combined with the results from
graduation tests and state standards-based
assessments, the resulting discussions have
necessitated a renewed focus on the role of race,
socioeconomic status, ethnicity and school
factors in shaping academic preparation, access
and performance. The persisting inequalities
require a renewed level of attention focused on
issues of equity.
3
Equity and Assessment

• The main point addressed in this presentation
  is straight forward.
• The constraints on Opportunities to learn for
  students of color and poor students are
  substantial.
• As a consequence, our assessment tools and
  practices are called into question, especially
  when they are the basis for high stakes decisions
  about students placements in the school setting
  .

4
Equity and Assessment

• A point of focus
• A key assumption in the conceptualization and
  design of most assessments is that all students,
  by and large, have received a substantial dose of
  the treatmentinstruction and learning
  experiences. This is a necessary assumption and
  one with we can all agree.
• Addressing the equity question however tells us
  immediately that this assumption is untenable for
  a substantial proportion of our students and for
  a number of reasons.

5
Critical Points Along the Educational Pipeline
6
Equity and Assessment

• Several factors influence the opportunity to
  learn and thereby impact assessment
• Segregated schooling
• Disparate school quality
• Teacher expectations
• Parent resources
• Community resources

7
In 1978, a divided court ruled to preserve the
use of race in college admissions. In that
decision, Justice Harry Blackmun wrote a more
frequently quoted phrase but one which is
challenging to operationalize"In Order to Get
Beyond Race, We Must First Take Account of Race.
There Is No Other Way."Justice Harry Blackmun in
BAKKE
Access and equity issues remain unresolved

• Fifty (50) years ago Brown was handed down by
  the U.S. Supreme Court

8
Equity and Assessment

• Segregated schooling is on the increase
• Hyper segregated-- high minority high poverty
  -- schools are especially harmful to student
  learning and attainment
• African American and Latino/a students are often
  in schools where the quality of courses available
  AP courses for example -- are limited
• African American and Latino/a students are often
  in schools where they are overrepresented in
  Special Education, Suspensions and Expulsions but
  underrepresented in Gifted Education
• Race and ethnicity, gender and poverty status at
  the student level, along with school level
  factors like school size, racial composition and
  teacher credentials and expectations are
  consequential for educational outcomes.

9
Equity and Assessment

• There are important correlates of school racial
  composition that implicate equity.
• For example
• High SES and race concentrations
• Differential College-going rates
• Differential Teacher expectations
• Differential access to quality teachers
• Differential access to a high quality curriculum

10
Race, Ethnic and Poverty Concentrations
11
Table 2. College Going Rates by High School
Racial Composition
12
Missouri V. JenkinsTeacher Efficacy - tables
Mean 1994 Standardized Math Test Scores by
Teacher Efficacy and Student Race
13
Missouri V. JenkinsTeacher Efficacy - tables
Mean 1994 Standardized Reading Test Scores by
Teacher Efficacy and Student Race
 
 
14
(No Transcript)
15
(No Transcript)
16
Access to AP courses by Race and Ethnicity.
Detroit Metro Area 1995-1998
17
Honors and AP Courses by High School
18
Equity and Assessment

• These correlates of school racial composition and
  differential access to high quality teachers and
  curriculum are associated with low and or poor
  performance on a variety of assessments.

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Percent Special Education and Gifted Blacks
24
Percent Special Education and Gifted Latino
25
Percent Special Education and Gifted Whites
26
Equity and Assessment

• These conditions are just some of those that
  challenge useful and effective assessment of
  student learning.
• Following are recommendations from NRC reports
  that can help guide good assessment practices in
  the face of these challenges.

27
High Stakes Findings and Recommendations
CROSS-CUTTING THEMES

• What teachers teach and what students learn
  vary widely by track, with those in lower tracks
  receiving far less than a world-class curriculum.
  If world-class standards were suddenly adopted,
  student failure would be unacceptably high (Linn,
  1998a).
• Recommendation Accountability for educational
  outcomes
• should be a shared responsibility of states,
  school districts,
• public officials, educators, parents, and
  students. High standards
• cannot be established and maintained merely by
  imposing
• them on students.
• Recommendation If parents, educators, public
  officials, and
• others who share responsibility for educational
  outcomes are
• to discharge their responsibility effectively,
  they should have
• access to information about the nature and
  interpretation of
• tests and test scores. Such information should be
  made avail-able
• to the public and should be incorporated into
  teacher
• education and into educational programs for
  principals, administrators,
• public officials, and others.

28
High Stakes Findings and Recommendations
CROSS-CUTTING THEMES

• Recommendation A test may appropriately be used
  to lead
• curricular reform, but it should not also be used
  to make high-stakes
• decisions about individual students until test
  users can
• show that the test measures what they have been
  taught.
• Recommendation Test users should avoid simple
  either-or
• options when high-stakes tests and other
  indicators show that
• students are doing poorly in school, in favor of
  strategies combining
• early intervention and effective remediation of
  learning problems.
• Recommendation High-stakes decisions such as
  tracking, pro-motion,
• and graduation should not automatically be made
  on
• the basis of a single test score but should be
  buttressed by other
• relevant information about the students
  knowledge and skills,
• such as grades, teacher recommendations, and
  extenuating circumstances.

29
High Stakes Findings and Recommendations
CROSS-CUTTING THEMES

• Recommendation In general, large-scale
  assessments should
• not be used to make high-stakes decisions about
  students who
• are less than 8 years old or enrolled below grade
  3.
• Recommendation All students are entitled to
  sufficient test
• preparation so their performance will not be
  adversely affected
• by unfamiliarity with item format or by ignorance
  of appropriate
• test-taking strategies. Test users should balance
  efforts to
• prepare students for a particular test format
  against the possibility
• that excessively narrow preparation will
  invalidate test
• outcomes.
• Recommendation High-stakes testing programs
  should routinely
• include a well-designed evaluation component.
  Policy-makers
• should monitor both the intended and unintended
  con-sequences
• of high-stakes assessments on all students and on
• significant subgroups of students, including
  minorities, English-
• language learners, and students with disabilities.

30
High Stakes Findings and Recommendations
APPROPRIATE USES OF TESTS IN TRACKING,PROMOTION,
AND GRADUATION
TRACKING

• Recommendation As tracking is currently
  practiced, low-track
• classes are typically characterized by an
  exclusive focus
• on basic skills, low expectations, and the
  least-qualified teachers.
• Students assigned to low-track classes are worse
  off than
• they would be in other placements. This form of
  tracking
• should be eliminated. Neither test scores nor
  other information
• Recommendation Since tracking decisions are
  basically placement
• decisions, tests and other information used for
  this purpose
• should meet professional test standards regarding
  placement.

31
High Stakes Findings and Recommendations
APPROPRIATE USES OF TESTS IN TRACKING,PROMOTION,
AND GRADUATION
PROMOTION AND RETENTION

• Recommendation Tests and other information used
  in promotion
• decisions may be interpreted either as evidence
  of mastery
• of material already taught or as evidence of
  student readiness
• for material at the next grade level. In the
  former case,
• test content should be representative of the
  curriculum at the
• current grade level. In the latter case, test
  scores should predict
• the likely educational effects of future
  placements
• whether promotion, retention in grade, or some
  other intervention
• options.
• Recommendation If a cut score is to be employed
  on a test
• used in making a promotion decision, the quality
  of the standard-
• setting process should be documented and
  evaluated
• including the qualification of the judges
  employed, the method
• or methods employed, and the degree of consensus
  reached.

32
High Stakes Findings and Recommendations
APPROPRIATE USES OF TESTS IN TRACKING,PROMOTION,
AND GRADUATION
PROMOTION AND RETENTION

• Recommendation Students who fail should have the
  opportunity
• to retake any test used in making promotion
  decisions
• this implies that tests used in making promotion
  decisions
• should have alternate forms.
• Recommendation Test users should avoid the
  simple either-or
• option to promote or retain in grade when
  high-stakes tests
• and other indicators show that students are doing
  poorly in
• school, in favor of strategies combining early
  identification and
• effective remediation of learning problems.

33
High Stakes Findings and Recommendations
APPROPRIATE USES OF TESTS IN TRACKING,PROMOTION,
AND GRADUATION
TRACKING

• Recommendation As tracking is currently
  practiced, low-track
• classes are typically characterized by an
  exclusive focus
• on basic skills, low expectations, and the
  least-qualified teachers.
• Students assigned to low-track classes are worse
  off than
• they would be in other placements. This form of
  tracking
• should be eliminated. Neither test scores nor
  other information
• Recommendation Since tracking decisions are
  basically placement
• decisions, tests and other information used for
  this purpose
• should meet professional test standards regarding
  placement.

34
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
STUDENTS WITH DISABILITIES

• Recommendation More research is needed to enable
  students
• with disabilities to participate in large-scale
  assessments in ways
• that provide valid information. This goal
  significantly challenges
• current knowledge and technology about
  measurement
• and test design and the infrastructure needed to
  achieve broad-based
• participation.
• Recommendation The needs of students with
  disabilities
• should be considered throughout the test
  development process.
• Recommendation Decisions about how students with
  disabilities
• will participate in large-scale assessments
  should be
• guided by criteria that are as systematic and
  objective as possible.
• They should also be applied on a case-by-case
  basis as
• part of the childs individual education program
  and consistent
• with the instructional accommodations that the
  child receives.

35
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
STUDENTS WITH DISABILITIES

• Recommendation If a student with disabilities is
  subject to
• an assessment used for promotion or graduation
  decisions, the
• IEP team should ensure that the curriculum and
  instruction
• received by the student through the individual
  education pro-gram
• is aligned with test content and that the student
  has had
• adequate opportunity to learn the material
  covered by the test.
• Recommendation Students who cannot participate
  in a large-scale
• assessment should have alternate ways of
  demonstrating
• proficiency.
• Recommendation Because a test score may not be a
  valid
• representation of the skills and achievement of
  students with
• disabilities, high-stakes decisions about these
  students should
• consider other sources of evidence such as
  grades, teacher recommendations,
• and other examples of student work.

36
High Stakes Findings and Recommendations
APPROPRIATE USES OF TESTS IN TRACKING,PROMOTION,
AND GRADUATION
TRACKING

• Recommendation As tracking is currently
  practiced, low-track
• classes are typically characterized by an
  exclusive focus
• on basic skills, low expectations, and the
  least-qualified teachers.
• Students assigned to low-track classes are worse
  off than
• they would be in other placements. This form of
  tracking
• should be eliminated. Neither test scores nor
  other information
• Recommendation Since tracking decisions are
  basically placement
• decisions, tests and other information used for
  this purpose
• should meet professional test standards regarding
  placement.

37
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
ENGLISH LANGUAGE LEARNERS

• Recommendation Systematic research that
  investigates the
• impact of specific accommodations on the test
  performance of
• both English-language learners and other students
  is needed.
• Accommodations should be investigated to see
  whether they
• reduce construct-irrelevant sources of variance
  for English-language
• learners without disadvantaging other students
  who
• do not receive accommodations. The relationship
  of test accommodations
• to instructional accommodations should also be
• studied.
• Recommendation Development and implementation of
  alternative
• measures, such as primary-language assessments,
• should be accompanied by information regarding
  the validity,
• reliability, and comparability of scores on
  primary-language
• and English assessments.

38
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
ENGLISH LANGUAGE LEARNERS

• Recommendation The learning and language needs
  of English-
• language learners should be considered during
  test development.
• Recommendation Policy decisions about how
  individual English-
• language learners will participate in large-scale
  assessments
• such as the language and accommodations to be
  used
• should balance the demands of political
  accountability with
• professional standards of good testing practice.
  These standards
• require evidence that such accommodations or
  alternate
• forms of assessment lead to valid inferences
  regarding performance.
• Recommendation States, school districts, and
  schools should
• report and interpret disaggregated assessment
  scores of English-
• language learners when psychometrically sound for
  the
• purpose of analyzing their educational outcomes.

39
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
ENGLISH LANGUAGE LEARNERS

• Recommendation Placement decisions based on
  tests should
• incorporate information about educational
  accomplishments,
• particularly literacy skills, in the primary
  language. Certification
• tests (e.g., for high school graduation) should
  be designed
• to reflect state or local deliberations and
  decisions about the
• role of English-language proficiency in the
  construct to be
• assessed. This allows for subject-matter
  assessment in English
• only, in the primary language, or using a test
  that accommodates
• English-language learners by providing
  English-language
• assistance, primary language support, or both.
• Recommendation As for all learners,
  interpretation of the
• test scores of English-language learners for
  promotion or graduation
• should be accompanied by information about
  opportunities
• to master the material tested. For
  English-language learners,
• this includes information about educational
  history,
• exposure to instruction in the primary language
  and in English,
• language resources in the home, and exposure to
  the main-stream
• curriculum.

40
High Stakes Findings and Recommendations
APPROPRIATE USES OF TESTS IN TRACKING,PROMOTION,
AND GRADUATION
TRACKING

• Recommendation As tracking is currently
  practiced, low-track
• classes are typically characterized by an
  exclusive focus
• on basic skills, low expectations, and the
  least-qualified teachers.
• Students assigned to low-track classes are worse
  off than
• they would be in other placements. This form of
  tracking
• should be eliminated.
• Neither test scores nor other information should
  ever be used to assign children to inefficacious
  treatments.
• Recommendation Since tracking decisions are
  basically placement
• decisions, tests and other information used for
  this purpose
• should meet professional test standards regarding
  placement.

41
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
ENGLISH LANGUAGE LEARNERS

• Recommendation Systematic research that
  investigates the
• impact of specific accommodations on the test
  performance of
• both English-language learners and other students
  is needed.
• Accommodations should be investigated to see
  whether they
• reduce construct-irrelevant sources of variance
  for English-language
• learners without disadvantaging other students
  who
• do not receive accommodations. The relationship
  of test accommodations
• to instructional accommodations should also be
  studied.
• Recommendation Development and implementation of
  alternative
• measures, such as primary-language assessments,
• should be accompanied by information regarding
  the validity,
• reliability, and comparability of scores on
  primary-language
• and English assessments.

42
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
ENGLISH LANGUAGE LEARNERS

• Recommendation The learning and language needs
  of English-
• language learners should be considered during
  test development.
• Recommendation Policy decisions about how
  individual English-
• language learners will participate in large-scale
  assessments
• such as the language and accommodations to be
  used
• should balance the demands of political
  accountability with
• professional standards of good testing practice.
  These standards
• require evidence that such accommodations or
  alternate
• forms of assessment lead to valid inferences
  regarding performance.
• Recommendation States, school districts, and
  schools should
• report and interpret disaggregated assessment
  scores of English-
• language learners when psychometrically sound for
  the
• purpose of analyzing their educational outcomes.

43
High Stakes Findings and Recommendations
FORMS OF TESTINGPARTICIPATION AND ACCOMMODATIONS
ENGLISH LANGUAGE LEARNERS

• Recommendation Placement decisions based on
  tests should
• incorporate information about educational
  accomplishments,
• particularly literacy skills, in the primary
  language. Certification
• tests (e.g., for high school graduation) should
  be designed
• to reflect state or local deliberations and
  decisions about the
• role of English-language proficiency in the
  construct to be
• assessed. This allows for subject-matter
  assessment in English
• only, in the primary language, or using a test
  that accommodates
• English-language learners by providing
  English-language
• assistance, primary language support, or both.
• Recommendation As for all learners,
  interpretation of the
• test scores of English-language learners for
  promotion or graduation
• should be accompanied by information about
  opportunities
• to master the material tested. For
  English-language learners,
• this includes information about educational
  history,
• exposure to instruction in the primary language
  and in English,
• language resources in the home, and exposure to
  the main-stream
• curriculum.

44
Equity and Assessment

• Resource materials from the National Academies
• High Stakes Testing for Tracking, Promotion, and
  Graduation (1998)
• Testing English-Language Learners in U.S.
  Schools Report and Workshop Summary (2000)
• Understanding Dropouts Statistics Strategies and
  High Stakes Testing (2001)