Universally Designed Assessments

1
Universally Designed Assessments
National Center on Educational
Outcomes University of Minnesota
2
Goals

• Define universally designed assessments within
  the current context of large-scale assessments
  and accountability
• Identify elements of universally designed
  assessments

3
Universally designed assessments

• are designed from the beginning to be accessible
  and valid for the widest range of students
• provide optimal, standard assessment conditions
  for the widest range of students

4
Who Benefits?

• Universal design does not apply exclusively to
  people with disabilities or limited English
  proficiency
• It applies to all individuals, with wide ranging
  characteristics

5
Think about universal design in architecture and
tool design

• Curb cuts and ramps
• Elevators that talk to you
• Door handles rather than knobs
• Special pen shapes that are easier to hold

6
Proposed Title I Regulations (open to comment at
this time) introduce the need for universally
designed assessments
Assessments must be designed to be accessible
and valid with respect to the widest possible
range of students, including students with
disabilities and students with limited English
proficiency.
Sec. 200.2(b)(2)
7

OFFICIAL BALLOT, PALM BEACH COUNTY, FLORIDA
8
REGULAR PEOPLE GET TRIPPED UP BY THE SIMPLEST
THINGS
Whats obvious to someone who knows the answer is
not always obvious to everyone.
9
Amazon.com Usability or Confusability?Many of
our customers have told us that Amazon.com's user
interface is easy to navigate, dependable, and
dedicated to ensuring prosperity. But we can do
more. Inspired by the Palm Beach County,
Florida, ballot form, we've devised a totally new
way for you to find and discover anything you
want to buy online. We've put the best possible
spin on this design, and we invite you to give it
a spin, too.
10
(No Transcript)
11
Conclusion by Amazon.comIt Matters!
Ease of Use
12
Why Might We Want Universally Designed
Assessments for Students with Disabilities?

• Current wide ranges in use of accommodations
  from 8 to 84 of students in latest NCEO survey
• Possible misuse (over or under use) of
  accommodations (in some cases)
• Better measurement of students with disabilities

13
No More Accommodations?

• universally designed assessments will not
  eliminate the need for accommodations
• universally designed assessments may reduce the
  need for accommodations
• universally designed assessments will reduce
  threats to validity and score comparability when
  accommodations are used

14
Elements of UD Assessments

• Inclusive assessment population
• Precisely defined constructs
• Items developed and reviewed for bias and
  accessibility
• Amenable to accommodations
• Simple, clear, and intuitive instructional and
  procedures

15
Elements of UD Assessments (continued)

• Maximum readability/comprehensibility
• Legible text
• Legible graphs, tables, and illustrations
• Legible response formats

16
Element 1
Inclusive Assessment Population

• Universally designed assessments are responsive
  to

• All types of students in the general curriculum
• A commitment to serve and be accountable for ALL
  students

17
Element 1
Inclusive Assessment Population

• Universally designed assessments are responsive
  to

• Equitable participation for all students,
  regardless of

• Cognitive ability
• Cultural background

18
Element 2
Precisely Defined Constructs

• Universally designed assessments reflect good
  measurement qualities

• Actually measure what they are intended to
  measure
• Remove all non-construct-oriented cognitive,
  sensory, emotional, and physical barriers

19
An Example
Mathematics Tests
The reading requirements of a math test often
prevent students with marginal reading ability
from demonstrating competency in math.
20
Ordering Pizza (Original Item)
The cafeteria manager surveyed the students in a
middle school to find out if they would buy Brand
X pizza on Friday if the manager sold it. She
made a circle graph to display the results of her
survey.
NO
YES
Based on the results of the survey, answer the
following questions

• What fraction of students would buy Brand X pizza
  on Friday?
• What percent of students would buy Brand X pizza
  on Friday?
• There are 1200 students in this school. How many
  students will buy Brand X pizza on Friday if the
  managers survey is accurate?

21
Ordering Pizza (Revised Item)
Maria surveyed the students in her school to find
out if they liked pizza on Friday. She made a
circle graph to display the results of her survey.
NO
YES

• What fraction of students said yes?
• What percent of students said yes?
• There are 1200 students in Marias school. How
  many students said yes?

22
The language used in questions on tests that
assess subjects other than language needs to
become as transparent as possible
23
Element 3
Bias and Accessibility Considered During Item
Development and Review

• Universally designed assessments incorporate
  accessibility as a primary dimension of test
  specifications

• Insist that item developers are trained
• Form Bias Review Panels that include individuals
  who know disability and language issues, as well
  as cultural, gender, and other issues

24
Bias includes anything in an item that could
potentially advantage or disadvantage any
subgroup of examinees.
It takes special thinking and review to ensure
that items are not biased for each and every
student who will be tested.
25
Element 4
Amenable to Accommodations

• Universally designed assessments allow needed
  accommodations to be used

• Plan for students who continue to need
  accommodations
• Facilitate the use of accommodations such as
  assistive technology

26
Element 5
Simple, Clear, and Intuitive Instructions and
Procedures

• Universally designed assessments focus on the
  knowledge and skills assessed, not on seeing
  whether the student can figure out how to respond

• Applies regardless of experience, knowledge,
  language skills, or concentration level
• Not knowing how to respond can invalidate a
  students test score

27
Element 6
Maximum Readability/Comprehensibility

• Universally designed assessments attend to
  various factors that affect readability

• Students previous experiences, achievement, and
  interests
• Features such as word and sentence difficulty,
  organization of materials, and format

28
Sample Readability Guidelines

• Use simple, clear, commonly used words,
  eliminating any unnecessary words
• Clearly define any technical terms that are used
• Break compound complex sentences into several
  short sentences. State the most important ideas
  first
• Introduce one idea, fact, or process at a time

29
Element 7
Legible Text

• Universally designed assessments use text that
  enables people to read quickly, effortlessly and
  with understanding

• The physical appearance of text shapes of
  letters and numbers conforms to several
  dimensions that characterize legible text

30
Legible Text Dimensions
Contrast Black type on matte pastel or
off-white paper produces good contrast and
reduces eye strain Type Size Print larger than
12 point increases legibility Spacing Space
between letters and between words in wide
31
Legible Text Dimensions
Leading White space between lines of type
(leading) is larger Typeface Standard typeface,
with upper and lower case letters, is better than
italic, small caps, or all caps Justification
Unjustified text is easier to read, especially
for poor readers
32
Legible Text Dimensions
Line Length Text should be about 40-70
characters, or about 8-12 words per line Blank
Space Space around paragraphs and between
columns of type increases legibility
33
Element 8
Legible Graphs, Tables, Illustrations

• Universally designed assessments use non-text
  materials just as carefully as text materials

• Symbols are highly distinguishable
• Only essential illustrations are used (ones
  referred to in text and necessary to answer
  question) illustrations for interest often draw
  attention away from construct being assessed

34
Element 9
Legible Response Formats

• Universally designed assessments consider the
  design of the response venue as well as the
  assessment itself

• Large bubbles that avoid most challenges of low
  vision or difficulty with fine motor skills
• Consideration of age of students in selecting
  format (avoid separate answer sheets for younger
  students)

35
More information?
NCEO Resources

• Visit www.education.umn.edu/nceo
• or Search for NCEO
• Web site includes
• Topic introduction
• Frequently Asked Questions
• Online and Other Resources

36
Question Are All Universally Designed
Assessments Computerized?

• No, universal design principles apply to all
  media used for assessments, including the current
  dominant one pencil and paper tests.

• But, it may be easier to provide an array of
  options to students through computerized
  assessments.

37
Computer-Based Assessments

• Must maintain each element of universal design
• Can be poorly designed and inaccessible in much
  the same way as paper and pencil assessments
• Should be used with great caution unless equity
  issues have been addressed

38
Equity Issues

• Access to computers
• Experience using computers
• Training and practice with assistive technology
  devices and software (e.g., screen readers,
  speech synthesizers)

39
Choice allowed by computer-based testing is a
significant benefit for students they can use
the options that are most useful to them!
40
Caution
While universally designed assessments can make
tests more equitable, producing results that are
more valid for all students, they cannot replace
instructional opportunity!