Two Categories of Test Accommodations for English Language Learners

1
The Promise of Technology-Based Assessment
Eva L. Baker
UCLA Graduate School of Education Information
StudiesCenter for the Study of
EvaluationNational Center for Research on
Evaluation, Standards, and Student Testing
CoSN, ISTE, SETDA Dallas, TX November 12, 2002
2
Technology-Based Assessment for High-Performance
Learning

• How do we meet and go beyond NCLB requirements?
• What is the role of technology in assessment?
• How can technology link classroom and external
  testing into a coherent system?

3
Technology for What?

• Most available technology-based assessment
  promotes efficiency rather than expands the
  measurement of learning
• Tendency to limit attention to accountability
  requirements
• Use technology to extend our understanding of
  student accomplishment and school quality

4
Basics

• Role of external testsaccountability and system
  monitoring
• Role of tests in classrooms and schools
• Formative assessment as a teaching strategy
• Linking through standards, content, cognitive
  demands (what thinking skills are required?)
• CRESST research-based models

5
CRESST Assessment Models

• Research based
• Focus on cognition and learning
• Reusable and cost sensitive
• Technology-supported authoring templates
• Technology-based administration, scoring, and
  reporting

6
Families of Cognitive DemandsModel-Based
Assessment
Content Understanding
Learning
Problem Solving
Teamwork and Collaboration
Metacognition
Communication
7
State, District, and Classroom Assessments Must
Be More Tightly Linked Than Present Standards
Permit

• A tiered system
• Conceptual linkage
• For accountability systems, all measures must
  count
• Used to identify those that need

8
How Can Authoring Really Work?

• Authoring systems (computer-supported guidance)
  to help teachers create and share various sorts
  of assessments intended to measure standards not
  covered by state test or that need deeper
  attention

9
Strategy to Link

• Cognitive demands and content
• Authoring system for use by teachers to either
  expand to other standards not externally measured
  or to link their assessments to standards
• Basis of linkage detailed knowledge of subject
  matter and of how to assess intellectual skills
  in the system

10
Authoring

• Teacher will get guidance
• What is measured? What content? What conditions?
  What intellectual skills?
• Scoring rubrics (based on expert performance)
  will be provided but can be edited
• Graphical representation

11
Assessment Authoring Benefits

• Common floor on assessments created by teachers
• Systems start with easy, fixed formats related to
  learning demands, and as teacher sophistication
  develops, move to more choices (mix and match)
• Will allow summaries of student work bubbling up
  from teachers formative assessments
• Could permit integration across different teachers

12
Authoring Systems

• Scored work made public (within the school, with
  privacy provisions, and among schools)
• Success depends upon teacher subject matter
  knowledge, access to needed information, and
  sharing
• Success depends on realistic link to external
  examinations

13
Elements for Improvement

• Infrastructure
• Capacity
• Integration with learning
• Congruence with external mandates
• Emphasis on students
• Tools

14
EXCERPTS from HAWAIIAN HISTORYPRIMARY SOURCE
DOCUMENTS
LILIUOKALANI
For many years our sovereigns had welcomed
the advice of American residents who had
established industries on the Islands. As they
became wealthy, their greed and their love of
power increased. Although settled among us, and
drawing their wealth from resources, they were
alien to us in their customs and ideas, and
desired above all things to secure their own
personal benefit. Kalakaua valued the
commercial and industrial prosperity of his
kingdom highly. He sought honestly to secure it
for every class of people, alien or native.
Kalakauas highest desire was to be a true
sovereign, the chief servant of a happy,
prosperous, and progressive people. And
now, without any provocation on the part of the
king, having matured their plans in secret, the
men of foreign birth rose one day en masse,
called a public meeting, and forced the king to
sign a constitution of their own preparation, a
document which deprived him of all power and
practically took away the franchise from the
Hawaiian race.
15
Hawaiian History WritingAssignment
BayonetConstitution
Imagine you are in a class that has been
studying Hawaiian history. One of your friends,
who is a new student in the class, has missed all
the classes. Recently, your class began studying
the Bayonet Constitution. Your friend is very
interested in this topic and asks you to explain
everything that you have learned about it.
Write an essay explaining the most important
ideas you want your friend to understand.
Include what you have already learned in class
about Hawaiian history, and what you have learned
from the texts you have just read. While you
write, think about what Thurston and Liliuokalani
said about the Bayonet Constitution, and what is
shown in the other materials.
Your essay should be based on two major sources
1. The general concepts and specific facts you
know about Hawaiian history, and especially
what you know about the period of the Bayonet
Constitution. 2. What you have learned from the
readings yesterday.
Be sure to show the relationships among your
ideas and facts.
16
History ExplanationScoring Rubric
1. General Impression of Content Quality 2.
Number of Principles or Concepts 3. Prior
Knowledge Facts 4. Use of Resources 5.
Misconceptions 6. Argumentation 7. English
Mechanics
17
Mathematics ExplanationTask (4th or 5th Grade)

Imagine a person from a television station has
asked you to give a demonstration on TV. You will
be on a show to help other students learn about
math. You are asked to explain everything
fifth-grade students should know about
fractions. Below are some questions you should
try to answer. These are questions that students
in the TV audience will ask you. For each
question you should draw as many pictures as you
can to show what you mean. Then write down what
you would say about your pictures on TV. Use as
many words and pictures as you need. What is a
fraction? Why are there two numbers in a
fraction? How many fractions are there between 0
and 1? How many fractions are equal to 1/2? What
other important ideas should students know about
fractions? Show how you would explain these
ideas. Use as many pictures and words as you need.
18
Genetics - High
19
History
20
Bicycle Pump - High
21
Search Strategy
22
Sample Screen List for ProbS Authors

• Assessment Purpose(s) diagnostic, readiness
  monitoring, certification
• Scenario
• Context, constraints, situation
• Problem Characteristics
• Fix, change usual sequence, improvise step(s),
  combo
• Problem Identification Menu
• Stated, embedded, multiply masked, barriers,
  inconsistent data from multiple sources, time
  bound, partial identification, prior knowledge
  requirements

23
ProbS Author Screens (contd)

• Macro Planning Menu
• Explicit courses of action, problem subdivision,
  backup strategies, help seeking, mix of
  domain-independent and domain-dependent cognitive
  strategies
• Trial and Feedback Menu
• Data capture of process, process feedback, help,
  iteration
• Solution(s)
• Convergent (right answer), multiple acceptable,
  partially acceptable, divergentwith scoring
  criteria, sequential, mixed

24
ProbS Examinee Screens

• Scenario
• Problem
• Information acquisition
• Macro strategy
• Micro strategies (domain specific)
• Solution trials
• Report of performance

25
ProbS Examiner Information

• Time spent on problem
• Trials to criterion
• Help access
• Solution paths
• Generalizability of solution
• Acceptability of solution
• Likely explanation for errors (e.g., lack of
  prior knowledge)
• Metric (standards for performance)

26
Summary

• Technology-based assessment needs to extend what
  we can do
• Authoring systems can help teachers design
  better, more sensitive tests and projects
• Technology can help us share findings
• Technology-based assessment requires the same
  evidence of technical quality
• Demand evidence, not business claims, before you
  buy