Jack B. MonpasHuber, Ph.D.

1
How Do We Know When Theyve Learned It? Guidance
for Development of Common Assessments
Jack B. Monpas-Huber, Ph.D. Director of
Assessment and Student Information
jack.monpas.huber_at_shorelineschools.org (206)
368-4774 Office [phone] Cell
2
Features of a quality common assessment

• The purpose of the common assessment is clear.
  Formative? Summative? Where in the learning
  process?
• What the common assessment is intended to measure
  is clear. What are the power standards, or
  learning targets. Develop a test map.
• The instrument gathers the right kind of data for
  the learning target. Knowledge, or process
  skill? Selected-response, or performance
  assessment?
• The instrument gathers the same kind of data in a
  consistent way. If a performance, need to
  develop a common rubric and agree on its
  application. The rubric needs to guide the
  scores, not teacher autonomy or preference.
• The instrument gathers enough data to provide
  sufficient evidence of learning, not chance.
  Three tasks.

3
Developing common assessments
Steps in the development process
Define power standards What are the big ideas
that we expect students to learn in this period
of time?
Develop an assessment (test) map How to measure
the power standards? What kinds of items are
appropriate? How large/long should this
assessment be? How many tasks do we need to
adequately measure mastery?
Develop (or populate with existing)
items/tasks/scoring rubrics What do you have
already? Which standards do they measure?
Review and Piloting Are we really measuring what
we say were measuring? Is anything confusing,
ambiguously worded, or biased?
Standard setting What counts as proficiency?
4
Developing common assessments
The importance of the test map

• It lays out a plan for assessing the
  expectations.
• It connects expectations to instruments.
• It ensures that the assessment covers what was
  taught/expected.
• It builds consistency into assessment practice.

A Test Map from 7th Grade Math
5
For established common assessments
Standard setting
Isa (judgmental) process of establishing cut
scores on examinations (Cizek, p. 225) Is nota
search for a knowable boundary that exists a
priori between categories, with the task of
standard setting participants simply to discover
it (Cizek, p. 227)
Standards must be set because decisions must be
made on some basis

• Recommendation
• Some variation of the bookmark method
• Items/tasks (re)ordered by difficulty (based on
  difficulty data)
• Judges place a bookmark where they believe the
  cutoff for proficiency should be

6
Collecting good evidence of instructional
effectiveness
The Split and Switch Design1 A variation on the
traditional pretest-posttest design

• Create two forms of a test, somewhat equal in
  difficulty.
• Split class into two halves
• Half the class takes Form A as pretest, the other
  half takes Form B as pretest
• Instruction
• Switch forms for posttest
• Blind-score all Form As (pres and posts
  scrambled), all Form Bs (scrambled)
• Calculate gains on Form A, Form B

Note Typically you would subtract the pre from
the post for each student and then average the
gain scores. Not in this case! Instead, youre
subtracting Form A pretest mean from Form A
posttest mean--even though those means are based
on different students. Can still make inferences
about instruction because all the same students
have received the same instruction (treatment).
Try it!
1Popham, J. (2001). The truth about testing.
Alexandria, VA Association for Supervision and
Curriculum Development.