PADI: Principled Assessment Design for Inquiry

1
PADIPrincipled Assessment Design for Inquiry

• Robert J. Mislevy University of Maryland
• Geneva Haertel SRI, International

Supported by NSF, via IERI NSF, DOE,
NIH Partners SRI International University
of Maryland University of California at
Berkeley (BEAR FOSS) Lawrence Hall of Science
(FOSS) University of Michigan (BioKIDS)
2
The Problem

• Creating good science inquiry tasks is hard.
• How to do it validly and efficiently?
• Whats the connection with science standards?
• Whats the connection with the learning
  psychology?
• How to implement within given content, for given
  kinds ages of students?
• How do you score it?
• How to sort out confounded evidence?
• How to leverage technology?

3
The PADI Solution

• Conceptual frameworks representations
• Evidence-centered design approach
• Object model for designing assessment arguments
  and assessment tasks.
• Software tools to aid processes of
• Design
• Implementation
• Delivery

4
The PADI Solution

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

5
The PADI Solution
PADI Design Patterns Narrative structures for
assessing pervasive kinds of knowledge / skill /
capabilities, such as model-based reasoning,
inquiry cycles, building scientific explanations.

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

6
The PADI Solution
PADI Templates Task Specs Technical
structures for specifying elements such as
measurement models, materials and presentation
capabilities, evaluation procedures,
interactivity.

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

7
The PADI Solution
Item authoring not a focus of PADI, although CAF
objects are whats needed for implementation.
Task specs are IMS/QTI compatible.

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

8
The PADI Solution
Delivery is also not a focus of this project,
although PADI is consistent with IMS/QTI
four-process delivery architecture of.
Exceptions PADI scoring engine for multivariate
IRT modeling and Mystery Powders delivery
system example.

• Evidence-Centered Design Layers
• From Mislevy Riconscente, PADI Technical Report
• Also in Downing Haladyna, Handbook on Test
  Development

9
A question

• Isnt this just new words for describing what we
  already do?

10
An answer (1)

• No.

11
An answer (2)

• An explicit, general framework makes similarities
  and implicit principles explicit
• To better understand current assessments
• To design for new kinds of assessment
• Inquiry investigations
• Technology-based tasks (e.g., simulations)
• Complex observations, student models, evaluation
• To foster re-use, sharing, modularity
• Concepts arguments
• Pieces of machinery processes (QTI)