Quality Assessment The Blueprinting Approach

1
Quality Assessment The Blueprinting Approach
Assoc Prof Craig Zimitat Director, Medical
Education Unit
2
Overview

• Background
• What is a blueprint?
• Why use a blueprint?
• What blueprints do we need?
• Examples

3
Perspectives

• Teachers and students tend to approach their
  tasks from different perspectives.
• Teachers focus on outcomes and content
• Students focus on assessment
• A mismatch between outcomes and assessment causes
  confusion for everyone in the teaching and
  learning enterprise.

4
Curriculum Alignment

• The curriculum is designed so that the learning
  activities and assessment tasks are clearly
  related to the intended learning outcomes.

5
Curriculum Alignment

• Too often assessment tasks are not related to
  stated outcomes, not valid for the outcome, or
  incongruent with teaching methods.

6
What is a blueprint ?

• A Blueprint is a plan that explicitly relates
  outcomes and assessment strategies .
• guides the development of assessment systems at a
  course level
• guides development of assessment as unit level
• guides the appropriate use of individual
  instruments
• It provides evidence of the design making
  processes involved in choosing appropriate
  assessment tasks, sampling of subject matter and
  balance.

7
Why use a blueprint?

• To explicitly align assessment processes with
  desired learning outcomes in terms of
• What will be assessed
• How it will be assessed
• To ensure an appropriate range of objectives,
  content and weightings are addressed when
  devising specific examination instruments
• Allows sharing and communication with others
• Provides a basis for consistency over time

8
Managing assessment

• Identifies gaps in current assessment
• Develops shared understanding amongst assessors
  in and across year groups
• Improves content validity
• Provides a framework for feedback to students
• Links with IDEAL database of questions
• Part of a QA Process

9
What will be assessed?

• State outcomes and define in terms of
  competencies (Curriculum mapping)
• Identify appropriate assessment strategies
• Look at timing
• Include other issues e.g. setting, system,
  patient age profile etc.

10
How it will be assessed

• Select the instruments that best assess the
  competencies defined
• Select instruments that are consistent with
  teaching strategies and course philosophy
• Blueprint can include
• Formative assessment tasks
• Summative assessment tasks
• Details about timing

11
Managing assessment

• Identifies gaps in current assessment
• Develops shared understanding
• Improves content validity
• Links with IDEAL database of questions
• Provides a framework for discussion and feedback
• QA Process

12
What blueprints do we need?

• At least three
• Level 1 Overall course (MGP)
• Level 2 Unit level
• consider year level for Y3 rotations
• Level 3 Instrument level
• Often OSCE will have a separate blueprint

13
Curriculum map
14
Blueprint for an assessment system
15
Blueprint Y3 Rotations OSCE
16
Blueprint Rotation OSCE x Presentation
17
QA of Current Assessment

• Map distribution of questions by instrument
• Map distribution of questions by topic
• Compare this distribution with curriculum map
• Look for links then appropriateness, balance and
  gaps
• What evidence guides current practice?
• BEECH data
• Medical Board and MDA data
• Tasmanian vs Australian data