Assessment for learning: why, what, and how?

1
Assessment for learningwhy, what, and how?

• Dylan Wiliam
• www.dylanwiliam.net

2
Overview of presentation

• Why raising achievement is important
• Why investing in teachers is the answer
• Why formative assessment should be the focus
• Why teacher learning communities should be the
  mechanism
• How we can put this into practice

Why, what how?
3
Raising achievement matters

• For individuals
• Increased lifetime salary
• Improved health
• Longer life
• For society
• Lower criminal justice costs
• Lower health-care costs
• Increased economic growth

Why?
4
Wheres the solution?

• Structure
• Smaller schools/larger schools
• Middle schools/getting rid of middle schools
• Alignment
• Curriculum reform
• Textbook replacement
• Governance
• Specialist schools
• Academies
• Technology
• Computers
• Interactive white-boards

Why?
5
School effectiveness

• Three generations of school effectiveness
  research
• Raw results approaches
• Different schools get different results
• Conclusion Schools make a difference
• Demographic-based approaches
• Demographic factors account for most of the
  variation
• Conclusion Schools dont make a difference
• Value-added approaches
• School-level differences in value-added are
  relatively small
• Classroom-level differences in value-added are
  large
• Conclusion An effective school is a school full
  of effective classrooms

Why?
6
Its the classroom

• Variability at the classroom level is up to 4
  times that at school level
• Its not class size
• Its not the between-class grouping strategy
• Its not the within-class grouping strategy
• Its the teacher

Why?
7
Teacher quality

• A labour force issue with 2 solutions
• Replace existing teachers with better ones?
• No evidence that more pay brings in better
  teachers
• No evidence that there are better teachers out
  there deterred by burdensome certification
  requirements
• Improve the effectiveness of existing teachers
• The love the one youre with strategy
• It can be done
• We know how to do it, but at scale? Quickly?
  Sustainably?

Why?
8
Cost/effect comparisons
Intervention Extra months of learning per year Cost/yr
Class-size reduction (by 30) 4 20k
Increase teacher content knowledge from weak to strong 2 ?
Formative assessment/ Assessment for learning 8 2k
Why?
9
The research evidence

• Several major reviews of the research
• Natriello (1987)
• Crooks (1988)
• Kluger DeNisi (1996)
• Black Wiliam (1998)
• Nyquist (2003)
• All find consistent, substantial effects

Why?
10
Unpacking formative assessment

• Key processes
• Establishing where the learners are in their
  learning
• Establishing where they are going
• Working out how to get there
• Participants
• Teachers
• Peers
• Learners

What?
11
Aspects of formative assessment
Where the learner is going Where the learner is How to get there
Teacher Clarify and share learning intentions Engineering effective discussions, tasks and activities that elicit evidence of learning Providing feedback that moves learners forward
Peer Understand and share learning intentions Activating students as learning resources for one another Activating students as learning resources for one another
Learner Understand learning intentions Activating students as ownersof their own learning Activating students as ownersof their own learning
12
Five key strategies

• Clarifying, understanding, and sharing learning
  intentions
• curriculum philosophy
• Engineering effective classroom discussions,
  tasks and activities that elicit evidence of
  learning
• classroom discourse, interactive whole-class
  teaching
• Providing feedback that moves learners forward
• feedback
• Activating students as learning resources for one
  another
• collaborative learning, reciprocal teaching,
  peer-assessment
• Activating students as owners of their own
  learning
• metacognition, motivation, interest, attribution,
  self-assessment

(Wiliam Thompson, 2007)
13
and one big idea

• Use evidence about learning to adapt teaching and
  learning to meet student needs

What?
14
Keeping Learning on Track (KLT)

• A pilot guides a plane or boat toward its
  destination by taking constant readings and
  making careful adjustments in response to wind,
  currents, weather, etc.
• A KLT teacher does the same
• Plans a carefully chosen route ahead of time (in
  essence building the track)
• Takes readings along the way
• Changes course as conditions dictate

What?
15
Putting it into practice
16
Implementing FA/AfL requires changing teacher
habits

• Teachers know most of this already
• So the problem is not a lack of knowledge
• Its a lack of understanding what it means to do
  FA/AfL
• Thats why telling teachers what to do doesnt
  work
• Experience alone is not enoughif it were, then
  the most experienced teachers would be the best
  teacherswe know thats not true (Hanushek, 2005
  Day, 2006)
• People need to reflect on their experiences in
  systematic ways that build their accessible
  knowledge base, learn from mistakes, etc.
  (Bransford, Brown Cocking, 1999)

How?
17
A model for teacher learning

• Content, then process
• Content (what we want teachers to change)
• Evidence
• Ideas (strategies and techniques)
• Process (how to go about change)
• Choice
• Flexibility
• Small steps
• Accountability
• Support

How?
18
Strategies and techniques

• Distinction between strategies and techniques
• Strategies define the territory of AfL (no
  brainers)
• Teachers are responsible for choice of techniques
• Allows for customization/ caters for local
  context
• Creates ownership
• Shares responsibility
• Key requirements of techniques
• embodiment of deep cognitive/affective principles
  
• relevance
• feasibility
• acceptability

How?
19
Examples of techniques

• Clarifying learning intentions and success
  criteria
• sharing exemplars
• Eliciting evidence
• mini white-boards
• Providing feedback that moves learners forward
• find it and fix it
• Activating students as owners of their learning
• coloured cups
• Activating students as learning resources for one
  another
• pre-flight checklist

How?
20
Design and intervention
Our design process
cognitive/affective insights
synergy/ comprehensiveness
set ofcomponents
Teachers implementation process
set of components
synergy/ comprehensiveness
cognitive/affective insights
21
Teacher learning takes time

• To put new knowledge to work, to make it
  meaningful and accessible when you need it,
  requires practice.
• A teacher doesnt come at this as a blank slate.
• Not only do teachers have their current habits
  and ways of teachingtheyve lived inside the old
  culture of classrooms all their lives every
  teacher started out as a student!
• New knowledge doesnt just have to get learned
  and practiced, it has to go up against
  long-established, familiar, comfortable ways of
  doing things that may not be as effective, but
  fit within everyones expectations of how a
  classroom should work.
• It takes time and practice to undo old habits and
  become graceful at new ones. Thus
• Professional development must be sustained over
  time

How?
22
Thats what teacher learning communities (TLCs)
are for

• TLCs contradict teacher isolation
• TLCs reprofessionalize teaching by valuing
  teacher expertise
• TLCs deprivatize teaching so that teachers
  strengths and struggles become known
• TLCs offer a steady source of support for
  struggling teachers
• They grow expertise by providing a regular space,
  time, and structure for that kind of systematic
  reflecting on practice
• They facilitate sharing of untapped expertise
  residing in individual teachers
• They build the collective knowledge base in a
  school

How?
23
Kings-Medway-Oxfordshire Formative Assessment
Project

• Polyexperiment design
• 24 teachers, each developing their practice in
  individual ways
• Each teacher chose which class to explore these
  ideas with
• Each teacher chose how to measure success
• Different outcome variables, so no possibility of
  standardized controls
• Synthesis by standardized effect size
• Impact on student achievement
• 0.3 standard deviations (i.e., about 8 months
  extra learning per year)
• Other small-scale replications (Hayes, 2003
  Clymer 2007) find similar effects

How?
24
Taking it to scale
25
How to set up a TLC

• Plan that the TLC will run for two years
• Identify 8 to 10 interested colleagues
• Should have similar assignments (e.g. early
  years, math/sci)
• Secure institutional support for
• Monthly meetings (2 hrs each, inside or outside
  school time)
• Time between meetings (2 hrs per month in school
  time)
• Collaborative planning
• Peer observation
• Any necessary waivers from school policies

How?
26
A signature pedagogy for teacher learning?

• Every monthly TLC meeting should follows the same
  structure and sequence of activities
• Activity 1 Introduction Housekeeping (5
  minutes)
• Activity 2 Hows It Going (50 minutes)
• Activity 3 New Learning about AfL (50 minutes)
• Activity 4 Personal Action Planning (10 minutes)
• Activity 5 Summary of Learning (5 minutes)

How?
27
The TLC leaders role

• To ensure the TLC meets regularly
• To ensure all needed materials are at meetings
• To ensure that each meeting is focused on AfL
• To create and maintain a productive and
  non-judgmental tone during meetings
• To ensure that every participant shares with
  regard to their implementation of AfL
• To encourage teachers to provide their colleagues
  with constructive and thoughtful feedback
• To encourage teachers to think about and discuss
  the implementation of new AfL learning and skills
• To ensure that every teacher has an action plan
  to guide their next steps
• But not to be the AfL expert

How?
28
Peer observation

• Run to the agenda of the observed, not the
  observer
• Observed teacher specifies focus of observation
• Observe teacher specifies what counts as evidence
• e.g., teacher wants to increase wait-time
• provides observer with a stop-watch to log
  wait-times

How?
29
Tight but loose
Some reforms are too loose (e.g., the Effective
schools movement) Others are too tight (e.g.,
Montessori Schools) The tight but loose
formulation
combines an obsessive adherence to central
design principles (the tight part) with
accommodations to the needs, resources,
constraints, and particularities that occur in
any school or district (the loose part), but
only where these do not conflict with the theory
of action of the intervention.

• Tight about
• Teacher choice
• Strategies
• Hows it going? action planning
• Size of TLC

• Loose about
• Timing and location of meetings
• Techniques
• New learning about AfL
• Make-up of TLC

How?
30
Summary

• Raising achievement is important
• Raising achievement requires improving teacher
  quality
• Improving teacher quality requires teacher
  professional development
• To be effective, teacher professional development
  must address
• What teachers do in the classroom
• How teachers change what they do in the classroom
• AfL/FA TLCs
• A point of (uniquely?) high leverage
• A Trojan Horse into wider issues of pedagogy,
  psychology, and curriculum

Why, what how?