Using assessment to improve learning: why, what and how?

1
Using assessment to improve learningwhy, what
and how?

• Dylan Wiliam
• Institute of Education
• Cambridge Assessment Network seminar on
  Assessment for Learning
• Cambridge, UK September 2006

2
Overview of presentation

• Why raising achievement is important
• Why investing in teachers is the answer
• Why assessment for learning should be the focus
• How we can put this into practice

3
Raising achievement matters

• For individuals
• Increased lifetime salary
• Improved health
• For society
• Lower criminal justice costs
• Lower health-care costs
• Increased economic growth

4
Wheres the solution?

• Structure
• Small schools
• Big schools
• Alignment
• Curriculum reform
• Textbook replacement
• Governance
• Specialist schools
• Vouchers
• Technology

5
Its the classroom

• Variability at the classroom level is up to 4
  times greater than at school level
• Its not class size
• Its not the between-class grouping strategy
• Its not the within-class grouping strategy
• Its the teacher

6
Teacher quality

• A labour force issue with 2 solutions
• Replace existing teachers with better ones?
• No evidence that more pay brings in better
  teachers
• No evidence that there are better teachers out
  there deterred by certification requirements
• Improve the effectiveness of existing teachers
• The love the one youre with strategy
• It can be done
• We know how to do it, but at scale? Quickly?
  Sustainably?

7
Learning power environments

• Key concept
• Teachers do not create learning
• Learners create learning
• Teaching as engineering learning environments
• Key features
• Create student engagement (pedagogies of
  engagement)
• Well-regulated (pedagogies of contingency)

8
Why pedagogies of engagement?

• Intelligence is partly inherited
• So what?
• Intelligence is partly environmental
• Environment creates intelligence
• Intelligence creates environment
• Learning environments
• High cognitive demand
• Inclusive
• Obligatory

9
Motivation cause or effect?
high
arousal
Flow
anxiety
challenge
control
worry
relaxation
apathy
boredom
low
low
competence
high
(Csikszentmihalyi, 1990)
10
Why pedagogies of contingency?

• Several major reviews of the research
• Natriello (1987)
• Crooks (1988)
• Kluger DeNisi (1996)
• Black Wiliam (1998)
• Nyquist (2003)
• All find consistent, substantial effects

11
Cost/effect comparisons
Intervention Effect (sd) Cost/yr/classroom
Class-size reduction (by 30) 0.1 20k
Increase teacher content knowledge by 1 sd 0.1 ?
Formative assessment/ Assessment for learning 0.20.3 2k
12
Five key strategies

• Clarifying and understanding learning intentions
  and criteria for success
• Engineering effective classroom discussions that
  elicit evidence of learning
• Providing feedback that moves learners forward
• Activating students as instructional resources
  for each other
• Activating students as the owners of their own
  learning

13
and one big idea

• Use evidence about learning to adapt instruction
  to meet student needs

14
Keeping Learning on Track (KLT)

• A pilot guides a plane or boat toward its
  destination by planning a route, taking constant
  readings and making careful adjustments in
  response to wind, currents, weather, etc.
• A KLT teacher does the same
• Plans a carefully chosen (possibly
  differentiated) route ahead of time (in essence
  building the track)
• Takes readings along the way
• Changes course as conditions dictate

15
Formative assessment Assessment for Learning

• Assessment for learning is any assessment for
  which the first priority in its design and
  practice is to serve the purpose of promoting
  pupils learning. It thus differs from assessment
  designed primarily to serve the purposes of
  accountability, or of ranking, or of certifying
  competence. An assessment activity can help
  learning if it provides information to be used as
  feedback, by teachers, and by their pupils, in
  assessing themselves and each other, to modify
  the teaching and learning activities in which
  they are engaged.
• Such assessment becomes formative assessment
  when the evidence is actually used to adapt the
  teaching work to meet learning needs.
• Black et al., 2002

16
Types of formative assessment

• Long-cycle
• Focus across units, terms
• Length four weeks to one year
• Medium-cycle
• Focus within and between teaching units
• Length one to four weeks
• Short-cycle
• Focus within and between lessons
• Length
• day-by-day 24 to 48 hours
• minute-by-minute 5 seconds to 2 hours

17
Putting it into practice
18
A model for teacher learning

• Content (what we want teachers to change)
• Evidence
• Ideas (strategies and techniques)
• Process (how to go about change)
• Choice
• Flexibility
• Small steps
• Accountability
• Support

19
Content strategies and techniques

• Distinction between strategies and techniques
• Strategies define the territory of AfL (no
  brainers)
• Teachers are responsible for choice of techniques
• Allows for customization/ caters for local
  context
• Creates ownership
• Shares responsibility
• Key requirements of techniques
• embodiment of deep cognitive/affective principles
  
• relevance
• feasibility
• acceptability

20
Design and intervention
Our design process
cognitive/affective insights
synergy/ comprehensiveness
set ofcomponents
Teachers implementation process
set of components
synergy/ comprehensiveness
cognitive/affective insights
21
Techniques for embeddingthe strategies in
practice
22
Questioning in Science

• What can we do to preserve the ozone layer?
• Reduce the amount of carbon dioxide produced by
  cars and factories
• Reduce the greenhouse effect
• Stop cutting down the rainforests
• Limit the numbers of cars that can be used when
  the level of ozone is high
• Properly dispose of air-conditioners and fridges

23
Questioning in English

• Which of these is a good thesis statement?
• The typical TV show has 9 violent incidents
• There is a lot of violence on TV
• The amount of violence on TV should be reduced
• Some programs are more violent than others
• Violence is included in programs to boost ratings
• Violence on TV is interesting
• I dont like the violence on TV
• The essay I am going to write is about violence
  on TV

24
(No Transcript)
25
Practical techniques

• Feedback
• Not giving complete solutions
• Three-fourths-of-the-way-through-a-unit test
• Sharing learning intentions
• Annotated examples of different standards to
  flesh out assessment rubrics (e.g. lab reports)
• Opportunities for students to design their own
  tests
• Students as owners of their own learning
• Red/green discs
• Students as instructional resources for one
  another
• Pre-flight checklist

26
Putting it into practice
27
Why research hasnt changed teaching

• The nature of expertise in teaching
• Aristotles main intellectual virtues
• Episteme knowledge of universal truths
• Techne ability to make things
• Phronesis practical wisdom
• What works is not the right question
• Everything works somewhere
• Nothing works everywhere
• Whats interesting is under what conditions
  does this work?
• Teaching is mainly a matter of phronesis, not
  episteme

28
Knowledge transfer
After Nonaka Tageuchi, 1995
29
Supporting Teachers and Schools to Change through
Teacher Learning Communities
30
Implementing AfL requires changing teacher habits

• Teachers know most of this already
• So the problem is not a lack of knowledge
• Its a lack of understanding what it means to do
  AfL
• Thats why telling teachers what to do doesnt
  work
• Experience alone is not enoughif it were, then
  the most experienced teachers would be the best
  teacherswe know thats not true (Hanushek, 2005)
  
• People need to reflect on their experiences in
  systematic ways that build their accessible
  knowledge base, learn from mistakes, etc.
  (Bransford, Brown Cocking, 1999)

31
Thats what TLCs are for

• TLCs contradict teacher isolation
• TLCs reprofessionalize teaching by valuing
  teacher expertise
• TLCs deprivatize teaching so that teachers
  strengths and struggles become known
• TLCs offer a steady source of support for
  struggling teachers
• They grow expertise by providing a regular space,
  time, and structure for that kind of systematic
  reflecting on practice
• They facilitate sharing of untapped expertise
  residing in individual teachers
• They build the collective knowledge base in a
  school

32
The synergy

• Content assessment for learning
• Process teacher learning communities
• Components of a model
• Initial workshops
• Support for TLC leaders
• Monthly TLC meetings
• Peer observations
• Drip-feed resources
• Web-site
• Writings
• New ideas

33
Summary

• Raising achievement is important
• Raising achievement requires improving teacher
  quality
• Improving teacher quality requires teacher
  professional development
• To be effective, teacher professional development
  must address
• What teachers do in the classroom
• How teachers change what they do in the classroom
• AfL TLCs
• A point of (uniquely?) high leverage
• A Trojan Horse into wider issues of pedagogy,
  psychology, and curriculum

34
Questions?Comments?