Developing assessment for learning with teacher learning communities

1
Developing assessment for learning with teacher
learning communities

• Dylan Wiliam,
• Institute of Education, University of London
• www.dylanwiliam.net

2
Overview of presentation

• Why raising achievement is important
• Why investing in teachers is the answer
• Why assessment for learning should be the focus
• Why teacher learning communities should be the
  mechanism
• How we can put this into practice

3
Raising achievement matters

• For individuals
• Increased lifetime salary
• Improved health
• For society
• Lower criminal justice costs
• Lower health-care costs
• Increased economic growth

4
Wheres the solution?

• Structure
• Small high schools
• K-8 schools
• Alignment
• Curriculum reform
• Textbook replacement
• Governance
• Charter schools
• Vouchers
• Technology

5
School effectiveness

• 3 generations of effectiveness research
• Raw results approaches
• Different schools get different results
• Conclusion Schools make a difference
• Demographic-based approaches
• Demographic factors account for most of the
  variation
• Conclusion Schools dont make a difference
• Value-added approaches
• School-level differences in value-added are
  relatively small
• Classroom-level differences in value-added are
  large
• Conclusion An effective school is little more
  than a school full of effective classrooms

6
Its the classroom

• Variability at the classroom level is up to 4
  times greater than at school level
• Its not class size
• Its not the between-class grouping strategy
• Its not the within-class grouping strategy
• Its the teacher

7
Teacher quality

• A labor force issue with 2 solutions
• Replace existing teachers with better ones?
• No evidence that more pay brings in better
  teachers
• No evidence that there are better teachers out
  there deterred by certification requirements
• Improve the effectiveness of existing teachers
• The love the one youre with strategy
• It can be done
• We know how to do it, but at scale? Quickly?
  Sustainably?

8
Why assessment for learning?

• Several major reviews of the research
• Natriello (1987)
• Crooks (1988)
• Kluger DeNisi (1996)
• Black Wiliam (1998)
• Nyquist (2003)
• All find consistent, substantial effects

9
Cost/effect comparisons
10
Types of formative assessment

• Long-cycle
• Span across units, terms
• Length four weeks to one year
• Medium-cycle
• Span within and between teaching units
• Length one to four weeks
• Short-cycle
• Span within and between lessons
• Length
• day-by-day 24 to 48 hours
• minute-by-minute 5 seconds to 2 hours

11
Effects of formative assessment

• Long-cycle
• Student monitoring
• Curriculum alignment
• Medium-cycle
• Improved, student-involved, assessment
• Improved teacher cognition about learning
• Short-cycle
• Improved classroom practice
• Improved student engagement

12
Kinds of feedback (Nyquist, 2003)

• Weaker feedback only
• Knowledge of results (KoR)
• Feedback only
• KoR clear goals or knowledge of correct results
  (KCR)
• Weak formative assessment
• KCR explanation (KCRe)
• Moderate formative assessment
• (KCRe) specific actions for gap reduction
• Strong formative assessment
• (KCRe) activity

13
Effect of formative assessment (HE)
14
Feedback and formative assessment

• Feedback is information about the gap between
  the actual level and the reference level of a
  system parameter which is used to alter the gap
  in some way (Ramaprasad, 1983 p. 4)
• Three key instructional processes
• Establishing where learners are in their learning
• Establishing where they are going
• Establishing how to get there

15
Aspects of formative assessment
16
Five Key Strategies
17
and one big idea

• Use evidence about learning to adapt instruction
  to meet student needs

18
Keeping Learning on Track (KLT)

• A pilot guides a plane or boat toward its
  destination by taking constant readings and
  making careful adjustments in response to wind,
  currents, weather, etc.
• A KLT teacher does the same
• Plans a carefully chosen route ahead of time (in
  essence building the track)
• Takes readings along the way
• Changes course as conditions dictate

19
Eliciting evidence of student achievement by
engineering effective classroom discussions,
questions and learning tasks
20
Practical techniques questioning

• Key idea questioning should
• cause thinking
• provide data that informs teaching
• Improving teacher questioning
• generating questions with colleagues
• closed v open
• low-order v high-order
• appropriate wait-time
• Getting away from I-R-E
• basketball rather than serial table-tennis
• No hands up (except to ask a question)
• class polls to review current attitudes towards
  an issue
• Hot Seat questioning
• All-student response systems
• ABCD cards, Mini white-boards, Exit passes

21
Questioning in math discussion

• Look at the following sequence
• 3, 7, 11, 15, 19, .
• Which is the best rule to describe the sequence?
• n 4
• 3 n
• 4n - 1
• 4n 3

22
Questioning in math diagnosis

• In which of these right triangles is a2 b2 c2
  ?

23
(No Transcript)
24
Questioning in science discussion

• Ice-cubes are added to a glass of water. What
  happens to the level of the water as the
  ice-cubes melt?
• The level of the water drops
• The level of the water stays the same
• The level of the water increases
• You need more information to be sure

25
Questioning in science diagnosis

• The ball sitting on the table is not moving. It
  is not moving because
• no forces are pushing or pulling on the ball.
  
• gravity is pulling down, but the table is in the
  way.
• the table pushes up with the same force that
  gravity pulls down
• gravity is holding it onto the table.
• there is a force inside the ball keeping it from
  rolling off the table

Wilson Draney, 2004
26
Dinosaur extinction

• Why did dinosaurs become extinct?
• Humans destroyed their habitat
• Humans killed them all for food
• There was a major change in climate

27
Save the ozone layer

• What can we do to preserve the ozone layer?
• Reduce the amount of carbon dioxide produced by
  cars and factories
• Reduce the greenhouse effect
• Stop cutting down the rainforests
• Limit the numbers of cars that can be used when
  the level of ozone is high
• Properly dispose of air-conditioners and fridges

28
Practical techniques

• Feedback
• Not giving complete solutions
• Three-fourths-of-the-way-through-a-unit test
• Sharing learning intentions
• Annotated examples of different standards to
  flesh out assessment rubrics (e.g. lab reports)
• Opportunities for students to design their own
  tests
• Students as owners of their own learning
• Red/green discs
• Students as instructional resources for one
  another
• Pre-flight checklist

29
Putting it into practice
30
A model for teacher learning

• Content (what we want teachers to change)
• Evidence
• Ideas (strategies and techniques)
• Process (how to go about change)
• Choice
• Flexibility
• Small steps
• Accountability
• Support

31
Strategies and techniques

• Distinction between strategies and techniques
• Strategies define the territory of AfL (no
  brainers)
• Teachers are responsible for choice of techniques
• Allows for customization/ caters for local
  context
• Creates ownership
• Shares responsibility
• Key requirements of techniques
• embodiment of deep cognitive/affective principles
  
• relevance
• feasibility
• acceptability

32
Design and intervention
Our design process
cognitive/affective insights
synergy/ comprehensiveness
set ofcomponents
Teachers implementation process
set of components
synergy/ comprehensiveness
cognitive/affective insights
33
Why research hasnt changed teaching

• The nature of expertise in teaching
• Aristotles main intellectual virtues
• Episteme knowledge of universal truths
• Techne ability to make things
• Phronesis practical wisdom
• What works is not the right question
• Everything works somewhere
• Nothing works everywhere
• Whats interesting is under what conditions
  does this work?
• Teaching is mainly a matter of phronesis, not
  episteme

34
Supporting Teachers and Schools to Change through
Teacher Learning Communities
35
Implementing AfL requires changing teacher habits

• Teachers know most of this already
• So the problem is not a lack of knowledge
• Its a lack of understanding what it means to do
  AfL
• Thats why telling teachers what to do doesnt
  work
• Experience alone is not enoughif it were, then
  the most experienced teachers would be the best
  teacherswe know thats not true (Hanushek, 2005)
  
• People need to reflect on their experiences in
  systematic ways that build their accessible
  knowledge base, learn from mistakes, etc.
  (Bransford, Brown Cocking, 1999)

36
Thats what TLCs are for

• TLCs contradict teacher isolation
• TLCs reprofessionalize teaching by valuing
  teacher expertise
• TLCs deprivatize teaching so that teachers
  strengths and struggles become known
• TLCs offer a steady source of support for
  struggling teachers
• They grow expertise by providing a regular space,
  time, and structure for that kind of systematic
  reflecting on practice
• They facilitate sharing of untapped expertise
  residing in individual teachers
• They build the collective knowledge base in a
  school

37
The synergy

• Content assessment for learning
• Process teacher learning communities
• Components of a model
• Initial workshops
• Support for TLC leaders
• Monthly TLC meetings
• Peer observations
• Drip-feed resources
• Web-site
• Writings
• New ideas

38
Summary

• Raising achievement is important
• Raising achievement requires improving teacher
  quality
• Improving teacher quality requires teacher
  professional development
• To be effective, teacher professional development
  must address
• What teachers do in the classroom
• How teachers change what they do in the classroom
• AfL TLCs
• A point of (uniquely?) high leverage
• A Trojan Horse into wider issues of pedagogy,
  psychology, and curriculum