Classroom Assessment: Minute-by-minute and day-by-day

1
Classroom Assessment Minute-by-minute and
day-by-day

• Dylan Wiliam
• www.dylanwiliam.net

2
Overview of presentation

• Why raising achievement is important
• Why investing in teachers is the answer
• Why formative assessment should be the focus
• Why teacher learning communities should be the
  mechanism
• How we can put this into practice

3
Raising achievement matters

• For individuals
• Increased lifetime salary
• Improved health
• Longer life
• For society
• Lower criminal justice costs
• Lower health-care costs
• Increased economic growth

4
Wheres the solution?

• Structure
• Smaller high schools
• K-8 schools
• Alignment
• Curriculum reform
• Textbook replacement
• Governance
• Charter schools
• Vouchers
• Technology
• Computers
• Interactive white-boards

5
School effectiveness

• Three generations of school effectiveness
  research
• Raw results approaches
• Different schools get different results
• Conclusion Schools make a difference
• Demographic-based approaches
• Demographic factors account for most of the
  variation
• Conclusion Schools dont make a difference
• Value-added approaches
• School-level differences in value-added are
  relatively small
• Classroom-level differences in value-added are
  large
• Conclusion An effective school is a school full
  of effective classrooms

6
How important is teacher quality?

• How much progress will an average student make
  when taught by a great teacher (i.e., the best
  teacher in a group of 50)?
• An extra month per year
• An extra two months per year
• An extra three months per year
• An extra four months per year
• An extra six months per year

7
Teacher quality

• A labor force issue with 2 solutions
• Replace existing teachers with better ones?
• No evidence that more pay brings in better
  teachers
• No evidence that there are better teachers out
  there deterred by burdensome certification
  requirements
• Improve the effectiveness of existing teachers
• The love the one youre with strategy
• It can be done
• We know how to do it, but at scale? Quickly?
  Sustainably?

8
The dark matter of teacher quality

• Teachers make a difference
• But what makes the difference in teachers?

9
Cost/effect comparisons
Intervention Extra months of learning per year Cost/yr
Class-size reduction (by 30) 3 30k
Increase teacher content knowledge from weak to strong 1.5 ?
Formative assessment/ Assessment for learning 6 to 9 3k
10
The research evidence

• Several major reviews of the research
• Natriello (1987)
• Crooks (1988)
• Kluger DeNisi (1996)
• Black Wiliam (1998)
• Nyquist (2003)
• All find consistent, substantial effects

11
Types of formative assessment

• Long-cycle
• Span across units, terms
• Length four weeks to one year
• Impact Student monitoring curriculum alignment
• Medium-cycle
• Span within and between teaching units
• Length one to four weeks
• Impact Improved, student-involved, assessment
  teacher cognition about learning
• Short-cycle
• Span within and between lessons
• Length
• day-by-day 24 to 48 hours
• minute-by-minute 5 seconds to 2 hours
• Impact classroom practice student engagement

12
Unpacking formative assessment

• Key processes
• Establishing where the learners are in their
  learning
• Establishing where they are going
• Working out how to get there
• Participants
• Teachers
• Peers
• Learners

13
Aspects of formative assessment
Where the learner is going Where the learner is How to get there
Teacher Clarify and share learning intentions Engineering effective discussions, tasks and activities that elicit evidence of learning Providing feedback that moves learners forward
Peer Understand and share learning intentions Activating students as learning resources for one another Activating students as learning resources for one another
Learner Understand learning intentions Activating students as ownersof their own learning Activating students as ownersof their own learning
14
Sharing learning intentions

• Explaining learning intentions at start of
  lesson/unit
• Learning intentions
• Success criteria
• Intentions/criteria in students language
• Posters of key words to talk about learning
• eg describe, explain, evaluate
• Planning/writing frames
• Annotated examples of different standards to
  flesh out assessment rubrics (e.g. lab reports)
• Opportunities for students to design their own
  tests

15
Eliciting evidence of achievement

• Key idea questioning should
• cause thinking
• provide data that informs teaching
• Improving teacher questioning
• generating questions with colleagues
• closed vs. open or low-order vs. high-order
• appropriate wait-time
• Getting away from I-R-E
• basketball rather than serial table-tennis
• No hands up (except to ask a question)
• Hot Seat questioning
• All-student response systems
• ABCD cards, Mini white-boards, Exit passes

16
Feedback that moves learning on

• Key idea feedback should
• cause thinking
• provide guidance on how to improve
• Comment-only grading
• Focused grading
• Explicit reference to mark-schemes and scoring
  guides
• Suggestions on how to improve
• Strategy cards ideas for improvement
• Not giving complete solutions
• Re-timing assessment
• (eg two-thirds-of-the-way-through-a-unit test)

17
Students as owners of their learning

• Students assessing their own work
• with rubrics
• with exemplars
• Self-assessment of understanding
• Traffic lights
• Red/green discs
• Colored cups

18
Students as instructional resources

• Students assessing their peers work
• pre-flight check-list
• two stars and a wish
• Training students to pose questions/identifying
  group weaknesses
• End-of-lesson students review

19
and one big idea

• Use evidence about learning to adapt teaching and
  learning to meet student needs

20
Keeping Learning on Track (KLT)

• A pilot guides a plane or boat toward its
  destination by taking constant readings and
  making careful adjustments in response to wind,
  currents, weather, etc.
• A KLT teacher does the same
• Plans a carefully chosen route ahead of time (in
  essence building the track)
• Takes readings along the way
• Changes course as conditions dictate

21
Putting it into practice
22
Implementing FA/AfL requires changing teacher
habits

• Teachers know most of this already
• So the problem is not a lack of knowledge
• Its a lack of understanding what it means to do
  FA/AfL
• Thats why telling teachers what to do doesnt
  work
• Experience alone is not enoughif it were, then
  the most experienced teachers would be the best
  teacherswe know thats not true (Hanushek, 2005
  Day, 2006)
• People need to reflect on their experiences in
  systematic ways that build their accessible
  knowledge base, learn from mistakes, etc.
  (Bransford, Brown Cocking, 1999)

23
A model for teacher learning

• Content, then process
• Content (what we want teachers to change)
• Evidence
• Ideas (strategies and techniques)
• Process (how to go about change)
• Choice
• Flexibility
• Small steps
• Accountability
• Support

24
Strategies and techniques

• Distinction between strategies and techniques
• Strategies define the territory of AfL (no
  brainers)
• Teachers are responsible for choice of techniques
• Allows for customization/ caters for local
  context
• Creates ownership
• Shares responsibility
• Key requirements of techniques
• embodiment of deep cognitive/affective principles
  
• relevance
• feasibility
• acceptability

25
Teacher learning takes time

• To put new knowledge to work, to make it
  meaningful and accessible when you need it,
  requires practice.
• A teacher doesnt come at this as a blank slate.
• Not only do teachers have their current habits
  and ways of teachingtheyve lived inside the old
  culture of classrooms all their lives every
  teacher started out as a student!
• New knowledge doesnt just have to get learned
  and practiced, it has to go up against
  long-established, familiar, comfortable ways of
  doing things that may not be as effective, but
  fit within everyones expectations of how a
  classroom should work.
• It takes time and practice to undo old habits and
  become graceful at new ones. Thus
• Professional development must be sustained over
  time

26
Thats what teacher learning communities (TLCs)
are for

• TLCs contradict teacher isolation
• TLCs reprofessionalize teaching by valuing
  teacher expertise
• TLCs deprivatize teaching so that teachers
  strengths and struggles become known
• TLCs offer a steady source of support for
  struggling teachers
• They grow expertise by providing a regular space,
  time, and structure for that kind of systematic
  reflecting on practice
• They facilitate sharing of untapped expertise
  residing in individual teachers
• They build the collective knowledge base in a
  school

27
How to set up a TLC

• Plan that the TLC will run for two years
• Identify 8 to 10 interested colleagues
• Should have similar assignments (e.g. early
  years, math/sci)
• Secure institutional support for
• Monthly meetings (75 to 120 minutes each, inside
  or outside school time)
• Time between meetings (2 hrs per month in school
  time)
• Collaborative planning
• Peer observation
• Any necessary waivers from school policies

28
A signature pedagogy for teacher learning?

• Every monthly TLC meeting should follows the same
  structure and sequence of activities
• Activity 1 Introduction Housekeeping (5-10
  minutes)
• Activity 2 Hows It Going (35-50 minutes)
• Activity 3 New Learning about AfL (20-45
  minutes)
• Activity 4 Personal Action Planning (10 minutes)
• Activity 5 Summary of Learning (5 minutes)

29
The TLC leaders role

• To ensure the TLC meets regularly
• To ensure all needed materials are at meetings
• To ensure that each meeting is focused on AfL
• To create and maintain a productive and
  non-judgmental tone during meetings
• To ensure that every participant shares with
  regard to their implementation of AfL
• To encourage teachers to provide their colleagues
  with constructive and thoughtful feedback
• To encourage teachers to think about and discuss
  the implementation of new AfL learning and skills
• To ensure that every teacher has an action plan
  to guide their next steps
• But not to be the AfL expert

30
Peer observation

• Run to the agenda of the observed, not the
  observer
• Observed teacher specifies focus of observation
• Observe teacher specifies what counts as evidence
• e.g., teacher wants to increase wait-time
• provides observer with a stop-watch to log
  wait-times

31
Implementations

• Current pilots in
• Cleveland Municipal School District, OH
• Austin Independent School District, TX
• Chico Unified School District, CA
• Mathematics and Science Partnership of Greater
  Philadelphia, PA/NJ
• St. Marys County Public Schools, MD
• State-wide pilot in 10 schools in Vermont
• Further details www.ets.org/klt

32
Summary

• Raising achievement is important
• Raising achievement requires improving teacher
  quality
• Improving teacher quality requires teacher
  professional development
• To be effective, teacher professional development
  must address
• What teachers do in the classroom
• How teachers change what they do in the classroom
• AfL/FA TLCs
• A point of (uniquely?) high leverage
• A Trojan Horse into wider issues of pedagogy,
  psychology, and curriculum