RE-THINKING HOW SCHOOLS IMPROVE: A Team Dialogue Model for Data-Based Instructional Decision Making

1
RE-THINKING HOW SCHOOLS IMPROVE A Team
Dialogue Model for Data-Based
Instructional Decision Making
2
The Big Picture

• In todays session, we are going to
• Re-think our understanding of how schools improve
  -- moving from the dysfunction of the old model
  to the requirements for what a new model might
  look like.
• Focus on a new model for improving performance
  that enables content, vertical, or departmental
  teams to use data more effectively for classroom
  instructional improvement and increased student
  learning.

3
Every organization is perfectly designed to get
the results it achieves. --W.
Edwards Deming
4
What are data?
Data are observations, facts, or numbers which,
when collected, organized and analyzed, become
information and, when used productively in
context, become knowledge.
5
The DRIP Syndrome
DATA RICH
INFORMATION POOR
6
Being Data Rich
Your school may suffer from
DATA OVERLOAD
You may need ways to organize and use all the
data you have.
7
Sources of Student Achievement Data

• External assessment data
• Benchmark or course-wide assessment data
• Individual teacher assessment data
• --Supovitz and Klein (2003)

8
Data-driven schools and school districts use data
for two major purposes

• Accountability (to prove)
• Instructional decision making
• (to improve)

9
The Hierarchy of Data for Accountability Purposes
External (State National) Assessments System
Benchmark Assessments Common School or Course
Assessments Classroom Assessments of Student
Work
10
The Hierarchy of Data for Instructional Decision
Making
Classroom Assessments of Student Work Common
School or Course Assessments System Benchmark
Assessments External (State National
Assessments)
11
Think about how long you have been engaged in the
school improvement process. Has the school gotten
better each year? Has the performance of each
student improved as a result of each year he/she
spends in the school? If your answer to one or
both questions is no, what will it take to change
it to yes?
12

• Think about your School Improvement Plan and the
  data on which it is primarily based. Is it . . .
• State assessment data? (data to prove)
• OR
• Classroom assessments of student work? (data to
  improve)

13
Then ask yourself this question
Do you have a school improvement plan?
Or a school accountability plan?
A SIP ?
Or a SAP?
14
WHY IS THE OLD MODEL OF SCHOOL IMPROVEMENT NOT
WORKING ANY MORE?
15
WHY IS THE OLD MODEL OF SCHOOL IMPROVEMENT NOT
WORKING ANY MORE? You may wish to stop the
presentation at this time to discuss your
facultys views on this question.
16
Why? Wrong Data

• We have been using the wrong data. State test
  data are
• Way too general
• Instructionally insensitive -- not designed
  for instructional improvement

17
Why? Wrong Time

• The data come at the wrong time. State test data
  are
• Out of date when they arrive
• For students we may no longer have
• The results of the changes that are implemented
  will not be known for a year.

18
Why? Wrong Team

• The SIT, a full department, or a Data Committee
  are the wrong groups to do the analysis.
• Membership is too diverse (often including
  parents)
• Meets too infrequently
• Not connected to immediate classroom needs

19
Why? Wrong Plan

• The initiatives that are put in place are
• Too global to address
• the diversity of students
• Aimed at performance increases
• of groups on average
• Looking for the silver bullet that
• will have a schoolwide impact

20
We need a new model.

• Uses real time data
• Sessions build on each other
• Addresses individual students needs
• Results in instructional improvements that will
  actually
• occur at a high level of quality
• Can be re-directed frequently
• Has meaning for teachers and is seen by
  teachers as a
• worthwhile use of their time

21
We need a new model.

• Uses real time data
• Sessions build on each other
• Addresses individual students needs
• Results in instructional improvements that will
  actually
• occur at a high level of quality
• Can be re-directed frequently
• Has meaning for teachers and is seen by
  teachers as a
• worthwhile use of their time
• You may wish to stop the presentation at this
  time and discuss the extent to which these
  positive characteristics are present in the data
  analysis process your school is currently using.

22
What should that new model look like?
School improvement is most surely and
thoroughly achieved when teachers engage in
frequent, continuous, and increasingly concrete
and precise talk about teaching practice . . .
adequate to the complexities of teaching, and
capable of distinguishing one practice and its
virtue from another. --Judith
Warren Little
23
Fundamental Concepts of Collaborative Learning
Communities

• Teachers establish a common, concise set of
  essential curricular standards and teach to them
  on a roughly common schedule.
• Teachers meet regularly as a team for purposes of
  talking in . . . concrete and precise terms
  about instruction with a concentration on
  thoughtful, explicit examination of practices
  and their consequences.
• Teachers make frequent use of common assessments.
• Continued on next slide

24
These elements, so rarely emphasized in school .
. . improvement plans, deserve our attention more
than anything else we do in the name of school
improvement. --Mike Schmoker (2006)
25
Components of THE NEW MODEL
THE CLASSROOM-FOCUSED IMPROVEMENT PROCESS
(CFIP) A Team Data Dialogue Protocol
26
The new process needs to be built on

• 1. Dialogue
• 2. Protocols
• 3. Triangulation of
• Data (use of multiple
• data sources)

27
Our Goal in the Data Dialogues

• Frequent, continuous, and increasingly concrete
  and precise dialogue by school teams, informed by
  data

28
What are the right teams to conduct data
dialogues?

• Teams that share common standards and assessments
• Grade-level teams
• Content teams
• Vertical teams

29
When is the right time to conduct data dialogues?

• At a minimum, devote at least one hour to data
  dialogues every two weeks.
• According to several studies, schools that
  realized the greatest results from a shift to a
  data culture scheduled data dialogues at least
  once a week.

30
Frequency of Data Dialogues
Source Stanford University,
Stanford Research Institute, Education Week,
January 24, 2004
31
What are the right data to use in the data
dialogues?

• Triangulate three types of data
• External Assessment Data
• Course-wide Benchmark Assessment Data
• Classroom Assessment Data
• --Supovitz Klein (2003)

32
What is the right plan where the results of the
data dialogues should be used?

• Conclusions are specific to students in the
  class.
• Conclusions are used to plan upcoming daily
  instruction.
• The plans are implemented.

33
What is the right way to use the results of the
data dialogues?

• Conclusions are used to identify enrichments and
  interventions for the students in the class.
• Conclusions are used to
• plan upcoming daily
• instruction.

34
What Is a Data Protocol?
A protocol consists of guidelines for dialogue
which everyone understands and has agreed to
that permit a certain kind of conversation to
occur, often a kind of conversation which people
are not in the habit of having. Protocols build
the skills and culture necessary for
collaborative work. Protocols often allow groups
to build trust by doing substantive work together.
35
Using a Data Protocol

• Protocols can help us to navigate difficult and
  uncomfortable conversations by
• Making it safe to ask challenging
• questions
• Making the most of scarce time
• Providing an opportunity for all to be
• involved
• Resulting in an analysis that will lead to
• positive action

36
Using a Data Protocol

• The point is not to do the protocol well, but to
  have team dialogue that is
• In-depth
• Insightful
• Concrete
• Precise

37
Six Easy CFIP Steps 1. Understand the data
source. 2. Begin with a question. 3. Look for
class-wide patterns. 4. Act on the class
patterns. 5. Address individual students
needs. 6. Improve instruction in the next
lesson.
38
CFIP Step 1 Understand the data source.

• Build ASSESSMENT LITERACY with questions like
  these
• What assessment is being described in this data
  report? What were the characteristics of the
  assessment?
• Who participated in the assessment? Who did not?
  Why?
• Why was the assessment given? When?
• What do the terms in the data report mean?
• Be sure you have clear and complete answers to
  these questions before you proceed any further.

39
CFIP Step 2 Identify the questions that can be
answered by the data.

• All data analyses should be designed to answer a
  question.
• Unless there is an important question to answer,
  there is no need for a data analysis.

40
CFIP Step 3A Look for class-wide patterns in a
single data source.

• What do you see over and over again in the data?
• What are the strengths of the class? What
  knowledge and skills do the students have?
• What are their weaknesses of the class? What
  knowledge and skills do the students lack?

41
CFIP Step 3B Identify patterns of class
strengths and weaknesses from multiple data
sources.

• TRIANGULATION
• In what ways are the results similar among data
  sources? For example, how do benchmark test
  results compare with ongoing classroom assessment
  data?
• In what ways do the results among data sources
  differ?
• Why might these differences occur?

42
Power When Multiple Types of Data Are Used

• Reduces the anxiety and the mistakes of relying
  on a single measure as the only definition of
  student success
• Provides more frequent evidence on which to act
• Develops and sustains a culture of inquiry in the
  school based on data

43
CFIP Step 4 Act on the class-wide patterns.

• What instructional factors might have contributed
  to the class-wide patterns?
• What will we do to address patterns of class
  needs?
• How and when will we reassess to determine
  progress?

44
CFIP Step 5 Drill down to individual
students. Identify and implement needed
enrichments and interventions.

• What are the implications for enrichments and
  interventions from what you learn from the data?
  
• Which students need enrichments and
  interventions?
• What should enrichments and interventions focus
  on?

45
CFIP Step 6 Reflect on the reasons for student
performance -- What in our teaching might be
preventing all students from being successful?

• To what extent did we implement research-based
  instructional practices as we
• Planned instruction?
• Introduced instruction?
• Taught the unit?
• Brought closure to instruction?
• Assessed formatively?

46
CFIP Step 6 Reflect on the reasons for
student performance. Identify and implement
instructional changes in the next unit.

• How will we change instruction in our next unit?
• Content focus
• Pacing
• Teaching methods
• Assignments

47
CFIP Step 6 Reflect on the reasons for
student performance. Identify and implement
instructional changes in the next unit.

• When will we review the data again to determine
  the success of the enrichments, interventions,
  and instructional changes?
• What do the data not tell us?
• What questions about student achievement do we
  still need to answer?
• How will we attempt to answer these questions?

48
The Next Steps

1. Unless teams emerge from the data analysis
   process with a clear plan of action for their
   classroom, they have wasted their time.
2. Implement the plan of interventions, enrichments,
   and changes in instruction.
3. Collect the next set of data.

49
Six Easy CFIP Steps 1. Understand the data
source. 2. Begin with a question. 3. Look for
class-wide patterns. 4. Act on the class
patterns. 5. Address individual students
needs. 6. Improve instruction in the next
lesson. You may wish to stop the presentation at
this time and review the six steps again, keeping
in mind that teams will study and practice using
the steps prior to their implementation of CFIP.
50
Caveats about CFIP

• It is a paradigm shift from the traditional
  lesson planning format.
• It is not easy, especially at first.
• Follow the steps faithfully until they become
  second nature.
• Expect mistakes and imprecision in the data.
• The results are worth the effort.