DataBased Decision Making

1
Data-Based Decision Making

• An RTI Action Network National Online Forum
• Moderated by
• Doris McMillon

2
Funding is provided by

• The Cisco Foundation
• The Lee Pesky Learning Center
• The Janet Shafran Memorial Fund
• Wireless Generation
• The National Center on Response to Intervention

3
Featuring Dr. Lynn Fuchs Nicholas Hobbs
Professor of Special Education and Human
Development
at Vanderbilt University Dr. Joseph
Kovaleski Professor of Educational and School
Psychology at the Indiana University of
Pennsylvania Mr. John Carruth Assistant
Superintendent of Special Programs and Projects

for the Vail Unified School District
in Tucson, Arizona
4
What is Response to Intervention (RTI)?

• RTI is a process of integrating assessment with
  instruction within a multi-level prevention
  system.
• Goal to reduce risk for long-term negative
  consequences that happen when students fail to
  learn in school

5
What is data-based decision making?

• Data-based decision making is making
  instructional decisions based on assessment data.
• RTI links assessment with intervention.

6
Data types used within the RTI model

• Three purposes for assessment within RTI
• Screening identify students at risk for academic
  difficulty
• Progress monitoring determine whether the
  student is responsive to given instruction
• Individualizing instruction use assessment data
  to develop stronger programs for those who dont
  respond to the standard general education or
  supplemental instruction programs

7
Data types used within the RTI model (cont.)
8
Useful sources of RTI data

• Numerous instruments have been developed
• Based on the work of Dr. Fuchs and others on
  curriculum-based measurement and other progress
  monitoring procedures

9
Different data for different purposes

• Use two distinct data types
• Generalized outcome measure
• Mastery measurement component

10
RTI used for special education eligibility

• Practitioners are prepared with assessment
  information when RTI is in place
• They are aware of the students characteristics.
• They know the students responsiveness to
  different interventions.
• Same data can be used in evaluation process
• More efficient and effective use of data

11
RTI used for special education eligibility (cont.)

• Breaking down walls between special and regular
  educators
• Data are not only relevant to the student but
  also to the curriculum being delivered.
• Without RTI, some students are placed in special
  education even though they dont have learning
  disabilities.

12
RTI used for special education eligibility (cont.)

• Implementation
• Take collected student data and class-wide
  screening data.
• Is the class overall performing well?
• Prescribe an intervention for at-risk students.
• Track growth systematically.
• If student does not progress, collect additional
  data to make eligibility determination.

13
First steps to using data

• Assess frequently, but not to the point that it
  is interrupting instruction.
• Be prepared to eliminate what is ruled
  non-effective
• Is what we have working for us?
• Kinds of information provided
• Efficiency of administration

14
Data collection

• While the principal is not necessarily collecting
  the data, its critical he or she is a part of
  the process.
• School psychologists abilities in measurement
  and assessment should be used school-wide in
  collaboration with principal and teachers.
• Assign an analytical teacher to be the data
  manager, who will manage and help assess the
  data.
• Organize all data in a computer program for quick
  analysis.

15
Data collection tools

• Resources to help schools select reliable, valid
  tools
• studentprogress.org
• rti4success.org
• RTINetwork.org

16
Universal screening measure vs. progress
monitoring

• Need to be careful some tools can be used for
  both, some cannot.
• Universal screening
• Administered to all children at all levels
• This one-time test should be used in the fall and
  the spring.
• Brief measure is useable for decision making.
• Its use is limited only determines who might be
  at-risk.
• Progress monitoring
• Used frequently, typically on a weekly basis
• Data is used to make decisions in determining
  students responsiveness.

17
Frequency of assessment

• Assessment should typically occur on a weekly
  basis if progress monitoring is in use
• Difficult to determine rate of improvement
  without frequent assessments

18
Interpreting the data

• Fifteen years ago, we were looking at data one
  student at a time (time consuming).
• The team format has now instilled a system for
  looking at all data for all teachers. It has
  abandoned the individual child-by-child structure.

19
Interpreting the data (cont.)

• Plan a meeting to look at all students. Include
• All teachers of the specified grade level
• Principal
• Data manager
• Various staff who are expert in specified subject
  (i.e., reading)
• Overall goal every student achieves proficiency
• Instilled by No Child Left Behind

20
Interpreting the data (cont.)

• Get teachers to be reflective on how they teach
• Discuss strategies to move all students to
  proficiency.
• How should we teach on a day-to-day,
  minute-to-minute basis with the core curriculum
  to advance our students?

21
Reviewing the data

• Universal screening data are typically collected
  in the fall, winter, and spring. Teams meet right
  after to ask the big questions about the whole
  group
• Identify how the group is doing as a whole to
  determine who is individually in need of more
  intensive intervention.
• At-risk students should be assessed weekly.

22
Points to discuss with data team

• For Tier 1, examine the group
• What do the data show us?
• Are all of our students where we need them to be?
• What goals should be set?
• What can the teachers collectively do for us to
  meet our goals and to make change?
• How can we use our core curriculum resources and
  instructional procedures to make change?

23
Points to discuss with data team (cont.)

• For Tier 2, examine the individuals
• How can we group students collectively across
  different sections?
• What are their common needs?
• How can we deliver specialized instruction as a
  group?

24
Scheduling team meetings

• Teams should meet at least twice a month.
• Meetings are most successful when teachers come
  together at their specific grade level.
• Schools need to incorporate the meetings into
  their Master schedule.
• Continue rich discussions about instruction to
  develop creative solutions.

25
The teachers modified role

• Frequent meetings to assess progress
• Determine if their current instruction is
  effective and look ahead at where they want to
  go.
• Generalized outcome measure quick reading on the
  students overall learning performance related to
  the annual curriculum

26
Reading scores

• Fall reading screening
• Gives a general indicator of the overall health
  of that classroom and of the school
• Examine it by grade level, by school, and by the
  individual student.

27
Reading scores (cont.)

• Winter screening
• By the second screening, there should be visible
  progress school-wide and individually. Note the
  number of students no longer identified as being
  at risk in this example.

28
Data meetings improve student outcomes

• Each year, teams of teachers get progressively
  more students to proficiency level and have
  progressively fewer students at the at-risk
  level.
• Data meetings have helped teachers to stop using
  strategies and materials that dont tend to work,
  resulting in higher quality teaching.

29
Data meetings improve student outcomes (cont.)

• There is a collaboration among teachers to search
  for what works in the classroom
• More communication among teachers
• Demonstrating strategies for each other
• Going to look for new techniques together when a
  strategy isnt effective
• This process changes the culture of the school in
  the way teachers deliver instruction.

30
How data is used for selecting interventions

• Look at students with similar data profiles,
  indicating very specific instructional needs
• Link those data with that group of students
  both differentiating that within the general
  classroom and providing supplemental or tier
  time.
• Tier time extra time during the school day when
  students who are behind can actually get
  intensive interventions to accelerate learning so
  they can catch up.

31
Does the process vary with different subjects?

• The process is the same for all subjects
  reading, math, and writing.
• There are students who need something more
  intensive than the general education program,
  even with a supplementary program
• We have a great progress monitoring tool for
  systematically experimenting with different
  instructional components.
• When progress monitoring is implemented, the
  result is a good individualized instructional
  program.

32
Benefits of using data to make instructional
decisions

• By implementing the data process, Johns district
  has moved from being at or below state averages
  to being in the top 5 of all districts across
  Arizona in reading, writing, and math.

33
How to determine a students responsiveness

• From the data collected, look at specific
  information about the student
• How long has the student been in our system?
• To what extent have we been providing effective
  instructional practices?
• What has our intervention been, and has it been
  showing progress?
• If the student has only been in this system, you
  know theyre getting good instruction, which will
  help to make a pretty solid decision about what
  needs to happen. Its more challenging when
  students come in with a varied background of
  instruction.

34
When do you change the intervention if theres no
progress?

• We are now choosing very specific interventions
  that are proven to be effective by research.
• When we have students who are behind and not
  making progress, changes to an intervention need
  to occur within a few weeks.
• The I in RTI (intervention) is essential
• We have to help schools have a means for
  selecting the best interventions to use as a
  supplement to the general education program.

35
Data points determine a change in intervention

• We know the number of data points that are
  required
• An intervention needs time to work
• You need a minimum of 3-4 weeks and 8 data points
  before you decide to abandon the intervention for
  a new one.
• Examine the data trail (see Sams graph in slide
  7)
• Whenever we see four consecutive data points
  below the goal line, research tells us that its
  highly unlikely that this student will achieve
  the year end goal. Its time to make an
  instructional change.

36
Setting benchmarks for acceptable progress

• There are national benchmarks in place for how
  much weekly progress we would expect at a
  particular grade level. Use these to set school
  benchmarks or goals.
• At a certain instruction level, the year-end
  performance needs to show us that the child can
  make good progress the following year
• This is what were using to set instructional
  goals, even for students who are substantially
  below grade level at the present time.

37
Parents as partners in RTI

• Parents should be exposed to their childs data
  as compared to the universal screening data for a
  greater understanding of where their child falls
  in regard to the particular subject (e.g.,
  reading).
• If a student is in a risk area, parents should be
  notified that there are some concerns.
• Information should always be clearly shared with
  the parents.

38
New student, new data?

• If a student transfers within Johns district,
  their information is readily available to a
  teacher anywhere in that district. The students
  intervention plan is placed in their student file
  and is tracked and transferred with that student.
• If a student moves to another district, it can
  get tricky
• Different states use different commercial
  products and strategies. However, may of the
  products used for universal screening and
  progress monitoring are recognizable from
  district to district.
• Different assessment systems are fairly
  understandable by all educators now.

39
Data used to demonstrate success

• Program evaluation is an important way to
  distinguish if systems in place are working
• Collect the data and then aggregate it
  electronically to examine both in team meetings
  with teachers and with administrators for a
  district-wide account.

40
Data collection with support staff

• The following related service professionals can
  give more information not only about individual
  students, but also collectively on different
  approaches
• Guidance counselors
• School psychologists
• Reading specialists
• Speech and language pathologists
• Occupational therapists
• Physical therapists

41
Professional development opportunities

• New teacher induction processes at the beginning
  of the school year
• Instructional coaches readily available
• The data analysis process is good training every
  time you do it.
• Wealth of professional development resources
• RTINetwork.org
• rti4success.org
• studentprogress.org

42
Progress monitoring for reading comprehension

• People tend to think of progress monitoring
  systems for reading as a narrow reading fluency
  measure, but they are really meant to be measures
  of reading comprehension.
• New computer-based assessments for reading
  comprehension are available on the Web (see
  previous slide).

43
Success in data-based decision making

• A school successfully performing this process
  has
• Teachers meaningfully collaborating, which
  elevates their level of creativity.
• The structure of making sure that things are
  consistent and making sure that the fidelity of
  the process is in place.

44
Challenges of data-based decision making

• Visit national RTI websites for very specific
  information on implementing RTI.
• To avoid obstacles, choose wisely. This may
  entail abandoning previous practices and
  assessments that youre familiar with using.
• Make sure to make good decisions about
  assessments that are incisive, that can get you
  the data you need but are very efficient. This
  will save as much instruction time as possible.

45
Data collection differs among grade levels

• Data collection for secondary school differs from
  elementary.
• Researchers are currently developing tools to use
  for screening and progress monitoring at the
  secondary levels.

46
Student involvement with their data

• Collecting their data is a good opportunity to
  teach students about math and what graphs mean.
• Students can be taught to set goals for
  themselves
• Understanding what their highest score has been
  can prompt the following questions
• How can I beat my highest score?
• How can I ask my teacher for some help in order
  to meet my own goal?

47
Does reporting more data lead to more scrutiny of
teachers?

• Reporting should lead to self-scrutiny.
• In a team format, we look at ourselves as
  teachers and what were doing collectively. If
  there is one who is struggling with gaining
  progress with their students, what kind of
  support can be delivered to that teacher?
• Data can be a guide to help distinguish which
  teachers need additional support in the
  classroom. Not about teacher evaluation but about
  moving everyone forward in an appropriate way.

48
Understanding of data

• Students tend to pick it up pretty quickly
• Get students in touch with their own data and
  their own awareness of growth.
• Stay positive about their growth, even it they
  are only advancing in small increments. They will
  catch up eventually.
• Help parents understand the importance of
  assessment.

49
Using data vs. intuition

• A gut-feel approach does not work
• If decisions are made on an intuitive basis, you
  will not receive the greater outcomes achieved
  from using data.
• The data gives specific information on how well
  teachers are doing
• This can be very encouraging and empowering.
• The data-based approach allows for more efficient
  and effective use of resources, so students get
  what they need when they need it.

50
Challenges with English language learner data

• Everyone should be screened, regardless of being
  an ELL.
• As kids learn English, we can track that growth,
  especially in regards to what it looks like in
  reading and writing measures.
• Need to be careful in identifying ELLs who are
  at-risk
• Criteria used for native English speakers often
  do not apply for ELLs.
• Same measures can be used, but the decisions will
  vary.

51
Progress monitoring and No Child Left Behind

• Good screening and progress monitoring measures
  are good at predicting outcomes on the high
  stakes state tests
• Used as a forecast of how children will perform
  at the end of the year on high stakes tests
• Once instructional changes are in place for
  effective teaching, you can monitor learning
  outcomes of the students. Once you focus on what
  those learning outcomes are, the No Child Left
  Behind piece takes care of itself
• Helps to identify and implement effective
  instructional strategies

52
Criteria for universal screeners beyond the
timeframe

• The major criterion classification accuracy
• Ability to classify students accurately as at
  risk or not at risk for achieving given
  end-of-year outcomes
• Tools chart on the rti4success.org website
• You can see if the ratings bubble under
  classification accuracy is filled in for the
  tools that are present.
• You can click on the bubble to see the data
  submitted by the vendor to get a sense of its
  accuracy and hit rate for classifying students as
  at risk or not at risk.

53
Translating complicated data into user-friendly
language

• To help parents understand their childs
  progress, put data on a graph.
• The visual display helps to enhance full
  understanding.

54
Final thoughts with Mr. John Carruth

• Data-based decision making has allowed us to go
  from a subjective implementation of instruction
  to a very objective, scientific-based process
• Marriage between science and practice
• The biggest focus should be our students
  learning outcomes.

55
Final thoughts with Dr. Joseph Kovaleski

• This process can be performed in any school
  across the country, but it requires a deep
  understanding of its components.
• It must be done correctly and in an organized way
  in order for it to have the proven results.

56
Final thoughts with Dr. Lynn Fuchs

• Our goal is to identify those who are at-risk for
  long-term serious consequences.
• RTI is an important mechanism to enable general
  and special education to work proactively and
  collaboratively together with the goal of
  preventing children from exiting school without
  the academic skills they need to lead successful,
  healthy lives.

57

• For more information about how you can help
  struggling learners, please visit the RTI Action
  Network online at
• www.RTINetwork.org