EvaluatingProgress Monitoring

1
Evaluating/Progress Monitoring

Problem Identification What is the problem?
Problem Analysis Why is it happening?
Progress Monitoring Did it work?
Intervention Planning What should be done about
it?
2
Some Universal Standard Intervention Tasks

• Develop Intervention
• Obtain Resources/Provide Training for
  Intervention
• Support and Implement Intervention
• Observe Fidelity of Implementation of Current
  Intervention
• Set Goal and Develop Progress Monitoring Plan
• Collect Progress Monitoring Data
• Decide If Educational Need and Benefit Warrants
  a Modified
• Intervention Plan or Potential Need for an
  IEP
• Districts adopt, adapt, drop, or add the tasks to
  fit their Problem-Solving model.

3
What is Progress Monitoring?A KEY Component of
an RtI Model

• Progress monitoring is a scientifically based
  practice that teachers can use to evaluate the
  effectiveness of their instruction and
  intervention plans
• the use of progress monitoring results in more
  efficient and appropriately targeted
  instructional techniquesthat move all students
  to faster attainment
• Progress monitoring is a significantly
  underutilized practice that works

From the US Department of Education, Office of
Special Education Programs Technical Assistance
and Dissemination Center
4
We Should Use Scientifically Based Progress
Monitoring Tools

• www.studentprogress.org

5
Most Tools that Meet Standards are Members of the
Curriculum-Based Measurement (CBM) Family Aimsw
eb Or DIBELS
6
Types of Progress Monitoring
7
We Take Our Scientifically Based Progress
Monitoring (CBM) and Use it for PM Across 3 Tiers
and Universal Screening
8
At each tier, decisions always based on

• Decision Rules and..
• Progress- Is their Benefit from the intervention?
• Discrepancy-Is the student reducing the
  Discrepancy/Need between their performance and
  that of average peer?
• What are Needs? Can they be met in general
  education?

9
Data Decision Rules
3-5 Points Above the Aimline IF 3-5 Data Points
are Consistent AND Above the Aimline Raise
Goal, Consider Need for Program, Fade the
Intervention, or that Problem is Resolved
10
Data Decision Rules
3-5 Points Below the Aimline IF 5 Data Points
are Consistent AND Significantly Lower than the
Aimline CONSIDER Changing the Intervention,
Especially if Integrity of Intervention is High,
or Cycling back through the Problem Solving
Process
11
Data Decision Rules
If LOTS of Variability in the Data Points
CONSIDER Examining Integrity of Intervention,
Differences in Assessment Materials, Tester, or
Influence of Student Motivation OR whether
student is still in emerging stage of skill
development
12
How Much Data Do You Need for Trendlines Visual
Analysis?
Generally, MORE Data (Getting to at least 5-7
data points) is BEST. Make Decisions About
Progress More Accurate But Come at a Cost of
Lost Critical Time So, Increase Data Collection
Frequency When These Issues are Apparent Severe
Problems, New Interventions Thinking
Required Sometimes Ineffective Interventions are
Obvious!
13
Decision Rules Linking RtI to Intervention
Decisions

• Positive, Questionable, Poor Response
• Intervention Decision Based on RtI (General
  Guidelines)
• Positive
• Continue intervention until student reaches
  benchmark (at least).
• Fade intervention to determine if student has
  acquired functional independence.
• Questionable
• Increase intensity of current intervention for a
  short period of time and assess impact. If rate
  improves, continue. If rate does not improve,
  return to problem solving.
• Poor
• Return to problem solving for new intervention

14
At Tier 3, you analyze the data the same way as
Tier 2..only more frequently

• 1. IS THE STUDENT MAKING PROGRESS TOWARD THE
  GOAL? BENEFITING
• 2. IS THE STUDENT DECREASING THE DISCREPANCY/NEED
  BETWEEN HIS/HER PERFORMANCE AND RATE OF
  IMPROVEMENT AND THAT OF GENERAL EDUCATION PEERS?
• 3. DOES THE STUDENT HAVE NEEDS BEYOND THE SCOPE
  OF THE GENERAL EDUCATION INSTRUCTIONAL PROGRAM?

15
A. Is the student making progress toward the goal?
Positive
Poor
Questionable
16
B. Is the student decreasing the discrepancy
between him/her and the general education peers?
Questionable
Poor
Positive
17
C. NEED Is the plan able to be maintained in
the general education setting ?
Poor
Positive
Questionable
18
Is the student Benefiting? Is there Progress?
19
Is the student Benefiting? Is there Progress?
20
Data on Closing of the Performance
Discrepancy/Need? Benefit/Progress?
21
Ex 3rd Grade PM (Tier 3)Benefit? Progress?
22
Student Progress Monitoring Is the student
making progress from the intervention? Is he
benefiting?
23
Significant Educational Benefit/Progress
Maintain the current intervention.
Discrepancy/Need? IS THIS STUDENT REDUCING THE
DISCREPANCY/NEED BETWEEN HIMSELF AND GRADE LEVEL
PEERS?
24
Educational Benefit/Progress?
25
Ex 2nd Grade PM (Tier 3) Benefit? Progress?
26
Benefit? Progress?
Tier 3
Tier 1
27
Is the student benefiting (making progress?) and
are they reducing the discrepancy/need?
28
Is the student Progressing? Benefiting? Is the
student benefiting from the intervention?Is the
students rate of improvement sufficient?Does
the intervention need to be modified?
Intervention line
Goal/Aimline
Change the intervention plan immediately.
Trendline
Data points
Error rates
29
Educational Benefit?
30
Reducing Discrepancy/Need?
31
Significant Educational Discrepancy/Need, No
Significant Educational Benefit/Progress Change
the Program

IS THIS STUDENT REDUCING THE DISCREPANCY BETWEEN
HIMSELF AND GRADE LEVEL PEERS?
32
(No Transcript)
33
The Problem Solving Model Also Has 4 Key
Beliefs/Cornerstone

• General education, even with quality tools,
  training, and support, cannot meet the
  educational and behavioral needs of all students.
• Some students
• --will not attain what wed like
• --unless substantial changes are made in their
  educational programs
• It is a Value Judgment as to when performance
  discrepancies are so severe that additional
  services beyond general education are warranted
• Because effective interventions cannot be
  predicted with certainty, we must try out our
  programs and monitor progress to see if they work

S.L. Deno, 1985 1986 1989 1995 2000
34
SHIFTING GEARS
35
COMPREHENSIVE EVALUATION

• For an RtI CASE STUDY EVALUATION

36
Special Education ConsiderationDomains completion

• Note Federal Regulation,
• (g) The child is assessed in all areas related
  to the suspected disability, including, if
  appropriate, health, vision, hearing, social and
  emotional status, general intelligence, academic
  performance, communicative status, and motor
  abilities. (34 C.F.R. 300.532) (emphasis added)
• Meaning? Note if appropriate

RESCHLY, RTI, 2006, NASP
37
Increase Support by Decreasing Testing
IDEA 2004 Eligibility Determination Use
existing evaluation data (including that
provided by the parents) current classroom-based
assessments and observations, and teacher and
related service providers observation. on the
basis on that review, and input from the childs
parents, identify what additional data, if any,
are needed to determine special education
eligibility needs Sec. 614 (c) (1) (A) (B).
(emphasis added).
RESCHLY, RTI, 2006,NASP
38
Traditional Assessment Core of Comprehensive
Evaluation

• The core procedure of a traditional
  comprehensive evaluation of LD is an objective,
  norm-referenced assessment of the presence and
  severity of any strengths and weaknesses among
  the cognitive processes related to learning in an
  academic area.
• Deficient-? No relationship between these
  strengths and weaknesses and accurate diagnosis
  or effective interventions
• Shift from Whats wrong with student? to what
  can we vary in instruction, curriculum, and
  environment to lead to effective interventions

RESCHLY, RTI, 2006, NASP
39
RTI Comprehensive Evaluation Core

• RTI Core is the analysis of achievement and
  behavior, using direct measures in natural
  settings leading to,
• Precise analysis and measurement of skills levels
• Precise analysis of alterable conditions- ICEL
• Application of powerful instructional
  design-evidence of research-based programs
• Assessment of rate of learning/benefit progress
  monitoring with formative evaluation
• Decisions based on intervention outcomes-
  reducing performance discrepancy/need
• Assessment focused on questions pertaining to
  what interventions would be effective

RESCHLY, RTI, 2006, NASP
40
RTI Comprehensive Evaluation Core

• POSSIBLE TOOLS
• RIOT (Review, Interview, Observe, Test/Assess)
• ICEL- Instruction, Curriculum, Environment,
  Learner
• Performance Discrepancy, Progress Data
• Assess Intervention for Scientific Basis and
  Implementation Integrity
• SLA, CBE
• Other? Any other assessment data should focus
  only on questions related to answering what
  interventions will be effective for student
• Adverse Effect and Exclusionary Criteria

41
Conducting a Survey Level Assessment
Students are tested in successive levels of
general curriculum, beginning with their current
expected grade placement, until a level at which
they are successful is determined.
42
Johns Problem Identification Data
43
Johns Survey Level Assessment (SLA) Results
44
SLA/CBE-R Modified form
45
From the Federal RegulationsGoogle IDEA 2004
Regarding Processing testing
46
From the Federal RegulationsGoogle IDEA 2004
Regarding Processing testing/cont.
47
From the Federal RegulationsGoogle IDEA 2004
Regarding IQ testing
48
SECTION 6. DETERMINING ELIGIBILITY

• How to do it- by using
• Progress
• Discrepancy Criteria
• Need

49
Lets Look at a Case
50
FRAMEWORK for READING INTERVENTIONS
SRA DI PROGRAMS- READING MASTERY HORIZONS,
CORRECTIVE READING SOAR TO SUCCESS EAROBBICS GREAT
LEAPS /SLANT REWARDS, LIPS 6 MIN.
SOLUTIONS REPEATED PHRASES REPEATED
READINGS BRINGING WORDS TO LIFE, CORE VOCAB.
HANDBOOK ELEMENTS OF READ.-VOCAB.
TIER III. Highly at-risk students Intensive
interventions
VOCABULARY THROUGH MORPHEMES K PALS, JOLLY
PHONICS/GRAMMAR M. HEGGERTY PROGRAM/ 1ST GR.
PALS EAROBICS GREAT LEAPS/ SLANT REWARDS, QUICK
READS 6 MINUTE SOLUTIONS REPEATED PHRASES,
REPEATED READINGS BRINGING WORDS TO LIFE, CORE
VOCAB. HANDBOOK. ELEMENTS OF READING-VOCABULARY ME
TACOGNITIVE STRAT.- COLLABORATIVE STRATEGIC
READING, SOAR TO SUCCESS
TIER II. At-risk students- Supplemental
interventions
Harcourt Trophies VOCABULARY THROUGH MORPHEMES K
PALS, JOLLY PHONICS, JOLLY GRAMMAR, M. HEGGERTY
PROGRAM/ 1ST GR. PALS 6 MINUTE SOLUTIONS, QUICK
READS, BRINGING WORDS TO LIFE, CORE VOCAB.
HANDBOOK ELEMENTS OF READING-VOCABULARY, EARLY
SUCCESS/ SOAR TO SUCCESS METACOGNITIVE STRAT.,
COLLABORATIVE STRATEGIC READING,
TIER I.
51
(No Transcript)
52
Whats the Problem?Problem IdentificationIs
there a discrepancy in performance between what
is expected and what is occurring?Between
students performance and that of peers? IS
THERE AN EDUCATIONAL NEED/DISCREPANCY?
53
(No Transcript)
54
What are We Going to Do About It?Intervention
Planning

• Tier 1 90 minutes of language arts block with
  Harcourt Trophies, including 1st Gr. PALS and
  Michael Heggerty

• Tier 2. (Oct. )An additional 30 min.
• per day of 1st Gr. PALS in a small
• group setting

TIER 1 2 INTERVENTIONS.
55
(No Transcript)
56
MHeggerty
1st Gr. PALS
57
Did Michael Benefit from the
intervention plan? Did he respond positively to
the interventions?What was his rate of
improvement?
Progress MonitoringDid It Work?
58
Is Michael Benefiting from Tier 1 and Tier 2
Interventions? Discrepancy in Rate of
Improvement? Performance Discrepancy?
59
Is Michael Benefiting from Tier 1 and Tier 2
Interventions? Discrepancy in Rate of
Improvement? Performance Discrepancy?
60
Problem Analysis Why is it Happening?
PROBLEM ANALYSIS
Learning problems results from a complex
interaction between several factors instruction,
curriculum, the environment and learner
characteristics. (Howell, 1993)
ICEL FACTORS
61
Problem Analysis Michael X
62
TIER 3 INTERVENTION
Tier 3. (Nov.) An additional 60 minutes per day
of Great Leaps (3 times daily for 20 minutes
each) and 15 minutes daily of Earobics
63
and alpha. principle
Great Leaps
MHeggerty
64
Is Michael Benefiting from this intervention
plan? Does a performance discrepancy Indicate an
educational need?
65
Entitlement Decision RtI Model
Using Response-to-Intervention data for
educational decision making, including
determining eligibility.
66
Conditions for Special Education Entitlement
1. Progress/Benefit

• Educational Progress - previous interventions
  have not sufficiently improved a students rate
  of learning and additional resources are needed
  to enhance student learning or the interventions
  that have sufficiently improved the students
  learning are too demanding to be implemented
  without special education resources

67
Conditions for Special Education Entitlement 2.
Discrepancy/Need

• Discrepancy - given equal or enhanced
  opportunities, the students current level of
  performance and/or rate of improvement is
  significantly lower than typical peers or
  identified standards

68
Conditions for Special Education Entitlement 3.
Instructional Needs

• Instructional Needs - instructional needs have
  been identified that are beyond what can be
  provided in general education. This is evident
  when curriculum, instruction or environmental
  conditions need to be very different for the
  student as compared to the needs of other
  students in the general education environment.

69
Entitlement Decision
A. Educational Progress
B. Discrepancy
C. Instructional Needs
Entitlement Decision



70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
OPTIONAL IQ Ability-Achievement Discrepancy
Chart Data
75
ANOTHER READING/ACADEMIC EXAMPLE

• JOHNNY

76
(No Transcript)
77
JOHNNYS CASE STUDY

• LISTED RESEARCH BASED INTERVENTIONS AND EVIDENCE
  OF IT.

78
(No Transcript)
79
Entitlement for Special Education
Assessment and Progress Data From Problem Solving
Process
Educational Progress
Discrepancy
Instructional Needs
Convergence of Data from a Variety of Sources
SAME CRITERIA FOR RE-EVALS
80

• The ultimate test of the value of special
  education is that, once identified, children
  close the gap with their peers.
• Make Special Education worth getting.
• (Presidents Commission, 2002)

81
Effectiveness of LD Programs based on Discrepancy
Model

• Special education placements tend to stabilize
  the reading growth of students with reading
  disabilities rather than accelerate it. (Vaughn,
  1998, Moody, 2000)
• Acceleration rates about .04 SD/year. It will
  take 8 years to move from 5th to 9th percentile
  (Torgeson, in press Hanushek, 1998)
• Students who enter special education 2 years
  below age mates can be expected to maintain
  disparity or fall farther behind.
• Its the nature of the program more than the
  label that makes the difference.

82
Tier 3 Frequent Monitoring toward
Individualized or IEP Goals
83
Significant Educational Need, No Significant
Educational Benefit Change the Program

IS THIS STUDENT REDUCING THE DISCREPANCY BETWEEN
HIMSELF AND GRADE LEVEL PEERS?
84
Annual Review Benchmark and IEP ProgressBenefit
in Reading
85
SHIFTING GEARS
86
TYING IT ALL TOGETHER
87
The possible competing goals of Special Education

• Accommodations/Modifications
• Interventions

88
Problem Solving Process
Problem Identification and Problem Analysis
What is the problem and why is it happening? What
are our hypotheses?
Evaluate the Intervention
Intervention Development
Did our plan work?
What are we going to do?
Interventions, not accommodations!
Intervention Implementation
Carry out the intervention
89
Definitions

• Accommodation - a change made in the students
  environment, requirements, and/or resources to
  help the student be more successful.

90
Definitions

• Intervention - a specific skill-building strategy
  implemented and monitored to increase student
  skill in a specific area (academic or behavioral).

91
My favorite definition.

• A true "quality" intervention would be comprised
  of 3 components
• 1. Teaching the skill/replacement behavior
• 2. Reinforcing/motivating student to use the new
  skill
• 3. Accommodations and modifications to increase
  the student's likelihood of success in using that
  new skill

92
Time to assess and work in team

• COMPLETE SURVEY
• Self Study
• Leads to Staff Development plan
• Think about
• Consensus Building
• Infrastructure
• Implementation
• And PS at 3 tiers.
• Goals and report back to group.

P. 4
93
State Required District Plans will likely need to
include

• General and Special Ed Collaboration
• Needs Assessment to identify what resources are
  needed
• Effective Data Collection Process
• Review of existing district practices/policies
  that
• support RtI
• Establishment of Process for Interventions
• Eligibility Process that incorporates RtI
• How does RtI fit into School Improvement Plans
• How this plan fits in the district Professional
  Development Plan

94
P. 5
95

• Identify the consensus level among staff for
  implementing RtI/PS.
• Determine next steps with leadership team.
• Plan to support this change initiative in the
  school.

96
Review what youve learned

• TIER 1
• TIER 2
• TIER 3
• THE TEAMING, THE DATA, THE INTERVENTION SYSTEM

97
You may need Boosters for Specific Skills

• Consensus Building/Leadership Characteristics of
  SBRR
• Role Changes 5 Big Areas of Reading
• Creating Risk Triangles Comprehensive
  Evaluation
• Problem Identification Determining Eligibility
• and Analysis Report Writing/RtI tab
• Progress Monitoring Instruction Planning Form
• Goal Setting Problem Analysis Wsheet
• Data Evaluation Progress, Treatment Integrity
  forms
• Discrepancy, Need Self Study
• Effective Teams and Teaming Professional
  Develop. Plan
• Problem Solving Steps and Process

98

• SOME PARTING WORDS OF WISDOM

99
An RtI Vision
Efficient Teaming and Problem Solving at each
Tier
Data-Based Decisions at each Tier
Intervention-rich environment at each Tier
100
To do this will take new learning for everyone
101
To do this will take courage
102
To do this will take trust
103
We will make mistakes and fix them along the way
104
This is our opportunity

• To collectively make a significant difference in
  our educational system..in school reform and
  improvement

Many people, 2007
105
Finally and againLessons from Oz

• To do this
• It take courage.
• It takes knowledge.
• It takes heart.
• We have all that is required to support a system
  of change and success for all children. We should
  make this our work, our passion, and our purpose.

106
Again, you have been a wonderful audience

• Good luck with your work!
• I CAN BE REACHED AT
• barbcurl_at_ladse.org

Thank You!
107
Excellent Websites

• NEW Doing What Works for ELL
• http//dww.ed.gov/learn/learn.cfm?PA_ID6T_ID13
  P_ID20rID1
• Illinois School Psychologists Association
  www.ilispa.org/consumer resources
• Florida Center for Reading Research www.fcrr.org
• Oregon Reading First Center reading.uoregon.edu
• Texas Center for Reading and Language Arts
  www.texasreading.org
• Consortium on Reading Excellence (CORE)
  www.corelearn.com
• What Works Clearing House http//www.whatworks.ed.
  gov/
• National Technical Assistance Center onPositive
  Behavioral Interventions and Supports (PBIS)
  www.pbis.org
• National Center on Student Progress Monitoring
  www.studentprogress.org