Assessing Response to Intervention: A Practitioner's Perspective

1
Assessing Response to Intervention A
Practitioner's Perspective

• Joseph F. Kovaleski, D.Ed., Indiana University of
  Pennsylvania
• Richard E. Hall, Ph.D., Eastern Lancaster County
  School District
• David J. Lillenstein, Ed.D., Derry Township
  School District
• Jennifer A. Lillenstein, Ed.D., Cornwall-Lebanon
  School District
• Jason Pedersen, Ph.D., Cornwall-Lebanon School
  District

2
Agenda

• Orientation to RTI and Multi-tier Models
  (Kovaleski)
• A Three-tier Model for Preventing Reading Failure
  (J. Lillenstein J. Pedersen)
• Assessment Driving Instruction (D. Lillenstein)
• Precision Teaching (R. Hall)
• Discussion

3
IDEA 2004 ChangesSpecific Learning Disabilities

• The LEA shall not be required to take into
  consideration whether the child has a severe
  discrepancy between achievement and intellectual
  ability in oral expression, listening
  comprehension, written expression, basic reading
  skill, reading comprehension, mathematical
  calculation, or mathematical reasoning.

4
IDEA 2004 ChangesSpecific Learning Disabilities
(cont.)

• In determining whether a child has a specific
  learning disability, a local educational agency
  may use a process which determines if a child
  responds to scientific, research-based
  intervention.

5
PA Regulations

• Academic assessment
• Behavioral assessment
• Intervention based on assessment
• Assessment of response to intervention
• Lack of instruction or limited English
  proficiency
• Ability of the regular education program to
  maintain the student
• Activities designed to gain the participation of
  parents

6
NCLB AND IDEIA 2004

• Scientifically based instruction, curriculum, and
  interventions.
• Identification of learning problems early.
• Ongoing monitoring to determine impact of
  curriculum and instruction.
• Design and implement remedial and individualized
  intervention for those who dont respond.
• Inclusion of students in single accountability
  system.
• Documentation of student outcomes through AYP.

7
Its not just about identification

• IDEIA and NCLB are companion laws.
• They are mutually referential.
• Together, they envision a seamless system of
  supports, based on the use of scientifically
  based instruction, in both general and regular
  education.
• The mission is the development of proficiency in
  basic skills (particularly reading) for all
  students.

8
What Is Response to Intervention?

• A comprehensive, multi-tiered intervention
  strategy to enable early identification and
  intervention for students at academic or
  behavioral risk.
• An alternative to the discrepancy model for the
  identification of students with learning
  disabilities.

9
RTI is the practice of

• (1) providing high-quality instruction and
  interventions matched to student needs and,
• (2) using learning rate over time and level of
  performance to
• (3) make important educational decisions. (p.5)

National Association of State Directors of
Special Education (2005) Response to
Intervention Policy Considerations and
Implementation, p. 5
10
Key Characteristics of RtI

• Universal Screening of academics and behavior
• Multiple tiers of increasingly intense
  interventions
• Differentiated curriculum-tiered intervention
  strategy
• Use of evidence-based interventions
• Continuous monitoring of student performance
• Benchmark/Outcome assessment

11
Building the Infra-structure for RTI

• Using RTI requires an infra-structure of
  assessment and intervention techniques.
• We do not recommend implementing RTI if the
  infra-structure is not in place.
• Therefore, initial efforts should be placed on
  building the infra-structure.

12
The Multi-tier Process

• Ensures that scientifically validated
  interventions are used at a high degree of
  fidelity.
• Allows for the collection of valid, reliable, and
  functionally meaningful data that inform both
  identification and treatment decisions.

13
Interventions organized into a 3 tiered model.
Layers of intervention responding to student needs
TIER I
Each tier provides more intensive and supportive
intervention
TIER II
TIER III
Aimed at preventing reading disabilities
(Torgeson, 2004)
14
Interventions organized into a 3 tiered model.
Universal Screening Evidence-based core program
TIER I
Supplemental programming in the regular classroom
(push in)
TIER II
TIER III
Specialized targeted intervention (pull-out
groups)
15
A Three-Tiered Model for Preventing Reading
Failure

• Cornwall-Lebanon School District
• Jennifer Lillenstein, Ed.D., NCSP
• Jason Pedersen, Ph.D., NCSP

16
Its all about education

• (and maybe a little salesmanship.)

17
Our Road Map

• Analyze current district literacy assessments to
  determine comprehensiveness.
• Is there an overemphasis on sight word knowledge?
• Present DIBELS to principal.
• Collect DIBELS data.
• Analyze data for systemic weaknesses and present
  findings to principals and district
  administration.
• Present data to principal and teachers, as well
  as sharing it in report form with district
  administration.
• Analyze data relative to high stakes outcome
  measures.
• Present DIBELS and the data to the school board
  compared with outcome data.
• Develop a prevention model matching interventions
  with needs, and presented it to the district
  administration and school board.

18

• The way to effect system change is to provide the
  decision-makers with the best information so that
  they can make the best decision.

19
Present, Present, Present!

• School psychologists need to actively and
  continuously educate administrators about how
  appropriate, reliable and valid measurement can
  assist them in achieving building and district
  goals.
• School psychologists have a lot of information
  about measurement and analyzing data to make
  appropriate instructional recommendations for
  students and shouldnt keep it to themselves.

20
Present assessment ideas to

• Building principals (and teachers) at minimum.
• Directors of elementary education
• Superintendents
• School boards

21
Pilot the data collection.

• Offering to pilot the data collection process
  allows you get the data to make a case for
  change. As well as demonstrating the feasibility
  of the screening process.
• Analyze it and make connections to weaknesses in
  curriculum and relationship to high stakes
  assessments.

22
Analyze the data yourself.

• Identify strengths and needs.
• Identify trends and patterns in the data.
• Identify systemic weaknesses.
• Begin to formulate instructional recommendations
  at the system level.
• Know your empirically validated interventions!

23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Present data analysis.

• Teachers - Show classroom connection.
• Principal - Show PSSA/curriculum connection.
• District Administration - Show NCLB connection.
• Remember you are educating them about consuming
  data.

27
Our Road Map

• Analyze current district literacy assessments to
  determine comprehensiveness.
• Is there an overemphasis on sight word knowledge?
• Present DIBELS to principal.
• Collect DIBELS data.
• Analyze data for systemic weaknesses and present
  findings to principals and district
  administration.
• Present data to principal and teachers, as well
  as sharing it in report form with district
  administration.
• Analyze data relative to high stakes outcome
  measures.
• Present DIBELS and the data to the school board
  compared with outcome data.
• Develop a prevention model matching interventions
  with needs, and presented it to the district
  administration and school board.

28
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolSouth Lebanon Elementary Grade Second
Class All Assessment May Academic
Year 2002-2003

• Odds of being above benchmark with Terra Nova in
  Second Grade when Low Risk with ORF in May of
  Second Grade is 40 out of 46, or 86.
• Odds of being above benchmark with Terra Nova in
  Second Grade when At Risk with ORF in May of
  Second Grade is 2 out of 11, or 18.

29
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolSouth Lebanon Elementary Grade Third
Class All Assessment May Academic
Year 2002-2003

• Odds of being above benchmark with Pennsylvania
  System of School Assessment (PSSA) in Third Grade
  when Low Risk with ORF in May of Third Grade is
  28 out of 38, or 73.
• Odds of being above benchmark with Pennsylvania
  System of School Assessment (PSSA) in Third Grade
  when At Risk with ORF in May of Third Grade is 0
  out of 8, or 0.

30
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolSouth Lebanon Elementary Grade Fourth
Class All Assessment May Academic
Year 2002-2003

• Odds of being above benchmark on Terra Nova
  Assessment in Fourth Grade when Low Risk on
  September 2003 is 29 out of 32 or 91
• Odds of being above benchmark on Terra Nova
  Assessment in Fourth Grade when At Risk on ORF
  September 2003 is 5 out of 12 or 41

31
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolDistrict Grade Second Class
All Assessment May Academic Year 2003-2004

• Odds of being above benchmark on Terra Nova
  Assessment in Fourth Grade when Low Risk on May
  2004 is 63 out of 79 or 80
• Odds of being above benchmark on Terra Nova
  Assessment in Fourth Grade when At Risk on ORF
  May 2004 is 8 out of 37 or 22

32
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolSouth Lebanon Elementary Grade Second
Class All Assessment May Academic
Year 2003-2004

• Odds of being above benchmark with Terra Nova in
  Second Grade when Low Risk with ORF in May of
  Second Grade is 41 out of 48, or 85
• Odds of being above benchmark with Terra Nova in
  Second Grade when At Risk with ORF in May of
  Second Grade is 5 out of 22, or 23

33
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolSouth Lebanon Elementary Grade Fourth
Class All Assessment May Academic
Year 2003-2004

• Odds of being above benchmark on Terra Nova
  Assessment in Fourth Grade when Low Risk on
  DIBELS ORF in May of Fourth Grade is 39 out of
  44, or 88.64
• Odds of being above benchmark on Terra Nova
  Assessment in Fourth Grade when At Risk on
  DIBELS ORF in May of Fourth Grade is 6 out of 22,
  or 27.27

34
Dynamic Indicators of Basic Early Literacy
Skills, University of OregonScatter
PlotDistrict Cornwall-Lebanon
SchoolSouth Lebanon Elementary Grade Fifth
Class All Assessment May Academic
Year 2003-2004

• Odds of being above benchmark on PSSA Assessment
  in Fifth Grade when Low Risk on DIBELS ORF in
  May of Fifth Grade is is 45 out of 51, or 88.24
• Odds of being above benchmark on PSSA Assessment
  in Fifth Grade when At Risk on DIBELS ORF in May
  of Fifth Grade is 0 out of 10, or 0

35
Develop and Present recommendations for
addressing systemic weaknesses.

• Prevention, not remediation, is the most
  efficient and effective way to make lasting
  change.
• Emphasize that addressing needs in K and First
  grade will change child outcomes for the balance
  of their academic careers.
• Special Education is not the solution, regular
  education reform is!

36
Develop and Present recommendations for
addressing systemic weaknesses (cont.)

• In accordance with NCLB mandates data is used to
  drive instruction. Therefore in conjunction with
  the other performance assessments we as a team
  identify which students are at risk for reading
  failure and which skill is not developed. Then a
  research-based intervention is selected to
  address the area of need.

37
Pilot Interventions.

• Pilot scientifically-validated interventions.
• Collect progress monitoring data.
• Evaluate data to determine effectiveness of
  interventions.
• Present results to administration.

38
Interventions organized into a 3 tiered model.
Layers of intervention responding to student needs
TIER I
Each tier provides more intensive and supportive
intervention
TIER II
TIER III
Aimed at preventing reading disabilities
(Torgeson, 2004)
39
CLSD Three tier model
40
Types of Interventions by Tier
41
Kindergarten Tier 2 Students Progress (SL)
42
First Grade Tier 2 Students Progress (SL)
43
Responder.
44
Inadequate Responder.
45
Kindergarten Data
46
Kindergarten Data
47
Kindergarten Data
48
Kindergarten Data
49
First Grade Data
50
First Grade Data
51
First Grade Data
52
First Grade Data
53
First Grade Data
54
First Grade Data
55
Second Grade Data
56
Second Grade Data
57
References

• Adams M.J. (2002, November). The promise of
  speech recognition. PowerPoint presentation at a
  Focus on Fluency Forum. San Francisco, CA.
  Available at http//www.prel.org/programs/rel/flue
  ncy/Adams.ppt
• Cornwall-Lebanon School District. ( 2003-2004).
  DIBELS Data
• Foorman, B. R., Francis, D.J., Shaywitz, S.E.,
  Shaywitz, B. A., Flethcer, J.M. (1997). The
  case for early reading intervention. In B.
  Blachman (Ed.), Foundation of reading acquisition
  and dyslexia. (pp. 243-264). Maywah, NJ
  Erlbaum.
• Good, R. H., Kaminski, R. A. (Eds.). (2002).
  Dynamic Indicators of Basic Early Literacy Skills
  (6th ed.). Eugene, OR Institute for the
  Development of Educational Achievement.
• Kaminski, R.A., Good, R. H. (1996). Toward a
  technology for assessing basic early literacy
  skills. School Psychology Review, 25, 215-227.
• National Reading Panel (2000). Teaching children
  to read An evidence-based assessment of the
  scientific research literature on reading and its
  implications for reading instruction Reports of
  the subgroups. Bethesda, MD National Institute
  of Child Health and Human Development.
• National Research Council (1998). Preventing
  reading difficulties in young children.
  Washington DC National Academy Press.
• Thompson, M. Thomason, J. Learning Focused
  Schools.
• Torgesen, J. K. (2004a, April). Immediate,
  Intensive Interventions Their critical role
  within a whole school program to prevent reading
  difficulties. PowerPoint presentation at Florida
  State University and Eastern Regional Reading
  First Technical Assistance Center RF Leadership
  Meeting, Eastern Region.
• Torgesen, J. K. (2004b, September). Leaving No
  Child Behind in Reading What Every Teacher
  Should Know. PowerPoint presentation at Success
  for Struggling Readers, Aiken, SC.

58
Derry Township School DistrictHershey, PA

• Assessment
• Driving
• Instruction

David Lillenstein, Ed.D., NCSP Director of
Psychological Services [phone]
x5436 dlillenstein_at_hershey.k12.pa.us
59
2002-2003 The Beginning Stages

• Attended PaTTAN DIBELS training w Roland Good
• PaTTAN IU 15 consultants DIBELS and
  supplemental intervention ideas
• DIBELS pilot with special education
• DIBELS and kindergarten screening
• Nibbles w DIBELS trainings K-1 and SWAT
• DIBELS and Ch. 14 evaluations

60
But.

• Rigby Literacy
• Balanced Literacy
• Rigby consultants
• DRA comes to town

61
2003-2004 DIBELS Unfolds

• DIBELS assessments
• K screening . then 1st then 2nd then 3rd
• All staff trained in purpose and administration
• SWAT team conducted benchmark assessments
• (School Workers Armed w Timers)
• Special ed., Psychologists, or IST conducted
  progress monitoring
• Baseline data collection
• DIBELS reported in ERs and IEPs
• Began using RTI language and progress graphs

62
2003-2004 Reading Interventions Expand

• Special Education Tier 3 Outcome drove change
  at Tier 1 and Tier 2
• Read Naturally
• Project Read rebirth
• Corrective Reading
• REWARDS.
• Regular education
• Phonemic Awareness for Young Children
• Road to the Code
• Reading to Read (repeated readings)
• Read Naturally.

63
2003-2004 Preliminary Data Trends

• DIBELS Benchmark data showed need for
  intervention in all grades
• Staff began questioning Rigby BL
• Used data to challenge effectiveness of Rigby BL
• Letter names vs. Letter sounds
• Sight word vs. Decoding skills
• Low fluency rates
• No RTI or decrease in skills as text difficulty
  increases

64
2004-2005 - Turning Around the Boat

• New Director of Curriculum Instruction
• Shared data trends based on DIBELS
• Administrative Literacy meetings
• Literature Reviews to challenge BL
• Articles - Torgeson, Lyon, Moats, Shaywitz,
  Adams, Beck, Vaughan, Fletcher
• Books - Voice of Evidence (McCardle), Overcoming
  Dyslexia (Shaywitz), CORE materials
• National Reading Panel
• Websites fcrr.org, Univ. of Oregon sites, etc.
• NASP literature

65
2004-2005 - Continued

• Trainings/Consultations
• IDA conferences Philly and D.C.
• Susan Hall _at_ IU 13
• PDE Annual Conference - Hershey
• Project Read K-5 Joy Mackenzie
• PaTTAN workshops and teleconferences on RTI,
  progress monitoring, and reading
• Phone consults with CLSD
• Assessment Focus Group challenged DRA and
  discrete assessment reviewed literature,
  developed map
• DIBELS expanded to 4ththen 5ththen 6th
• Roger Farr - Past President of IRA brought in by
  Rigby but told what to say and what not to say
  !!!
• Rigby consultants censored contract
  terminated!!!

66
2004-2005 - More Changes

• Board presentations on DIBELS and Literacy
• I dont care what you do, just call it balanced
  literacy
• 3 Tier Intervention model discussions
• Elementary schedule changes
• Designated time for reading instruction flipped
  schedule
• Double Dipping model more need more time
• Classroom is 1st line of intervention huge role
  change!!
• Data-based decision making no more referrals
  students identified by a team for intervention -
  based on data
• IST and reading specialist changed to
  intervention specialist

67
2005-2006 Assessment Driving Instruction

• Assessment Map
• DIBELS, CORE Phonics, STAR, Project Read Tests,
  4Sight
• BL to bl - 5 Big Ideas of Reading becomes bl
• Reading Interventions
• Project Read and Read Naturally in every reg. ed
  classroom, PAYC, Road to the Code, Corrective
  Reading, etc.
• Homogeneous Grouping based on data (pod mtgs)
• Benchmark vs. Strategic vs. Intensive

68

• Data-based decision making Data Profiles
• Data Review Mtgs., Pod Mtgs., Intervention
  Planning mtgs.
• IST changed to Intervention Team for
  non-responders
• 2 Literacy Coaches K-3, 4-8

69
Trainings

• Susan Hall DIBELS data analysis and
  intervention
• Joy Mackenzie Double Dipping, Project Read,
  structured multi-sensory language instruction
• Literacy Coaches homogeneous grouping, small
  group individualized instruction, Project Read,
  PA training
• International Dyslexia Association National
  Conference, Jack Fletcher
• IU 13 Data Coach training, PVAAS
• Psychologists RTI, IDEIA consulting w other
  districts
• Johns Hopkins University 4Sight Assessments

70
Whats Next ?? (2005-2006 and beyond)

• The (Ch 14) Evaluation is the Intervention
• Or(if you prefer)
• The Intervention is the (Ch 14) Evaluation
• IQ only assessed when suspecting MR or Gifted
• Developing Line of Inquiry for Interventions
• Add to Tier 1 and Tier 2 tool box
• Piloting AIMSweb MAZE
• RTI at the middle school
• RTI for math, writing, gifted, SL
• consider core curriculum

71
Role Change for School Psychologists at Derry
Township School District

• Data coach
• Intervention Facilitator/Learning Facilitator
• ABE for at-risk Regular Ed. And Special Ed.
  Students
• Increased SAP presence K-12
• Disseminator of Research Reading, Learning,
  etc.
• Program Evaluator
• Derry Township School District Psychological
  Services

72
Conclusion

• If you do more of what youve always done,
  youll get more of what youve always got
• - Confucious, 2005

73
Precision Teaching

• Richard E. Hall, Ph.D.
• Eastern Lancaster County SD

74
What is fluency?
75
(No Transcript)
76
Fluency in Schools

• What is occurring in the acquisition stage?
• Is anything happening in the proficiency stage?

77
Precision Teaching and Fluency

• Fluency has an extensive research database from
  Precision Teaching.
• Precision Teaching - a method for systematically
  and precisely evaluating learning and
  facilitating decision making.

Kubina, 2004
78
Kubina, 2004
79
What is Fluency?

• Fluency is "the fluid combination of accuracy
  plus speed that characterizes competent
  performance.
• Binder also suggests fluency is a metaphor
  referring to a collection of observations about
  relations between response frequency and critical
  learning outcomes (p. 164).

Kubina, 2004
80
(No Transcript)
81
Kubina, 2004
82
Critical Learning Outcomes Associated with Fluency

• Students who lack endurance may engage in
• Distractibility
• Escape or avoidance behaviors
• Lack of interest

83
Skill See-say letter sounds /a/ /r/ /n/ /s/
/e/ /m/ /t/ Application to See-say
blending (sounding out) words) - ran, me, sat,
seat Skill See-say words in context (oral
reading) Application to Hear-say
comprehension questions (e.g., Who ran down the
street with Jack?)
84
How to Develop Fluency

• Question How to develop fluency?
• Answer Practice.

85
(No Transcript)
86
(No Transcript)
87
(No Transcript)
88
(No Transcript)
89
(No Transcript)
90
Kubina, 2004
91
Fluency

• Precision Teaching incorporates fluency into
  teaching.
• Precision Teaching uses the most efficient
  practice procedures to develop fluency.
• All students benefit from fluency.

92
Developing Behavioral Fluency through Precision
Teaching Effects on Student Performance
93
School Comparisons Blue Ball is the First School
to Incorporate PT into Tier 1 Instruction (2nd
Grade 2004-05 DIBELS Data)
94
School Comparisons1st Grade Nonsense Word Fluency
95
School Comparisons1st Grade Nonsense Word Fluency
96
School ComparisonsOral Reading Fluency 1st to
2nd Grade
97
Individual EDK Students Who received PT as a
Component of Fluency Building for Nonsense Word
Decoding
98
Individual ELK Students Comparison Nonsense Word
Fluency
99
Individual Case Background Grant

• C.A. 11-2, in 5th grade
• Home schooled parents report using a phonics
  program.
• Baseline ORF
• 19 wcpm in 5th grade level text
• 25 wcpm in 2nd grade level text
• Math skills are at or near grade level
• CTOPP Data
• Phonological Awareness Below Average
• Phonological Memory Below Average
• Rapid Naming Below Average

100
Individual Case Background Grant cont.

• Phonics Assessment
• Frustration with long vowel CVCE
• High Frequency Words 1st grade instructional
  level
• DIBELS
• Letter naming fluency Low Risk
• Nonsense word fluency Low Risk
• Phoneme segmentation Established
• ORF Instructional level 2nd grade 25 wcpm
• Goal Grant will read 2nd grade level text with
  at least 95 oral accuracy and at a rate of 110
  wcpm by May 2005 (DIBELS Benchmark for 2nd grade)
  

101
Intervention Plan (Essential Components) Grant

• Instructional match
• Horizons Direct Instruction group.
• Precision teaching focus
• High frequency words 2nd grade level
• CVCE decoding
• Words with b/d confusions
• Fluency practice with 2nd grade level text
• Charting - Short-term Goal-setting -
  Reinforcement
• Home Fluency practice

102
Individual LFT Case Grant
Precision Teaching Added to Plan
103
Grants ORF on 5th Grade Level Text
104
Follow-up
105
Grant is now a 6th Grade Student Fall 05
Assessments

• He is now close to Benchmark in oral read of 6th
  grade level material!
• He score in the Proficient range on a test of
  6th grade level reading comprehension (Degrees of
  Reading Power)!!
• He scored in the Proficient range on the end of
  5th grade PSSA!!!

106
Anecdotal Comments

• Improvements are noted in spelling
• Initial first grade September testing
• Improved short vowel
• Improved consonant blends
• Fewer reversals are noted in spellingincluding
  b/d reversals
• Reading Mastery Plus lessons take less time.

107
The Number of Initial District Generated MDT
Referrals K-6th Grade Since Initiation of LFT
Process
108
The of Students Achieving DIBELS Benchmark for
ORF in 1st Grade for 2003-04 2004-05
109
The of Students Achieving DIBELS Benchmark for
ORF in 2nd Grade for 2003-04 2004-05
110
The of Students Achieving DIBELS Benchmark for
ORF at 3rd Grade for 2003-04 2004-05
111
The of Students At-Risk according to DIBELS
ORF at Grades 1-3 for 2003-04 2004-05
112
Contact Information

• Dick Hall dick_hall_at_elanco.k12.pa.us
• 717-354-1514
• Joe Kovaleski jkov_at_iup.edu www.coe.iup.edu/koval
  eski
• 724-357-3785
• David Lillenstein DLillenstein_at_hershey.k12.pa.us
• 717-534-2501
• Jennifer Lillenstein Jlillenstein_at_clsd.k12.pa.us
• 717-270-7227
• Jason Pedersen Jpedersen_at_clsd.k12.pa.us
• 717-273-4546