Child Outcomes, Problem Solving and RTI in the Future of School Psychology

1
Child Outcomes, Problem Solving and RTI in the
Future of School Psychology
Daniel J. Reschly dan.reschly_at_vanderbilt.edu Penn
sylvania Department of Education Hershey, PA
February 23-24, 2005
2
Overview

• Focus of assessment, classification and
  intervention?
• Child outcomes
• Reject claims not supported by data
• Rigorous experimental or quasi-experimental
  designs
• Scientifically-based interventions
• Work for groups
• Need to be tested at individual level

3
What To Do With Egbert??

• 1st Grade, falling behind in reading
• Slow progress compared to peers
• Likely to miss benchmarks related to passing 3rd
  Grade reading test
• Distractible, inattentive, disruptive
• What next??????
• Sound Familiar

4
Egbert in the Traditional System

• Refer Egbert
• Preferral intervention (check a box)
• Comprehensive Evaluation-Battery of Tests,
  common battery?
• Assessment largely outside of the natural context
• Dubious generalizations from test behavior to
  classroom
• Eligibility assessment unrelated to intervention
• Team decision-making

5
Egbert in the Traditional System

• LD eligibility
• Discrepancy rarely attained at early grades
• Refer-test 1st ditto in 2nd ditto in 3rd
• Wait to fail
• Eligible (finally) in 3rd or 4th grade
• Quality of prereferral interventions ?
• Value of comprehensive evaluation in design of
  instruction and behavioral interventions?

6
Old Assumptions About Special Education and
School Psychology re High Incidence Disabilities

• ? Disabilities Inherent in Individual?-BUT
• Context and prevention are crucial
• ? Identify and Treat Underlying Causes-BUT
• Failure of process training
• ? Prescribe Methods that Capitalize on Strengths
  and Avoid Weaknesses-BUT
• Failure of Aptitude by Treatment Interaction
  Research and Practice
• Matching does not work

7
Old Assumptions About Special Education and
School Psychology re High Incidence Disabilities

• ? Unique Treatment Methods by Disability-
• Same methods work for virtually all SWD,
  especially LD, ED, EMR
• ? Unique Teacher Training by Disability-BUT
• Same methods for LD, ED, EMR Generic licensure
• ? IQ Essential to Accurate Classification-BUT
• Same kids found with problem solving processes
  and measures

8
Old Assumptions About Special Education and
School Psychology re High Incidence Disabilities

• ? Finding the right kids for special education is
  very difficult ?
• Right kids easy to find, but doing something
  significant about their educational trajectories
  is very difficult

9
Are the High Incidence Categories Meaningful?

• PA
• MR 0.4 (NJ) to 3.0 (WV) 7Xs 1.4
• ED 0.1 (AR) to 2.0 (MN) 20Xs 1.2
• LD 2.7 (KY) to 9.3 (RI) 3Xs 6.9
• Sp/L 0.8 (HI) to 3.8 (WV) 5Xs 2.0
• OHI 0.1 (MS) to 2.1 (RI) 21Xs 0.3
• All 9.7 (CO) to 17.9 (RI) 1.8Xs 12.5
• What Accounts for the Differences?? Also
  differences between LEAs within states
• 2002-2003, age 6-17, school enrollment, Table
  AA-13, www.IDEAdata.org

10
What Does Work? Placement??

• Treatment/Intervention aEffect Size
• EMR/Sp. Ed. Placement (IQ 50-75) -.14
• Slow Learner/Sp. Ed. IQ 75-90 -.34
• SLD and E/BD Sp. Ed. .29
• Traditional Placement Practices???
• Weak Relationships to Outcomes
• Note Effect size is expressed in SD units,
  analogous to a z-score

11
Centrality of ATI Foundation

• Diagnosis Focused on Level and Pattern of
  Performance
• Match Up Teaching Methodology to Aptitudes
  Avoid Dead Tissue (Reynolds, 1992)
• Many Applications in Special Education (e.g...,
  Neuropsychology, Learning Styles, Multicultural
  Teaching Methodology, Simultaneous vs Sequential,
  Information Processing Modality, Right Hemisphere
  -- Left Hemisphere, visual vs Auditory Learners

12
Aptitude by Treatment Matching???

• Treatment/Intervention aEffect Size
• Modality Matched Instr. (Aud.) .03
• Modality Matched Instr. (Vis.) .04
• Simultaneous/Successive .00
• Right Brain/Left Brain .00
• Cultural Leaning Style .00
• NOTHING FOR KIDS
• FEEL GOOD ASSESSMENT

13
Neuropsychology in Special Education and School
Psychology

• Distinguish between neuropsychology and
  neuroscience
• Neuropsychology is dependent on psychometric
  profiles
• Difference scores are less reliable
• Scatter is normal
• Base rates for profile variations
• Flat profiles are atypical
• Nearly all have profile variations

14
Old vs New Models of Brain and Learning
15
Neuroscience

• Instruction in decoding changes brain functioning
  on fMRI
• Neurological functioning more dynamic, less
  static
• Little practical application of fMRI to current
  school psychology practice
• No unique LD markers!!

16
Another Dry Hole Cognitive Processes in LD
Identification

• Why? Match LD definition??
• Benefits to kids?? (vs test authors)
• Process training? Like ATI, no benefits
• Diagnose LD? Abandoned by OSEP in 1977 No
  reason to reinstate
• OSEP Federal LD classification criteria have
  NEVER required cognitive process assessment 34
  CFR 300.540

17
What Does Work Some Examples

• Treatment Effect Size
• Applied Behavior Analysis. 1.00
• CBMGraphingFormative
• Evaluation .70
• CBMGraphingFormative
• EvaluationReinforcement 1.00
• Many other effective instructional and behavior
  change principles

18
School Psychology Transition From Correlational
to Experimental Science

• Cronbach, 1975, "One monitors responses to the
  treatment and adjusts it .." (p. 126).
• Problem Solving-self correcting methodology.
  Scientific method
• Steps and components to follow
• Change, not prediction
• Disconfirm predictions

19
Correlational to Experimental

• Problem Solving
• Systematic, data-based
• Vehicle for application of knowledge to learning
  and behavior problems
• Self-correcting methodology
• Single-subject, time series designs
• Applicable during prevention, early
  identification and intervention, and special
  education treatments

20
Part III Policy and Legal Influences

• Presidents Commission on Excellence in Special
  Education (2002) report, A New Era Revitalizing
  Special Education for Children and their
  Families.
• http//www.ed.gov/inits/commissionsboards/whspecia
  leducation/reports.html National Academy of
  Sciences/National Research Council Panel Report
• National Academy of Sciences/National Research
  Council Panel Report
• http//www.nap.edu/catalog/10128.html
• LD Summit Researchers Recommendations (Bradley et
  al., 2002)

21
Commonalties in Policy

• Accountability-Improved results for all students
  and better results are possible!!
• Multiple tiers of intervention defined by
  increasing measurement precision and intervention
  intensity
• Scientifically-based interventions
• Progress monitoring with formative evaluation
• Decisions at all levels driven by child response
  to intervention
• Problem Solving

22
IDEA 2004 Reauthorization

• General Themes
• Accountability for results
• Integration with NCLB
• Flexibility increased for SEA and LEA
• Prevention emphasized, particularly with high
  incidence disabilities in early grades
• LD changes
• Disproportionality
• Regulations will be critical

23
IDEA Reauthorization LD

• Re SLD, the local educational agency shall not
  be required to take into consideration whether
  the child has a severe discrepancy between
  achievement and intellectual ability
• The LEA may use a process which determines if a
  child responds to scientific, research-based
  intervention
• RTI

24
LD Congressional Intent

• House Report
• LEAs can continue to use IQ-achievement
  discrepancy acc. to NASP article ????
• RTI is recommended
• DE directed to conduct research on RTI and other
  options
• Senate Report
• Encourages states to develop research-based
  models of LD identification
• Produce greater consistency in LD
• Discourages continued use of IQ-Ach discrepancy

25
Prevention-Early Intervention

• LEA can use 15 of federal IDEA funds to support
  prevention and early identification-treatment
• Purpose minimize over-identification and
  unnecessary sp ed referrals
• Provide academic and behavioral supports
• Supports professional development and provision
  of interventions including early literacy
  instruction
• Scientifically-based
• Golden opportunity for schools!!

26
Alternative Criteria

• Large Discrepancy in Relevant Domain(s) of
  Behavior Using Direct Measures in the Natural
  Context with Local Norms
• Insufficient Response to High Quality
  Interventions in General Education
• Documented Adverse Impact on Education
• Documented Need for Special Education
• Exit Criteria
• If LD retained, add exclusion factors

27
Multi-Tiered Academic Interventions of Increasing
Intensity and Measurement Precision

• Academics (Empirically validated instruction)
• Level I General Education All students
• Level II Standard Protocol Treatments Small
  group tutoring (3-4) in general education
• Level III. Problem Solving Individualized
  interventions in general education leading to, in
  some cases, sp ed eligibility
• Level IV Special education More intense
  services brought to student

28
Multi-Tiered Academic Interventions of Increasing
Intensity and Measurement Precision

• Behavior-Empirically validated
• Level I General Education School wide positive
  discipline
• Level II Standard Protocol Treatments
  Classroom organization and management
• Level III. Problem Solving Targeted individual
  interventions in general education
• Level IV Special education More intense
  services brought to the students

29
Issues Quality of problem solving IF used for
eligibility

• 1. Precise Definition of the Problem in Terms of
  Observable Behavior
• 2. Valid and Reliable Measure of the Behavior in
  the Natural Setting
• 3. Validate the Existence of the Problem
  Estimate the Severity (Age Norms/Peer
  Comparisons)
• 4. Establish Intervention Goals in Terms of the
  Target Behavior
• 5. Analysis of Antecedent (including prior
  knowledge), Situational, and Consequent
  Conditions

30
Issues Quality of problem solving IF used for
eligibility

• 6. Formulate an Intervention Plan Based on
  Principles of Behavior Change or Instructional
  Design
• 7. Systematic Implementation of the Intervention
  with Treatment Integrity and Frequent Monitoring
  of Progress
• 8. Revision of the Intervention as Needed
  According to Progress Toward Goals
• 9. Evaluation of the Intervention with Further
  Problem Solving as Needed
• 10. Systematic Plan for Maintenance and
  Generalization of Behavior Change

31
PROBLEM SOLVING CHART
Does the damn thing work?
Yes
No
Dont mess with it!
Did you mess with it?
You Idiot!
Yes
No
No
Will you catch hell?
Hide it!
Yes
Does anyone else know?
No
Yes
You poor slob!
Ignore it
Can you blame somebody else?
No
Yes
NO PROBLEM
32
What To Do With Egbert??Problem Solving

• Step 1 Parent Involvement
• Inform
• Seek consent
• Describe subsequent steps and procedures
• Invite participation
• Involve, at parents discretion, in problem
  solving
• Schedule meeting

33
Egbert RTI Problem Solving

• Step II Problem Statement
• Interview significant others with questions and
  summary statements
• Specific behaviors, observable and measurable,
  domains of behavior, settings
• Alterable characteristics of student and
  environment
• Avoid discussion of internal child deficits

34
Egbert RTI Problem Solving Step II Problem
Statement

• Low reading based on poor oral reading fluency in
  tutoring sessions
• Poor decoding skills based on ORF observations
  and criterion referenced measures
• Social behaviors that interfere in general and
  special education, inattentive, non-compliant
  defined

35
RTI Problem Solving Step IIISystematic Data
Collection

• Brainstrom appropriate measures
• Accurate quantification of difference between
  current and desired levels,
• Direct,
• Match setting and domain,
• Frequent and repeated measurement
• Sensitive to growth in academic and/or behavioral
  skills
• What measures are appropriate?

36
RTI Problem Solving Step IIISystematic Data
Collection

• RIOT Principle
• Review records
• Interview significant others
• Observe
• Test
• Multiple Informants
• Multiple Settings
• Convergent Validity Principle

37
RTI Problem Solving Step IIISystematic Data
Collection

• Reading-CBM
• ORF measures-validity? Check it.
• Decoding criterion referenced, specify skills
  mastered and needed
• Social Behaviors
• Parent and Teacher Rating Scales-SSRS
• Systematic behavior observation, peer referencing
• Event recording of non-compliance and work
  completion

38
Behavioral Assessment and CBM Measures

• Focused on determination of change
• Formative evaluation critical
• Tied to effective practices and better outcomes
• Applications in general, remedial, and special
  education
• Controversial Useful in identification of SWD

39
Why Behavior Assessment (including CBM)

• Determine current levels in academics and
  behavior degree of need
• Monitor progress, assess change
• Foundation for formative evaluation-improving
  interventions
• Determine success of interventions
• Decisions based in child response to interventions

40
Foundations of CBM

• Denos Advance
• Brief samples of behavior
• Use of oral reading fluency samples
• Production per unit of time
• Fluency and accuracy combined
• Words read correct per minute
• Math-digits correct
• Spelling-letters correct
• Content drawn from curriculum

41
Prior Barriers to CBM Use

• Cumbersome for practitioners, developing own
  passages
• Conceptual issues Passages from curriculum or
  generic passages?
• Teachers concerns about comprehension Word
  calling??
• Inertia satisfaction with current practices
• IDEA assessment of change not required

42
Recent Advances

• Availability of materials Off the Shelf
• Roland Good DIBELS http//dibels.uoregon.edu/
  OR
• http//www.dibelsassessment.com/
• Gary Germann and Mark Shinn AIMSWEB
• WWW.AIMSWEB.COM
• WWW.EDFORMATION.COM
• James Wright
• www.interventioncentral.org

43
Oral Reading Fluency

• What is it?
• Reading aloud fluently and accurately from text.
• Why do it?
• Indicator of proficiency in reading that is
  sensitive to growth
• Highly correlated with performance on
  standardized tests and tests of comprehension
• Provides information that may be used to evaluate
  effects of instruction

44
Examiners Administration Rules

• After reading instructions to students,
• Start timer. If the student fails to say the
  first word of the passage after 3 sec., tell
  him/her the word and mark it incorrect. If the
  student stops or struggles with a word for 3
  seconds, tell the student the word and mark it
  incorrect. If the student reaches the end of the
  page and does not continue, point to the first
  word and ask the student to start over. At the
  end of 1 minute, place a bracket after the last
  word and say, please stop.

45
ORF Passage Making Friends

• There once was a little girl named Ann who
  9
• was very shy. She was too shy to make friends.
  19
• Ann lived in an apartment building with her
  mother 28
• and brother. Ann liked to play at the playground
  37
• near her apartment building. 41
• One day Ann was playing on the swings when 50
• Total words read 49
• Words read incorrectly 3
• Words read correctly 46

46
Egbert Problem Solving Step IIISystematic Data
Collection

• Behavior Observation
• Focus on problematic behaviors, attending, noisy,
  physically negative, in appropriate locale
• Peer referenced, determine typical levels of
  performance
• Across relevant settings
• Multiple measures

47
Peer Referenced Behavior Observation Momentary
Time Sampling 15 second intervals
48
Egbert Problem Solving Step IIISystematic Data
Collection

• Work completion-Gen Ed Classroom
• Egbert40 Peers90
• Work completion-Tutoring
• Egbert60 Peers95
• Non-Compliance (event recording)
• General classroom5 per day
• Tutoring2 per day
• Parents5 per day

49
Egbert Problem Solving Step IIISystematic Data
Collection

• Educationally Relevant Discrepancies?
• Reading-ORF and Decoding
• Classroom and Tutoring, interfering behaviors
• Non-compliance-observed in general classroom,
  tutoring, and home
• Is There Justification for Intense Interventions?

50
Egbert Problem Solving Step IVProblem Analysis

• IF-THEN Statements
• If this is done, then Egbert will ______.
• If these conditions are established, then Egbert
  will ________________.
• Intervene with each educationally significant
  discrepancy
• Implement powerful intervention principles from
  the instructional design and behavior change
  literatures

51
Egbert Problem Solving Step IVProblem Analysis

• Interpretation Egbert exhibits
• Skills deficits (poor reading fluency, poor
  decoding) and
• Performance problems (inattentive, disruptive
  classroom behavior, non-compliance at school and
  home).
• Emotional regulation is adequate

52
Egbert Problem Solving Step IVProblem Analysis

• Skills deficits Focus on instruction, teaching,
  learning conditions
• Performance problems Focus on antecedents and
  consequences.
• Emotional regulation Focus on modeling, guided
  practice, simulation, self-instruction, control
  triggers

53
Egbert Problem Solving Step IVProblem Analysis

• IF more explicit instruction is provided, with
  precise goals, progress monitored 2 Xs per week,
  graphed, formative evaluation applied, Egbert
  will make more rapid gains in reading
• If greater individualization and task analysis is
  conducted, focusing specifically on decoding
  skill development, Egbert will acquire better
  decoding skills and improve reading

54
Egbert Problem Solving Step IVProblem Analysis

• IF fluency training is implemented, Egbert will
  improve ORF (e.g., rereading)
• In behavioral expectations are made more explicit
  during each classroom activity and systematic
  reinforcement contingencies established for
  improved attentiveness, etc., Egbert will improve
  behavior and learn more rapidly

55
Egbert Problem Solving Step IVProblem Analysis

• If consistent interventions are used at home and
  school involving more frequent communication,
  greater consistency across settings, and more
  powerful reinforcement for improved work
  completion and compliance to requests, Egbert
  will improve in both areas and in reading.

56
Egbert Problem Solving Step V Goals

• ORF will increase by 2 WRC per week (class
  average is 1.5 per week)
• Word list fluency will increase by 2 WRC per week
• Word attack skills will increase by 2 units per
  week as defined by the district adopted criterion
  referenced
• Work completion and accuracy will increase by 5
  per week until Egbert meets or exceeds class
  averages

57
Egbert Problem Solving Step V Goals

• Inattentiveness, physically negative,
  inappropriate locale, and noisy will improve by
  5 per week until peer averages are attained.
• Non-compliance at school and home will be reduced
  to no more than 1 per day

58
Intervention Plan Development

• Powerful instructional and behavior change
  principles, scientifically based
• Reschlys General Principles
• Effective classroom organization and behavior
  management-Good Beh Game
• Instruction at the childs skill level
• Teacher directed, skills based
• Strong curriculum scope and sequence defined
  skill hierarchy

59
Intervention Plan Development

• Reschlys General Principles cont.
• Ambitious goals
• High rate of student response/feedback
• Time on task
• Monitor progress, graph results in relation to
  goals
• Formative evaluation rules and instructional
  changes
• Reinforcement, matched to group or child

60
Reading Curriculum/Interventions

• 5 Components of good reading instruction
• Phonemic awareness (Ktg. 1st grade)
• Alphabetic principles
• Fluency
• Vocabulary
• Comprehension
• Systematic instruction in each

61
Egbert Problem Solving Step VI Intervention Plan
Development

• Reading instruction augmented by 20 minutes per
  day of direct instruction on fluency and decoding
  skills with progress monitored 2 Xs per week
• Explicit expectations for behavior-sign system
  designed by psychologist and teacher. Activity
  reinforcers in school and home used to consequate
  improved behavior

62
Egbert Problem Solving Step VI Intervention Plan
Development

• Daily home school noted will be implemented based
  on the goals and progress monitoring. Meeting
  goals produces an additional 30 minutes per day
  of TV. Failure to meet goals results in
  reduction of 30 minutes per day
• Psychologist will provide support in the
  development of measures, intervention materials,
  graphing, progress monitoring, and formative
  evaluation
• Plan reviewed every 2 weeks

63
Egbert Problem Solving Step VII Intervention
Plan Implementation

• Prompts and props provided
• Intervention check-list
• Follow-up contacts, in person and phone
• Revise interventions that turn our to be not
  feasible
• Observe implementation (with teachers or
  parents permission)

64
Step VIII Progress Monitoring and Formative
Evaluation

• Time series analysis graph with goals for
  behavior change
• Frequent measurement, 2x per week in academics
  daily (if feasible) on behavior goals
• Rules for making changes in interventions
• Implement changes as needed

65
Continue Intervention and Monitor Progress
Change Intervention
Change Goal
Words Correct Per Minute
Class Growth
Class24
Egbert11
Egbert goal line
0 1 2 3 4 5 6 7 8 9
10 12 14 16 18 20
Weeks
66
Gap Not Closing Consider Eligibility and More
Intensive Interventions
Change Intervention
Class WCM54
Words Correct Per Minute
Class Growth
Egberta WCM32
0 1 2 3 4 5 6 7 8 9
10 12 14 16 18 20
Weeks
67
LD Identification Tier I, Tier II, Then What?

• Tier I and Tier II are done well and are
  insufficient (significant double discrepancy
  exists) THEN WHAT???
• Four Options Implications and Value
• Cognitive assessment, processes, pattern of
  strengths and weaknesses??
• IQ-achievement discrepancy??
• Intense, individualized problem solving
• Nothing-child is eligible

68
Critical Skills/Competencies

• Problem solving-interviewing skills
• Behavior assessment including CBM
• Powerful instructional interventions
• Powerful behavior change interventions
• Relationship skills
• Tailoring assessment to referral concerns

69
Continuing Education

• Problem solving and system design
• Reschly, D. J., Tilly, W. D. III, Grimes, J. P.
  (Eds.). (1999). Special education in transition
  Functional assessment and noncategorical
  programming. Longmont, CO Sopris West.
• Bergan, J. R., Kratochwill, T. R. (1990).
  Behavioral consultation and therapy. New York
  Plenum.
• www.interventioncentral.org

70
Continuing Education

• CBM, CBA, and behavior assessment
• Shinn, M. R. (Ed.). (1989). Curriculum-based
  measurement Assessing special children. New
  York Guilford Press.
• Shinn, M. R. (Ed.) (1998). Advanced applications
  of curriculum-based measurement New York
  Guilford Press.
• Shapiro, E. S. (Ed.) (1996). Academic skills
  problems Direct assessment and intervention (2nd
  Ed.). New York Guilford Press.
• Shapiro, E. S., Kratochwill, T. R. (Eds.).
  (2000). Behavioral assessment in schools
  Theory, research, and clinical applications (2nd
  Ed.). New York Guilford Press.

71
Continuing Education

• Interventions-behavioral and academic
• Sulzer-Azaroff, B., Mayer, G. R. (1991).
  Behavior analysis for lasting change. Fort
  Worth, TX Holt, Rinehart, Winston.
• Howell, K. Nolet, V. (2000). Curriculum-based
  evaluation Teaching and decision making (3rd
  Ed.). Atlanta, GA Wadsworth.
• Shinn, M.R., Walker, H.M., Stoner, G. (2002). 
  Interventions for academic and behaviors problems
  II  Preventative and remedial approaches. 
  Bethesda, MD NASP

72
Sense of Humor

• Three things that are real God, human folly, and
  laughter
• The first two are beyond our comprehension
• So we must do what we can with the third.
  John F. Kennedy
• Best wishes to you for a great convention and
  year