An Overview of a Problem Solving Model for Decision Making

1
An Overview of a Problem Solving Model for
Decision Making

• A module for pre-service and in-service
  professional development
• MN RTI Center
• Author Kerry Bollman, SSP, NCSP
• www.scred.k12.mn.us click on RTI Center

2
MN RTI Center Training Modules

• This module was developed with funding from the
  MN legislature
• It is part of a series of modules available from
  the MN RTI Center for use in preservice and
  inservice training

2
3
Overview

• Historical Context and Evolution of Problem
  Solving Models
• Problem Solving within a Response to Intervention
  Framework
• Problem Solving Steps and Questions

3
4
(No Transcript)
5
Historical Application of a Problem Solving
Approach

• Cascade of Services (E. Deno, 1970)
• Behavioral Consultation (Bergen Kratochwill,
  1990)
• In the early days, the problem solving model
  was predicated on the n1 approach.
• Problem Solving approach promoted many
  improvements to service delivery relative to a
  traditional SPED testing model.

6
Problem Solving System Addressed Many School
Problems
7
Problem Solving System Addressed Many School
Problems
8
However, Not Yet a Perfect System

• Five Major Challenges (Tilly, 2008)
• Efficiency
• More reactive than proactive
• Capacity of individual teachers to implement
  multiple interventions.
• Perception that problem solving continues to be
  the way to get students into special education.
• Reengineering the problem solving model to
  incorporate new developments in research and
  practice.

9
Continued Evolution of the Problem Solving Model

• Moving away from an N1 model toward one in which
  the problem solving model is to be used for all
  students in the system, not just those who
  struggle.
• Examination of core curriculum
• School-wide data collection for purposes of
  screening and program evaluation
• Ability to implement data-based group and
  individual interventions
• (continued)

10
Continued Evolution of the Problem Solving Model
(cont.)

• Application of the problem solving model equally
  to both academic and social behavior concern
  areas
• Move away from sole use as an expert driven
  consultation model, toward a model that supports
  collaboration between general and special
  education staff

10
11
Problem Solving Within the Context of a School
Wide Response to Intervention Framework
Assessment
Instruction
Response to Intervention
Problem-Solving Organization
12
Processing Activity

• List three important improvements that a problem
  solving model offers to schools as compared to a
  traditional service delivery model
• List three reasons why utilizing problem solving
  as a systems improvement model rather than solely
  a 1 student at a time model benefits schools
• Discuss how the identification of students who
  may need intervention support through a review of
  systems wide screening data as opposed to relying
  entirely on individual teacher referral may
  reduce bias.

13
Decision-Making Model Used by Problem-Solving
Teams
14
(No Transcript)
15
Step 1 Problem Identification

• Question What is the discrepancy between what is
  expected and what is occurring?
• 1. List problem behavior(s) and prioritize.
• 2. Collect baseline data on primary area of
  concern (target student and comparison).
• Record Review
• Interview
• Observation
• Testing
• 3. State discrepancy between target student
  performance and peer or expected performance.

16
Problem Identification Key Points

• Collect analyze regular school-wide screening
  data on most common referral concerns for
  efficient problem identification in these areas
• A decision must be made about how to define
  expected (local norms / national norms /
  criterion) so teams know which students to
  identify as at-risk
• (continued)

17
Problem Identification Key Points (cont.)

• One concern must be prioritized at a time
• Trying to take on everything at once often
  results in getting nothing done
• Concern needs to be stated measurably
• Avoid problem admiration during this phase
• Data from a variety of sources should converge to
  certify the problem
• Screening data other available evidence

17
18
Problem Identification Key Points (Cont.)

• Converging Evidence
• Multiple sources of data that each point to a
  consistent conclusion
• RIOT Review, Interview, Observe, Test
• Consider all these sources when seeking
  convergent evidence to certify a problem
• ICEL Instruction, Curriculum, Environment,
  Learner
• Consider all these domains when seeking
  convergent evidence

19
Processing Activity
Choose at least 1 pair above, and brainstorm
about the information you could collect.
Example Reviewing Curriculum What core and
supplemental materials have been used? How many
lessons are included on the topic of
difficulty? Do the materials provide sufficient
modeling and practice? Is mastery of this skill
expected at this grade level? Was homework
included?
20
Problem Identification in Practice

• Step 1 Screen all students
• Step 2 Identify all students at risk (all those
  who scored below the target score established for
  the assessment
• Step 2a For students not at risk, plan to
  re-screen in 3-4 months
• Step 3 For all students identified as at risk,
  consider other data sources across domains to see
  if you have convergent evidence of a problem
  (RIOT ICEL).
• Step 3a For students where no convergent
  evidence of a problem is found, confirm that
  strong core curriculum is in place and consider
  periodic monitoring
• Step 4 For students where convergent evidence
  of a problem is found, proceed with problem
  solving to develop an intervention
• Plan for with group interventions where reasonable

21
Step 2 Problem Analysis

• Problem Analysis is the process of gathering
  relevant information in the domains of the
  instruction, curriculum, environment, and the
  learner (ICEL) through the use of reviews,
  interviews, observations, and tests (RIOT) in
  order to evaluate the underlying causes of the
  problem.
• Heartland AEA II

22
Step 2 Problem Analysis

• Question Why is the problem occurring?
• Consider what you know about the target behavior
  that is relevant to determining why the problem
  is occurring and a possible solution
• Based on what you know list possible causes for
  the students problem (hypotheses)
• Consider all domains (Instruction, Curriculum,
  Environment, Learner)
• Differentiate between skill and performance
  problems
• Determine situations in which the problem is most
  and least likely
• (continued)

23
Step 2 Problem Analysis (cont.)

• For each hypothesis, list supporting and
  non-supporting data
• Narrow down to the most validated and alterable
  hypothesis
• Collect any additional data you need to validate
  the hypothesis that the team considers to be the
  most likely
• Need at least 2 pieces of convergent evidence,
  one should be quantitative

23
24
Problem Analysis Key Points

• There can be several possible underlying reasons
  why a student is doing poorly in an academic
  area. It is crucial to determine the reason(s)
  for poor performance in order to select an
  appropriate intervention
• They dont want to do it - The student has the
  necessary skills but lacks the motivation to
  complete the task
• They havent had enough practice - The student
  possesses the necessary skills but is not yet
  fluent and automatic in those skills

25
Problem Analysis Key Points

• They havent had enough instruction The student
  lacks the necessary skills to perform the
  academic task needs additional instruction,
  modeling, and feedback.
• They havent had to do it in that way before -
  The student needs support to generalize skills to
  new settings
• The skill is too hard - the student needs
  instruction in pre-requisite skills

25
26
Problem Analysis Processing Activity

• How would you find out whether the cause of an
  academic problem for a student was due to lack of
  practice with the skill, or due to a need for
  additional instruction?

27
Step 3 Plan Development

• Question What is the goal?
• Write the goal, a measurable statement of
  expected outcomes.
• Question What is the intervention plan to
  address the goal?
• Define logistics (e.g., what strategies/procedures
  will be used, when and how often the
  intervention will occur, who will implement the
  intervention and where it will be implemented,
  and when it will begin).
• Question How will progress be monitored?
• Define logistics (e.g., what materials are used,
  when and how often data will be collected, where
  data will be collected, and who is responsible).
• Decide on decision-making rules for plan
  evaluation.

28
Plan Development Key Points

• For goal setting, it is helpful to determine rate
  of growth for goal that would result in a
  reduction of the discrepancy of student
  performance
• Oral Reading Fluency 2 words per week or to
  local spring target
• Written Expression 1/2 CWS per week or to local
  spring target
• Math Facts 1/2 fact per week or to local spring
  target
• Behavior 10 improvement per week
• (continued)

29
Plan Development Key Points (cont.)

• Any intervention idea chosen for the student
  should be scientifically research based
• Within a district, developing a master list of
  interventions used for which staff have training
  and necessary materials is helpful
• (continued)

29
30
Plan Development Key Points (cont.)

• Access resources to assist with development of
  interventions list
• MN RTI Center intervention list
  http//www.scre.k12.mn.us click on MN RTI Center
• http//ies.ed.gov/ncee/wwc/
• http//www.fcrr.org
• http//www.interventioncentral.com
• Important for teams to understand the difference
  between an intervention and a modification

31
Plan Development Key Points (cont.)

• Role of master schedule in planning interventions
  - planning time and staff
• Tool chosen for progress monitoring must have
  adequate technical adequacy for this purpose -
  outcomes measurement
• Teams must determine what represents adequate
  response to the intervention

31
32
Plan Development Processing Activity

• Generate a list of all the scientifically based
  interventions you are aware of
• Could be scripted programs like DI Reading
  Mastery, or protocols like Incremental Rehearsal
• Could be for academics as above, or like Check
  and Connect for social behavior concerns

33
Step 4 Plan Implementation

• Question How will implementation integrity be
  ensured?
• Provide training and support to those
  implementing interventions.
• Observe intervention in action.
• Make adjustments to intervention plan if needed.
• Collect and graph data on intervention goal.

34
Plan Implementation Key Points

• Intervention Scripts Training
• Increases likelihood that specifics of
  intervention will be well understood by those
  performing the intervention
• Are preferred by interventionists rather than
  global intervention descriptions
• Training should include modeling, practice, and
  feedback with adults prior to use with students

35
Plan Implementation Key Points

• Integrity checks It is impossible to evaluate
  the success of a plan if the team is not certain
  that the plan was implemented as designed
• Did the program happen as planned at least once?
  (formal observation)
• Do you have data to indicate that the student
  participated fully in the intervention?
  (attendance, time logged in, accuracy of
  participation in intervention)

36
Example Integrity Script
Script developed at St. Croix River Education
District
37
Step 5 Plan Evaluation

• Question Is the intervention plan effective?
• 1. Use data to determine student progress.
• 2. Evaluate intervention acceptability.
• 3. Determine as a team what to do next.

38
Example Instruction is working for student.
Reading
East Riser Elementary
Sunnydale
Smith
09-10
District
Year
Jacob
Name
School
Teacher
By May 31, Jacob will read 110 words correct per
minute on 3rd grade material.
Goal Statement
Expected Level of Performance
1
2 3 4

Service Providers
Parent Participation
120
Baseline
110
Goal
100
I-Phonics for Rdg.
90
80
70
60
Words correct per min.
50
40
30
20
10
0
M M M M M M M M M M M M M M M M M M M M M M M M M
M M M M M M M
Day
Heartland AEA II
39
Example Instruction is NOT working for the
student.
I-PALS Reteach division
Heartland AEA II
40
Plan Evaluation Key Points

• Ensure agreement on implementation integrity
  prior to evaluating outcomes of an intervention
• Evaluate student outcomes only for interventions
  that have been fully implemented
• Plan Evaluation does not happen without a graph
• Determination of next steps teams might
• Identify a new problem
• Consider a different hypothesis for the same
  problem
• Plan a new intervention to address the same
  problem and hypothesis
• Rework intervention to be able to achieve
  implementation integrity
• Celebrate student success!

41
(No Transcript)
42
Case Study - Problem Solving Process

• Beginning School Wide
• Continuing with Small Group Intervention
• Resulting in Individual Intervention

43
Step 1Problem Identification

• Question What is the discrepancy between what is
  expected and what is occurring?
• Third grade team met in September to review fall
  R-CBM and NWEA MAP Reading data
• Identified a group of 14 students in general
  education with scores of concern
• Below fall grade 3 targets on R-CBM and/or MAP
• Teacher report (last year, this year)
• Statements of discrepancy based on individual
  scores relative to expectation

44
Fall Grade 3 DataStudents of Concern
45
Step 2 Problem Analysis

• Question Why is the problem occurring?
• Team noticed that most of these students
  demonstrate high error rates in oral reading
• Review of records indicate pattern of performance
  across years
• Teachers report poor phonics skill mastery among
  these students
• Hypothesize that these students are discrepant
  because they need additional instruction in
  phonics

46
Step 3 Plan Development

• Core instruction response K-2
• Considered allotted time for reading instruction
• Worked on increasing time within reading classes
  for students actively interacting with text
• Reviewed scope and sequence for phonics and added
  more modeling and practice on these skills to
  core program
• Collect grade-wide screening data and set goals
  to increase percentages of students meeting
  spring targets by grade level

47
Step 3 Plan Development

• Question What is the goal?
• Team sets a goal that in 10 weeks, all identified
  students will increase their fluency on 3rd grade
  passages by 15 WRC per minute.
• 1.5 words per week growth from baseline
• Question What is the intervention plan to
  address the goal?
• Replace silent reading time each day with small
  group focused instruction in phonics.
• Chose Phonics for Reading level 2
• Set up extra incentive program for independent
  reading at home with parents for these 10 weeks
• Question How will progress be monitored?
• Weekly monitoring of CBM.

48
Step 4 Plan Implementation

• Sped teacher provided 2 classroom teachers with
  training on phonics intervention
• Students moved to one of these two classes to
  receive intervention each day
• Half of non-participating students in each
  classroom of teacher providing intervention moved
  to another classroom for silent reading
• School counselor from team conducted 1
  observation for implementation integrity for each
  teacher
• Teacher kept track of time spent on intervention
• Sped para collected weekly progress monitoring
  data

49
Step 5 Plan Evaluation
100 90 80 70 60 50 40 30
Matt
Baseline 9/17 9/24 10/1 10/8 10/15 10/22
10/29 11/5 11/12
50
The Individual Student

• Matt is in the 3rd grade group of students
  receiving additional phonics instruction in place
  of silent reading
• Grade level team meet in November to review
  outcomes data from group intervention.
• Teachers note that Matt is not making adequate
  progress toward his goal despite this
  intervention.
• Make a referral to the building-based problem
  solving team

51
Matt Problem Identification
80 70 60 50 40 30
Aimline - desired rate of progress
Trendline - actual rate of progress
Baseline 9/17 9/24 10/1 10/8 10/15 10/22
10/29 11/5 11/12
52

• In November, Matt is reading 44 WRC/Min on 3rd
  grade level text. Fall expectation was 70.
  Winter expectation is 91.
• Matt is making less than 1 word per week increase
  in WRC on 3rd grade probes, while his expected
  growth is at least 1.5 words per week.

53
Step 2 Problem Analysis

• Question Why is the problem occurring?
• Conduct a teacher interview to clarify the
  problem and define current program.
• Examine CBM Survey-Level Assessment data.
• Develop hypotheses.

54
Third Grade Instructional Planning Form
55

• Grade Level Passage 1 Passage 2 Passage 3
• 3 39/10 42/10 44/8
• 2 55/7 53/5 63/7
• This pattern of performance is also confirmed
  through an interview with Ms. Bensinger.
• High rate of meaning violating errors in both
  grade levels of text

56
Review of Phonics Lessons

• Teacher reports Matt is making good progress in
  the phonics lessons, and reads stories within
  lessons with few errors
• Review of student workbook shows about 90 first
  time correct responses
• Teacher reports that most times when an error is
  pointed out to Matt, he self-corrects immediately.

57

• Hypothesis
• Matt reads grade level texts with low fluency
  (low rate and high errors) BECAUSE he needs
  additional support to generalize or apply the
  phonics skills he knows to novel text for first
  time correct reading.

58
Step 3 Plan Development
Question What is the goal? In 10 weeks, Matt
will read 3rd grade material with a fluency rate
of 57 WRC per minute.
70 60 50 40 30
Goal
Aimline Desired rate of progress.
11/26 12/3 12/10 12/17 12/31 1/7
1/14 1/21 1/28 2/4
59

• Question What is the intervention plan to
  address the goal?
• Since problem analysis points to high levels of
  errors contributing to low fluency scores,
  develop a plan to address error rates.
• Maintain phonics instruction since within program
  progress appears strong
• Add additional intervention time to allow for
  more practice to build fluency

60
Third Grade Instructional Planning Form
61

• Question How will progress be monitored?
• Continue collecting CBM reading data weekly with
  3rd grade probes

62
Step 4 Plan Implementation

• Implement NEW instructional program monitor
  treatment integrity.
• This intervention was observed twice to confirm
  that it was being implemented with fidelity.
• The interventionist kept track of Matts
  attendance to ensure he was getting as much
  practice as the team had hoped.

63
Step 5 Plan Evaluation

• Question Is the current intervention plan
  effective?
• Collect on-going CBM reading data.
• Graph CBM data
• Regularly compare trendline to aimline.
• Determine plan effectiveness.

64
80 70 60 50 40 30
Goal
Trendline Actual rate of progress.
Aimline Desired rate of progress.
11/26 12/3 12/10 12/17 12/31 1/7
1/14 1/21 1/28 2/4
65
Plan Evaluation Summary

• The revised intervention program
• Did result in improved reading fluency.
• Significantly decreased error rates
• Surpassed the desired rate of progress.
• Team chose to continue the intervention

66
Problem Solving Take Home Messages

• Define problems using data
• Identify causes over which you have control
• Set specific student goals
• Intervene, not just accommodate
• Monitor progress
• Fidelity
• Define success using data

67
Big Ideas Problem Solving Teams

• Teams are well-balanced among general and special
  education, and across grade level representation
• Active work to avoid perception that these are
  special education hoop-jumping teams.
• Teams meet regularly and follow specific agendas
• Teams guided by a problem solving model
• Assessment is based on what question is being
  asked at each step of the model
• Assessment is linked to intervention

68
Discussion Questions

• Most buildings have some form of a general
  education intervention team. List elements of
  the problem solving process as described in this
  presentation that may not be present in a typical
  team
• What benefits do you see to teams adopting a more
  comprehensive problem solving model such as this?

69
Concept GeneralizationList One Word In Each Box
70
Fill each word into the sentence below, and
complete the sentence
Problem solving teams are like a ________
because...
71
Web resources

• www.rti4success.org
• www.rtinetwork.org
• www.progressmonitor.net
• www.interventioncentral.org
• www.centeroninstruction.org

72
References

• Bergen, J., Kratochwill, T.R. (1990).
  Behavioral consultation and therapy. New York
  Plenum Press.
• Burns, M. K., Jacob, S., Wagner, A. R. (2008).
  Ethical and legal issues associated with using
  response-to-intervention to assess learning
  disabilities. Journal of School Psychology,
  46(3), 263-279.
• Daly, E. J., Witt, J. C., Martens, B. K., Dool,
  E. J. (1997). A model for conducting a functional
  analysis of academic performance problems. The
  School Psychology Review, 26(4), 554-574.
• Deno, E. (1970). Special education as
  developmental capital. Exceptional Children, 37,
  229-237. 
• Deno, S. L., Fuchs, L. S., Marston, D., Shin,
  J. (2001). Using Curriculum-based measurement to
  establish growth standards for students with
  learning disabilities. School Psychology Review,
  30(4), 507-524.
• Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz,
  L., Germann, G. (1993). Formative evaluation of
  academic progress How much growth can we
  expect? School Psychology Review, 22, 27-48.
• Tilly, W. D., III (2008). The evolution of school
  psychology to science-based practice problem
  solving and the three tiered model. Best
  Practices in School Psychology, V.

72
73
References

• Hintze, J. M., Christ, T. J. (2004). An
  examination of variability as a function of
  passage variance in CBM progress monitoring.
  School Psychology Review, 33, 204 217.
• Hintze, J. M., Shapiro, E. S. (1997).
  Curriculum-based measurement and literature-based
  reading Is curriculum-based measurement meeting
  the needs of changing reading curricula? Journal
  of School Psychology, 35(4), 351-375.
• Hintze, J. M., Pelle Petitte, H. A. (2001). The
  generalizability of CBM oral reading fluency
  measures across general and special education.
  Journal of Psychoeducational Assessment, 19(2),
  158-170.
• Lane, K.L., Bocian, K. M., MacMillan, D. L.,
  Gresham, F. M. (2004). Treatment integrity An
  essential- but often forgotten- component of
  school-based interventions. Preventing School
  Failure, 48(3), 36-43.
• Noell, G. H., Witt, J. C., LaFleur, L. H.,
  Mortenson, B. P., Ranier, D. D., LeVelle, J.
  (2000). Increasing intervention implementation
  in general education following consultation A
  comparison of two follow-up strategies. Journal
  of Applied Behavioral Analysis, 33(3), 271-284.
• Noell, G. H., Witt, J. C., Slider, N. J.,
  Connell, J. E., Gatti, S. L., Williams, K. L., et
  al. (2005). Treatment implemenatation following
  behavioral consultation in schools A comparison
  of three follow-up strategies. School Psychology
  Review, 34.
• Silberglitt, B., Hintze, J. M. (2007). How much
  growth can we expect? A conditional analysis of
  R- CBM growth rates by level of performance.
  Exceptional Children, 74(1), 71-84.

74
Quiz

• Most buildings have some form of a general
  education intervention team. List some elements
  of the problem solving process that may not be
  present in a typical team.
• List the steps of the problem-solving model
  discussed.
• True or False A challenge of the problem
  solving system is that it is more reactive than
  proactive.
• Which of the following is NOT a way to collect
  baseline data on primary area of concern
• Record Review
• Interview
• Observation
• Testing
• All of the above are ways to collect baseline
  data.
• What may be some benefits of teams adopting a
  more comprehensive problem-solving model such as
  this?
• True or False Problem admiration is often a
  helpful tool to use within the problem-solving
  model.

74
75
The End ?

• Note The MN RTI Center does not endorse any
  particular product. Examples used are for
  instructional purposes only.
• Special Thanks
• Thank you to Dr. Ann Casey, director of the MN
  RTI Center, for her leadership
• Thank you to Aimee Hochstein, Kristen Bouwman,
  and Nathan Rowe, Minnesota State University
  Moorhead graduate students, for editing, writing
  quizzes, and enhancing the quality of these
  training materials