Progress Monitoring

1
(No Transcript)
2
Progress Monitoring

• Strategies for Writing Individual Goals in
  General Curriculum and More Frequent Formative
  Evaluation
• Mark Shinn, Ph.D.Lisa A. Langell, M.A., S.Psy.S.

3
Big Ideas About Frequent Formative Evaluation
Using General Outcome Measures and the Progress
Monitoring Program

• One of the most powerful interventions that
  schools can
• use is systematic and frequent formative
  evaluation.
• Benchmark Assessment is not enough for some
  students because they may be in ineffective
  programs too long. (3 mos )
• The solution is to write individualized goals and
  determine a feasible progress monitoring
  schedule.
• The core of frequent progress monitoring is
• Survey-Level Assessment
• Goal setting using logical educational practices
• Analysis of student need and resources for
  determining progress monitoring frequency.

4
Formative Assessment

• Formative Assessment Process of assessing
  student achievement during instruction to
  determine whether an instructional program is
  effective for individual students.
• When students are progressing, keep using your
  instructional programs.
• When tests show that students are not
  progressing, you can change your instructional
  programs in meaningful ways.
• Has been linked to important gains in student
  achievement (L. Fuchs, 1986) with effect sizes
  of .7 and greater.

5
Systematic formative evaluation requires the use
of

• Standard assessment tools
• That are the same difficulty
• That are Given the same way each time.

6
More Severe Achievement Problems and/or More
Resource Intensive Programs Require More Frequent
Formative Evaluation
Benchmark Testing (3 - 4 x Per Year) is not
enough for some students.
7
With Very Low Performers, Not Satisfactory to
Wait This Long!
8
Programs That are More Resource Intensive

• Title I, English Language Learning, Special
  Education
• Should monitor student outcomes more frequently
  than the Benchmark Testing schedule.

9
Formative Evaluation of Vital Signs Requires
Quality Tools
Technical adequacy (reliability and validity)
Capacity to model growth (able to represent
student achievement growth within and across
academic years) Treatment sensitivity (scores
should change when students are
learning) Independence from specific
instructional techniques (instructionally
eclectic so the system can be used with any type
of instruction or curriculum) Capacity to
inform teaching (should provide information to
help teachers improve instruction) Feasibility
(must be doable).
10
Thinking About A Students Data

• Sample Student
• Melissa Smart
• 3rd grade student
• Progress Monitor

11
8
Melissa Smart
110
92
77
50
34
12
Formative EvaluationIs simply data enough?
13
Formative Evaluation Is data and a goal enough?
14
Formative Evaluation Are data, goals trends
enough?
15
Formative Evaluation is Impossible without all
dataGoals Make Progress Decisions Easier
16
Current Goal Setting Practices Are Unsatisfying!
Do you like these IEPs? I do not like these
IEPs I do not like them Jeeze Louise We test, we
check We plan, we meet But nothing ever seems
complete. Would you, could you Like the form? I
do not like the form I see Not page 1, not 2, not
3 Another change A brand new box I think we
all Have lost our rocks!
17
Need Shift to Few But Important Goals

• Often Ineffective Goal Smorgasboard!
• Student will perform spelling skills at a high
  3rd grade level.
• Student will alphabetize words by the second
  letter with 80 accuracy.
• Student will read words from the Dolch Word List
  with 80 accuracy.
• Student will master basic multiplication facts
  with 80 accuracy.
• Student will increase reading skills by
  progressing through Scribner with 90 accuracy as
  determined by teacher-made fluency and
  comprehension probes by October 2006.
• To increase reading ability by 6 months to 1 year
  as measured by the Woodcock Johnson.
• Student will make one year's growth in reading by
  October 2006 as measured by the Brigance.
• Student will be a better reader.
• Student will read aloud with 80 accuracy and 80
  comprehension.
• Student will make one year's gain in general
  reading from K-3.
• Students will read 1 story per week.

18
Improving the Process of Setting Goals for
Formative Evaluation
Set a few, but important goals. Ensure goals are
measurable and linked to validated formative
evaluation practices. Base goal setting on
logical educational practices.
19
Reduce the Number of Goals to a Few Critical
Indicators
Reading In () weeks (Student name) will read ()
Words Correctly in 1 minute from randomly
selected Grade () passages. Spelling In ()
weeks (Student name) will write () Correct
Letter Sequences and () Correct Words in 2
minutes from randomly selected Grade () spelling
lists. Math Computation In () weeks (Student
name) will write () Correct Digits in 2 minutes
from randomly selected Grade () math
problems. Written Expression In () weeks
(Student name) will write () Total Words and ()
Correct Writing Sequences when presented with
randomly selected Grade () story starters.
20
Ensure the Goals are Measurable and Linked to
Validated Formative Evaluation Practices
Goals should be based on quality tests like
CBM. Based on validated practices such as how
often, how many samples, etc.
21
Conducting a Survey Level Assessment
Students are tested in successive levels of
general curriculum, beginning with their current
expected grade placement, until a level at which
they are successful is determined.
22
John5th grader5th grade passage 26/12
John4th grade passage49/7
John3rd grade passage62/4
Conducting a Survey Level Assessment
23
Base Goal Setting on Logical Educational Practices
Example of PLEP statement John currently reads
about 26 words correctly from Grade 5 Standard
Reading Assessment Passages. He reads Grade 3
reading passages successfully 62 correct words
per minute with 4 errors, which is how well
beginning 3rd grade students read this material.
24
Goal Setting Strategies
Current Performance Information based on
Survey-Level Assessment (SLA). Know the Time
Frame for the Goal. Determine a Future
Performance Level.
25
Setting the Time Frame, Goal Level Material, and
Criterion
Time Frame End of Year (At Risk or Grade-Level
Expectations) In 18 Weeks Annual IEP Goals
(Special Education) In 1 year (or) In 32 Weeks
26
Setting the Goal Material

• Logical Task--
• Matching or Not Matching Expected Grade Placement
• Title I Fourth Grader--Grade 4 Material?
• Grade 4 Special Education Student--Grade 4
  Material?

27
When Grade-Level Expectations Are Not Appropriate

• Consider the Severity of the Discrepancy
• Consider the Intensity of the Program

28
Determining the Criterion for Success Options
to use

1. Local Benchmark Standards.
2. Linkage to High Stakes Tests.
3. Normative Growth Rates.
4. Developing Your Own Sample of Standards.

29
1. Benchmark Standards Advantages and
Disadvantages
Advantages Disadvantages
Easily Understood Uncomfortable, Especially in Low Achievement Environments
Can Indicate When Student No Longer Needs Specialized Instruction Issues of Equity
Determining Least Restrictive Environment (LRE)
30
2. Linkage to High Stakes Standards Advantages
and Disadvantages
Advantages Disadvantages
Reduces Problems of Equity when Local Achievement is Low Need Linkage to High Quality High Stakes Test
Increases Usefulness of High Stakes Tests Linkage Must Be Established Empirically
Helps Specify Long-Term Targets (What Grade 2 Student Needs to Read to Be Successful on Grade 6 Test) Adoption of Assumption that the Attainment of the Target Guarantees Passing High Stakes Test
31
Normative Growth Rates
Criterion for Success Score on SLA (Grade
Growth Rate times of Weeks) Score on SLA (30)
(Ambitious Grade Growth Rate (2.0) times of
Weeks (32) Or 30 (2.0 32) or 30 64
Annual goal of 94 WRC
32
3. Growth Rate Standards Advantages and
Disadvantages
Advantages Disadvantages
Easily Understood May Underestimate What Can Be Attained with High Quality Instruction
33
Developing Your Own Sample of Standards
Developing a Small Local Norm
34
Benchmark Standards Advantages and Disadvantages

Advantages Disadvantages
Same Advantages as Benchmark Standards Same Disadvantages of Benchmark Standards
Small Sample Size
35
How Frequently to Assess?
Balancing IDEAL with FEASIBLE
36

• Making Data-Based Decisions With Progress Monitor
  
• Need at LEAST 4-7 data points before making
  programming decisionand you may sometimes
  want to collect more if you are uncertain.
• Err on the side of caution
• Criteria To Consider
• Trendline meets Aimline for ultimate
  goal Consider return to LRE.
• Trendline and AIMline will intersect in
  relatively near future?
• Keep with current intervention until goal is
  reached.
• Trendline exceeds AIMline?
• Consider increasing goal or difficulty level.
• Trendline not going to intersect AIMlinemoves in
  opposite direction Consider adding additional
  intervention, changing variable, and/or
  insensifying program changes (LRE).

37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52

• The End