Welcome to the Workshop

1
Welcome to the Workshop!
Note There is more information about what was
covered in the workshop in the accompanying
handouts.
2
What Do the Numbers Mean?Making Sense out of
Outcome Data

• Donna Spiker
• Kathy Hebbeler
• SRI International

Outcomes Workshop Albuquerque, NM April 2006
3
Transforming the Assessment Information into Data
on Child Outcomes
4
Need to transform the data

• Under any assessment option, state needs a way to
  transform the assessment information into the
  indicators/evidence statements.
• No assessment provides information directly on
  the 3 functional outcomes areas
• No assessment provides information directly on
  the 3 categories (a,b,c) in the OSEP indicators

5
Need to transform the data

• if assessment process involves multiple sources
  of information
• if more than one assessment is being used in the
  state

6
DesiredOutput
Input
?
OSEP Indicators Social a, b, c Knowledge
a, b, c Meet Needs a, b, c
Time 1 Scores Cognitive Communication Social Adapt
ive Motor
Time 2 Scores Cognitive Communication Social Adapt
ive Motor
(for 100s of children!)
7
Decisions related to transforming the assessment
data

• What is the process by which the data gets
  transformed?
• What is the rubric used to roll up the data?
• Where does the transformation occur?
• State
• Local program

8
A way to think about how children are doing with
regard to each outcome
9
(No Transcript)
10
Definitions for Outcome Ratings
11
(No Transcript)
12

• Topic 1
• Using Assessment Data to Contribute Information
  to the ECO Summary Rating

13
Table Item Results for 5 Children
14
Activity 1. Assign a Summary Rating to each
child based on the score pattern. Table Item
Results for 5 Children
15

• Topic 1.1
• Adding Value for Local Programs Reports Produced
  for Local Programs
• (Note This applies if assessment data are
  entered into an online system or if program will
  be producing reports in other ways)

16
Table Outcome Ratings for Class 3c by Child
17
Table Percentage of Children Scoring 5 or
Higher by Class (Example of an Aggregated Report
for Program)
18

• Topic 2
• Basic State Reports
• Using Data from the Summary Rating to Produce
  OSEP Data

19
(No Transcript)
20
Activity 2. Be a computer. Assign an ECO
Category and an OSEP Category to the records
below. State data file
21
FMA

• Findings
• Meaning
• Action

22
Table Outcome 1 Percentage of Children in
Each OSEP Category, 2008-2010
23
Table Outcome 1 Percentage of Children in
Each OSEP Category by Local District
24

• Topic 3
• Using Data from the Summary Rating to Produce
  Reports for the State
• No Additional Data

25
Table Percentage of Children in Different
Progress Categories
26
Table Percentage of Children Who Maintained or
Moved Closer to Age-Appropriate Development,
2007-2009
27
Table Comparison of Populations at Entry and
Exit
N 500
28
Table Progress of Children Between Fall and
Spring
29
Table Using the Summary Rating
30
Table Outcome 1 at entry
31

• Topic 4
• Using Outcomes Data to Produce Reports for the
  State
• Additional Data Available

32
Examples of Additional Variables

• childs age at assessment
• length of time between assessment completions
• childs condition, delay, or eligibility
  criterion and/or severity (if applicable)
• childs teacher or care coordinator
• type and/or intensity of services received
• length of time in program
• quality rating on program

33
Two ways to get additional variables

• merging the outcomes data with other data
• or
• 2. collecting additional data as part of the
  outcomes data collection

34
Table Extent Of Change for Outcome 1 by
District For Children Who Entered Programs
Between 16 and 20 Months and Exited between 30
and 36 Months
35
Table Outcome 1 Rating at Exit by Age at Entry
into the Early Intervention System
36
Table Outcome 2 Rating by Average Amount of
Speech Therapy Received per Week
37
Summary and Take Away Messages

• The value of the outcome data depends on what a
  state does with it.
• Minimal value meeting the federal reporting
  requirement
• Maximal value meeting the needs of a variety of
  audiences in the state
• Outcomes data has the potential to answer many
  important questions.

38
Summary and Take Away Messages (Continued)

• There are many different audiences for outcomes
  data.
• Federal government
• State administrators
• State policy-makers
• Local administrators
• Providers and teachers
• Families

39
Summary and Take Away Messages (Continued)

• Different audiences have different questions and
  need the data reported in different ways.
• Thoughtful planning for the kinds of reports the
  state wants for different audiences is necessary
  to insure the system is built to meet the needs
  of different audiences.

40
Summary and Take Away Messages (Continued)

• The major cost in building an outcomes system is
  in the data collection. Generating one or 101
  reports adds minimally to the cost.
• States need to maximize their investments by
  making sure the system is going to produce the
  answers (the reports) they want.

41
Just Remember
If you have been good, the Data Fairy will bring
reports to you!! For more
information
www.the-eco-center-org.