Measuring Child Progress: Two State

1
Measuring Child ProgressTwo States Journeys

• Barbara Jackson, NE
• Beppie Shapiro, HI
• Measuring Child and Family Outcomes
• Albuquerque, NM
• April 25, 2006

2
Format of Presentation

• Meet each state
• Overview of the 4 key questions to be discussed
• For each key question
• Description of each states plans
• Questions and comments from participants

3
What Counts

• Measuring Benefits of Early Intervention in
  Hawaii
• Beppie Shapiro, University of Hawaii
• beppie_at_hawaii.edu

4
About Hawaii

• About 17,000 births per year
• Over 7 of each birth cohort served in Part C
• No majority ethnicity
• 4.1 of population are immigrants, primarily
  Asians and Pacific Islanders
• Geographic isolation
• Five main islands different access to services

5
Special education in Hawaii

• Part C under Department of Health
• Part B under Department of Education
• Outcomes process differs from C to 619
• SEA 1 LEA

6
Part C in Hawaii, 2006

• Broad eligibility
• Three DOH agencies provide services statewide
• Recent development of statewide IFSP to be used
  by all programs
• No Part C database 2 agencies have their own

7
Standardized IFSP form, process

• 2005 All Part C programs trained to use new
  statewide IFSP form
• Present Levels of Development
• Family-driven Outcomes and Objectives
• Services to support Objectives
• Transition

8
What Counts?.... Measuring the Outcomes of
Early Intervention in Nebraska

• Barbara Jackson, Munroe Meyer Institute
• bjjackso_at_unmc.edu

9
About Nebraska

• Birth Mandate State
• Co-Leads for Part C Health and Human Services
  Education
• Outcome data process will be the same across the
  birth through 5 age group

10
Nebraska (continued)

• 460 School Districts
• Number of children birth to 3 1303
• Number of children ages 3 and 4 2811
• Number of children 3 through 5 4707

11
Key Question

• How will the state determine childrens status on
  each of the outcomes?

12
Assessment in Hawaii

• Part B Section 619, Hawaii Assessment
• Early Brigance
• Part C Since 2004 3 assessment tools approved
  for CDE Hawaii Development Charts (HELP),
  Michigan EIDP, or ASQ

13
Hawaiis Part C Assessment Ages and Stages
For babies eligible due to
environmental risk

• Forms for specific age levels
• Scores indicate only Typical, Monitor,
  Refer
• Referred for CDE and services by program serving
  DD
• Care coordination may remain with original program

14
What Counts Measuring the Benefits of Early
Intervention in Hawaii

• What Counts Design Team convenes monthly to
  develop/review progress
• How can we assign a score to
• each child on each EI Goal?

15
Hawaiis Part C Outcomes (Goals) Measurement
Process

• Data Collection Intervals Rationale
• At every IFSP initial, review, annual
• Starting when child is at least 4 months old

16
Hawaiis Part C Outcomes (Goals) Measurement
Process
Assigning scores decisions rationale

• WHO IFSP team
• Following Present levels of Development
  description
• HOW all team members reach consensus on rating
  using modified ECO Child Summary Reporting Form

17
Hawaiis 619 Outcomes Measurement Process

• Family involved in assessment process
• Family provides input on written form with
  open-ended questions
• A professional assigns rating on each goal
• Other providers asked for input to rating
• Ratings assigned at entry and annually

18
Nebraskas Process

• Child Outcome Task Force convened to guide
  process

19
Nebraska calls for Child Assessment that.

• Is based on ongoing observation of children
  engaged in real activities, with people they
  know, in natural settings
• Engage families and primary caregivers as active
  participants
• Is individualized to address each childs unique
  ways of learning
• Reflects that development and learning are rooted
  in culture and supported by the family
• Integrates information across settings

20
Nebraskas Assessment Process

• Which assessments?
• AEPS
• Creative Curriculum
• High Scope COR for Infants/ Toddlers
  Preschoolers

21
Why Selected?

• Assessment approach parallels other Early
  Childhood Program assessment processes
• Use information from multiple sources (e.g.,
  family, providers) and multiple observations
• Curriculum-based Assessment can be used for
  multiple purposes
• Can build the capacity of our system to support
  children and their families

22
Nebraskas Process for parent input?

• Parents provide input during the assessment
  process

23
How is the assessment information transformed?

• Working with publishers to determine feasibility
  of computer-based formulas
• Scores will be reviewed by team to assure
  validity of score

24
Nebraskas Assessment Process

• Data Collection Schedule
• Districts will be mandated to report entry and
  exit data
• Entry data will collected within the 45-60 days
  after IFSP/IEP meeting
• Exit data collected within 2 months of leaving
  the program
• Districts will be encouraged follow publishers
  guidelines for frequency of assessments

25
Key Question

• 2. What reporting categories has the state
  chosen to use?

26
What Counts Measuring Benefits of Early
Intervention in Hawaii

• Reporting Categories in Hawaii

27
Reporting Categories in Hawaii
Part C Decision Rationale

• Use ECO 5 Categories
• maintained typical functioning
• made progress to achieve typical functioning
• moved nearer to typical functioning
• D. progress but not enough to move nearer to
  typical
• E. did not make progress
• Meaningful, Program Improvement

28
Reporting Categories in Hawaii

• 619
• OSEP Categories

29
Reporting Categories in NE

• Four Categories
• OSEPs 3 categories
• Plus of children who attain typical
  development

30
Key Question

• 3. How and in what form will data get from local
  programs to the state?

31
Getting Part C Data to the State In Hawaii

• Simpler than most states?
• More difficult than many states?
• Each Agency will collect and summarize data
  from its programs

32
Data to explain results

• Length of enrollment
• Age at enrollment
• ICD9s/conditions

33
Getting Data to the State Part C

• Electronic data entry at local program
• Transmitted to Agency
• Agency calculates number of children in each
  category
• Agency sends these numbers to State
• State creates OSEP and cross-Agency reports

34
Unresolved data questions

• Local programs send explanatory data to Agency
• How much of this data goes to State? How?
• How to identify potential duplicates

35
Getting Data to the State 619

• Teacher enters data into stand-alone EXCEL
  spreadsheet
• Spreadsheets collected at school, sent to State

36
NE Getting Data to the State

• Using the publishers internet system- State will
  be the licensed manager
• Link with the State Data System

37
Key Question

• 4. What will you do to maximize the reliability
  of the data?

38
Training, Support to Maximize Data Reliability in
Hawaii

Training Support Web
7 hrs, all EI programs, by community 1-2 months
post training On site? Conference call? FAQs,
email listserv
39
Maximizing Data Reliability in Hawaii Evidence
for Rating

• IFSP teams document evidence used to select
  rating category
• Pilot showed need for training on what makes good
  evidence
• Supervisors provide support and QA

Criteria for Evidence (in progress) Different
contexts Highest level of achievement Specific
to Goal Area
40
Examining Reliability in Hawaii

• Another measurement of how well child is doing
• Compare ratings
• Compare ratings for children with different
  conditions/diagnoses

41
Challenges for Hawaii Quality Assurance - Part
C

• How reliable are assessments?
• Does every IFSP team have someone knowledgeable
  about typical child development?
• How reliable are summary form ratings?

42
Reliability of Data in NE Training

• Intensive training on assessments throughout
  state with consistent trainers

43
Reliability of Data in NE Implementation

• Review of data at the state level
• Ongoing questions/answers communication document
  based on local questions
• Team approach to assessment process
• Pilot process to determine reliability issues