Experiences and Lessons Learned Using Publishers Online Assessment Systems for OSEP Reporting

1
Experiences and Lessons Learned Using Publishers
Online Assessment Systems for OSEP Reporting

• Presenters
• Barbara Jackson Munroe-Meyer Institute, UNMC
• JanThelen, Nebraska Dept. of Education
• Ruth Littlefield, New Hampshire Dept. of
  Education
• August 27th, 2007

2

• State Overview

3
What is Results Matter in Nebraska?

• A statewide child and family outcomes system
• Designed to improve programs and supports
• For all young children birth to five, including
  Part C and Part B 619
• Served by school districts, the Early Development
  Network, and their partners
• Results Matter applies to all school-based early
  childhood programs in Nebraska

Results Matter in NE
4
Results Matter in Nebraska Purpose

• improve experiences, learning, development and
  lives of young children (birth to age five) and
  their families
• inform program practices
• demonstrate program effectiveness
• guide the development of local and state policies
  and procedures
• provide data to demonstrate results

5
Results Matter in Nebraska calls for curriculum
and child outcomes assessment that ...

• is based on ongoing observation of children
  engaged in real activities, with people they
  know, in natural settings
• reflects evidence-based practices
• engages families primary care providers as
  active participants
• integrates information gathered across settings
• is individualized to address each childs unique
  ways of learning
• informs decisions about day-to-day learning
  opportunities for children
• reflects that development and learning are rooted
  in
• culture and supported by the family

Results Matter in NE
6
Implementation Process for Results Matter in
Nebraska

• NECTAC state planning framework with state
  management team in January 2005
• State Local Infrastructure
• Personnel Development
• Implementation/Practices
• Child/Family/Program Outcomes
• Engage State Level Child Measurement Task Force
  on implementation process
• Implemented six pilot sites in 2006
• Statewide implementation in January 2007

Results Matter in NE
7
Results Matter in Nebraska

• Three child measurement tools.
• The High/Scope Child Observation Record (COR)
• The Creative Curriculum Developmental Continuum
• Assessment, Evaluation and
• Programming System (AEPS)
• Use of publishers online systems for
• supporting documentation of required
• assessments

Results Matter in NE
8
NH Preschool Outcome Measurement System

• A statewide system to measure outcomes for
  preschoolers with disabilities
• Designed to improve programs supports
• For all preschool children with disabilities with
  an IEP (effective November 1, 2006)
• May be used more broadly, at local discretion

9
NH Preschool Outcome Measurement System

• GSEG Cornerstone grant for Part C 619
• Engage stakeholders
• Align Child Outcomes (Birth through 5) with Early
  Learning Guidelines
• Consider how to address family outcomes
• Federal reporting AND Improving outcomes
• Focus on progress, not comparison

10
NH Preschool Outcome Measurement System

• Two Lead agencies
• Part C 10 Area Agencies with 14 programs
• Part B 619 177 School districts with multiple
  programs
• Shared goal improve results for children with
  disabilities and their families
• Part C COSF
• 619 Publishers web-based data systems
• Behind the scenes conversion to OSEP reporting
  requirements

11
NH Preschool Outcome Measurement System

• Three child measurement tools
• Assessment, Evaluation, and Programming (AEPS)
  http//www.brookespublishing.com/store/books/brick
  er-aeps/osep.htm
• Brigance Inventory of Early Development II
  (IED-II) http//www.curriculumassociates.com
• The Creative Curriculum Developmental Continuum
  Assessment for Ages 3-5 http//www.creativecurric
  ulum.net/
• Allowed for flexibility
• Assessor preference
• Best fit for the child

12

• State Administration

13
NDE Administrative Structure

• 254 School Districts 19 Regional Service Units
• State License Subscriptions
• School Districts
• Head Start Programs
• NDE memos
• Authorization forms completed as part of
  enrollment process
• NSSRS NDE student identification number

Results Matter in NE
14
NDE Online Structure
Results Matter in NE
15
NDE Contracts and Costs

• Jan 2006 NDE emails surveys to districts to
  collect
• Information on of children in school district
  early childhood programs
• Selection of tool(s)
• Fall 2006 developed contracts with three
  Publishers
• Brookes Publishing AEPSi.com 11.00
• Pearson Learning Highscope.net 11.00
• Teaching Strategies Creativecurriulum.net 10.95
  
• Fall 2007 Head Start Agencies contract with NDE
  to be part of the license

Results Matter in NE
16
NH Administrative Structure

• 177 School districts in 80 School Administrative
  Units
• NHDOE memos
• Statewide Preschool Technical Assistance Network
  (PTAN)
• Professional development, TA and networking
• Immediate communication vehicle

17
NH Administrative Structure

• NHSEIS
• (NH Special Education Information System)
• Student Identification numbers
• Limited authority to view personal data
• Limited resources to fund the system
• High level of commitment to quality and
  simplicity for districts

18
NH Contracts

• Local Districts sign on with publishers
• No State-Level Umbrella agreements
• Districts negotiate directly with publishers
• NHDOE online access to reports for OSEP
• district and state level ( )
• run 3 reports and compile as needed
• Districts access to all assessment data
• Inform instruction
• Demonstrate effective practices
• Promote discussion with families and other early
  childhood partners

19
NH Cost

• Survey Monkey estimate number of children and
  users for each tool (for pricing training)
• State provided estimated children and or users to
  publishers
• Paid for by local districts
• NHDOE worked with publishers to establish best
  rate (not as good as umbrella agreement)

20
NH Cost

• AEPS
• One-time state fee
• Establish data reporting system Create state
  account 1750
• Special rate to Beta Testers 9/child
• Rate for other NH users 11/child
• Complete AEPS, Second Edition (Administration
  Guide, Test for Birth to Three Years and Three to
  Six Years, Curriculum for Birth to Three Years,
  Curriculum for Three to Six Years) 239.00 with
  10 discount when subscribe to AEPSi

21
NH Cost

• Brigance IED II
• One Inventory per user 159.00 each
• One Developmental Record Book per student
  35.00/10-pack or 329.00/100-pack
• Web-based IED-II Management System annual
  subscription 124.95 each user first year (99
  for subsequent years)
• State account charged at same rate as local user
  (124.95)

22
NH Cost

• Creative Curriculum
• 11.75 per child
• No state fee
• No additional cost for tools/materials (unless
  want hard copies)

23
NDE Customizing Needs

• Funding Source-allows for differential children
• Special Education Funding
• Early Intervention Part C (Birth-3)
• Early Childhood Sp Ed Part B (3-5)
• Not Applicable
• State Funded EC Funding Source
• EC Grant Program
• Even Start
• Migrant Program
• Title I
• Not Applicable
• Other Funding Source
• Head Start
• Early Head Start
• Not Applicable
• Child Newly Verified (IFSP/IEP) after 8.1.06
  Yes /No

Results Matter in NE
24
NH Customizing Needs

• NH Head Start Collaboration office is working
  with NHDOE to better coordinate outcome
  measurement across systems
• Need to consider Work Sampling (used by some NH
  Head Start)
• Work Sampling
• user enters assessment information
• the COSF data point is generated
• the user can adjust up or down

25
NH Customizing Needs

• Each Publisher offers different benefits and
  challenges
• Each has a representative dedicated to NH
• Areas for negotiation
• Tracking students who move within the state
• Online and phone TA for users
• Ease of federal reporting
• Always ask the next question!

26

• Components for Implementation

27
NH Training/Professional Development

• Assessment Fair
• May 2006
• Publishers paid us they provided demonstrations
  and resource
• PTAN
• local networking
• phone and online TA
• Coordinated series of publisher trainings
• Preschool Leadership Conference (Sept 07)

28
NH Supports and Resources

• Statewide Preschool TA Network (PTAN)
• http//ptan.seresc.net/
• Preschool Outcome Measurement Toolkit
• NHDOE Special Education Memos
• Links to ECO
• Developing new FAQ

29
NDE Training/Professional Development

• Statewide Training Series on Assessment Rolled
  out January 2006
• Publishers provided first training (January)
• State trainers provided at three regional sites
  (Spring)
• Results Matter State Conference on Measurement
  Spring 2007
• Regional Training Series on Early Childhood
  Curriculum Planned for Fall 2008
• Developed collaboratively and supported through
  the Early Childhood Training Center

Results Matter in NE
30
NDE Supports and Resources

• http//www.nde.state.ne.us/ECH/ECH.html
• Click on Results Matter
• NDE Administrative Guidelines
• NDE Assessment Fidelity Process

Results Matter in NE
31
Why implement a fidelity process?

• Maximize the credibility of observational data
  collected for Results Matter
• Reliable and valid data maximizes quality of
  information
• Consistency among providers in scoring
  observations (intra- and inter-reliability)
• Documentation to support scoring (validity)
• Quality information forms the basis for program
  planning

Results Matter in NE
32
Who does the plan affect?

• All providers responsible for administering and
  scoring observational assessments for Results
  Matter
• Early Childhood Practitioners
• Early Childhood Special Education Practitioners
• Related Services Providers

Results Matter in NE
33
Levels of Expertise

• Fidelity process is based on providers
  experience
• Tier 1 Providers new to the assessment
• Tier 2 Providers experienced with assessment
• Tier 3 Experienced providers who can mentor
  others

Results Matter in NE
34
Five Components of the Fidelity Process

• All providers must meet reliability standards by
  April 1, 2008
• New provider orientation
• Continued group monitoring
• Individual monitoring
• Ongoing mentoring of providers

Results Matter in NE
35
Sample Plan District Level Strategies
Results Matter in NE
36

• Reports Available and
• How They Are Used

37
AEPSi

• Near entry only reports now available
• OSEP near entry near exit report available as
  of Sept 2007
• State and district level reports
• Districts access online information about their
  children and programs

38
AEPSi Near Entry Report
39
Creative Curriculum

• OSEP Reports
• Child Progress Reports
• Development of 9 point scale to assess at-risk
  children
• CreativeCurriculum.net Special Needs
  Accountability Pack (SNAPTM) and OSEPreports.net
• District and State level reports
• Local school districts all information on their
  children/programs

40
Creative Curriculum
41
High/Scope COR

• Child Outcome Reports
• OSEP report Data Downloaded to High/Scope
  Foundation for analysis
• Report is in progress available Sept 2007

Results Matter in NE
42
NH Brigance Inventory of Early Development
43
Publisher/State Challenges

• Different stages of online development
• Different levels of experience
• Creating interface with state data systems and
  processes
• Addressing children with significant delays
• Responding to OSEP timelines
• Emerging OSEP guidance

44

• Discussion