Great Expectations: Practice and Policy for Developing a Quality Outcomes Assessment System A Tale of Two States

1
Great Expectations Practice and Policy for
Developing a Quality Outcomes Assessment
SystemA Tale of Two States
Measuring Child and Family Outcomes 2008

• Meredith Cathcart
• Anne Kuschner
• Larry Edelman
• Patty Salcedo

2
Californias Child Outcomes System Quality
Assessment to Quality Programs

• Overview of Californias DRDP Assessment System
• Policies and practices that support quality
  assessment systems
• Demonstration of an online learning module
  designed for new assessors
• Use of data to improve services and outcomes

3
The Context of Diversity in California
Size
Geographic
Ethnic
Linguistic
Socio-economic
4
(No Transcript)
5
California Department of Education Desired
Results System Assessment Instruments
Child Development Division (CDD)
Special Education Division(SED)

• Desired Results Developmental Profile - Revised
  (DRDP-R) for
• Infant/toddler
• Preschool
• School-age

• Preschool Desired Results Developmental Profile
  Revised for Children with IEPs (PS DRDP-R)
• Desired Results Developmental Profile access
  (DRDP access)

6
Purposes of the DRDP Assessment System

• Assure that preschool-age children with IEPs and
  their families benefit from Californias ECSE
  services and programs
• Provide information about childrens development
  in relation to research and state standards so
  that programs, services, and supports are
  designed to maximize childrens potential
• Establish a common ruler for describing
  childrens progress across the state

7
Goals of the DRDP Assessment System

• Improve results for children and families
• Provide families with information about child
  progress and ways to support their childrens
  learning
• Provide teachers with authentic assessment
  information to inform program planning
• Provide program administrators, staff, and
  stakeholders with data to improve overall program
  design
• Enable Californias preschool special education
  system to comply with federal reporting
  requirements

8
Principles Guiding the Development of the DRDP
Instruments

• Desired Results are for all children
• Content of the instruments is based on child
  development research
• Instruments utilize an authentic assessment model
  and are based on observation in typical
  environments
• Multiple sources of data should be gathered over
  time
• Assessments are universally designed
• Instruments include adaptations for children with
  disabilities

9
Basics of Californias System

• In addition to the SPP, California must meet IDEA
  requirements for statewide assessment
• Every 3,4,and 5 year old is assessed 2x per year
• Assessment windows cover 6 month intervals
• The SELPA is the unit of accountability for
  reporting results
• IEP teams determine which DRDP instrument to use
  and which adaptations should be in place
• Data is reported using a statewide web-based
  system, SEDRS
• T/TA system supports fidelity at all levels

10
(No Transcript)
11
and so on
12
The DRDP Instruments for Preschool Special
Education
13
Navigating the DRDP Instruments

• Our first online module in learning to use the
  DRDP instruments

14
DRDP Adaptations

1. Augmentative or Alternative Communication System
2. Alternative Mode for Written Language
3. Visual Support
4. Assistive Equipment or Device
5. Functional Positioning
6. Sensory Support
7. Alternative Response Mode

15
(No Transcript)
16
Training, Technical Assistance, and Professional
Development
17
Training Facilitates Quality Assessment Practices

• Teachers understand the requirements and
  responsibilities of the DRDP Assessment System
• Teachers feel confident and competent using the
  DRDP assessment tools
• The quality of data collected increases with
  fidelity to the use of the instrument

18
What Worked for the First 2 Years of
Implementation

• Creating a phase-in of the system over several
  years
• Creating a training cadre drawn from highly
  qualified, experienced statewide trainers
• Built on collaboration between ECSE T/TA
  contractors, SELPAs, and CDE
• Systematically mentoring less experienced
  trainers
• Conducting a mandatory Train-the-Trainer
  institute
• Piloting the training sessions
• Training trainers on the use of technology
• Insisting on minimum standards for training
  facilities

19
Current Training System Components

• Hold two statewide trainings annually for new
  assessors
• Support and build capacity for local training
  events
• Provide online training modules for assessors and
  data entry personnel
• Provide online guidance documents, teacher
  resources, and discussion forum and listservs
• Disseminate updates at statewide and local
  conferences and events
• Communicate with SELPA and link with CDE

20
A Brief Tour of www.draccess.org
21
Making Meaning of DRDP Data
22
What Data is Collected?

• Every preschool child with an IEP is assessed on
  the ten DRDP Indicators twice per year
• Current demographic information is gathered on
  each child at each assessment period
• Raw data is submitted by assessors or SELPAs to
  SEDRS
• Scale scores are generated by SEDRS and reported
  back to SELPAs, district administrators, and
  service providers

23
Primary Uses of Assessment Information

• Serves multiple purposes
• accountability
• monitoring of child progress
• program planning
• Provides immediate, useful feedback about an
  individual childs progress and status
• Assessment scores are easily understood by staff
  and families, e.g.
• plot a childs developmental trajectory
• display status relative to typical development

24
The DRDP Contributes to

• Formative assessment
• information is gathered on an ongoing basis
• observation of child behavior is conducted in
  typical routines and activities
• information is gathered from families, child care
  providers, and/or related service providers
• Summative assessment
• data relative to the three OSEP child outcomes is
  aggregated by CDE and sent to OSEP
• data for each SELPA is reported to the public
  according to OSEP requirements
• data for programs is be provided through the use
  of DRDP scale scores

25
The blue colored band represents the typical
developmental range for the childs age
The black line marks the place represented by the
childs scale score in that particular indicator
25
26
This report depicts the change between two
assessment periods.
26
27
Group Measures into Indicators
28
Facilitated Process Reflective Data Analysis
(pilot 2008-09)

• When data analysis is done within the context of
  the team it yields useful information to programs
  and adds to increased data quality.
• Build time to review and analyze data into team
  meetings.
• Build a classroom and program summary
• Ask questions about the completeness and accuracy
  of the data
• Ask questions about the implications of the
  results for program design, services, and staff
  development

28
29
Some Questions about Completeness and Accuracy of
Data

• If children have a developmental age less than 3
  years they should NOT be assessed on the PS
  DRDP-R
• Are IEP teams using an appropriate
  decision-making process?
• What training is needed for IEP teams?
• Assessors should be able to rate 90 of
  Measures
• Do teachers understand the developmental
  constructs being measured by the DRDP?
• What are the barriers to observing collecting
  data?
• Most children with visual impairments should be
  assessed with visual supports
• What is the use of this adaptation in your
  program?
• Is more support needed for teachers?

29
30
(No Transcript)
31
Current Activities
32
Current Activities

• Continue activities to support quality
  professional development
• Modify the DRDP access instrument to align with
  Learning Foundations/preschool standards and
  ongoing revisions to the PS DRDP-R
• Increase support to local communities on
  professional development, data reporting, and
  data analysis
• Expand activities to interpret results in useful
  ways at local and state levels
• Create additional reports in the data collection
  and analysis software (SEDRS) to assist teachers,
  parents, and staff

33
Thank you!