Developmental Disabilities Program Independent Evaluation DDPIE Project

1
Developmental Disabilities Program Independent
Evaluation (DDPIE) Project

• Lynn Elinson, Ph.D.
• Project Director

2
Evaluation Standards

• Utility
• Feasibility
• Propriety
• Accuracy
• Joint Committee on Standards for Educational
  Evaluation

3
Purpose of the Independent Evaluation

• Demonstrate impact of DD Network programs on
• Individuals
• Families
• Service providers
• State systems
• Provide feedback to ADD to help improve the
  effectiveness of its programs and policies
• Promote accountability to the public

4
Purpose for UCEDDs

• Identification of accomplishments from external
  organization
• Identification of areas that need improvement

5
DDPIE Project

• Independent evaluation
• 2 phases
• Phase 1 development and testing of tools
• Phase 2 full-scale evaluation
• Westat contracted by ADD to implement Phase 1

6
Evaluation
Standards
Indicators
What do we hope to achieve?
What do we observe (measurement of indicators)?
Comparison
Are there differences/discrepancies?
What is the nature and extent of the differences?

What action needs to be taken?
7
Open Systems Model
Effectiveness
Structure (Input)
Process
Output (Product)
Outcome
Efficiency
8
Basic Evaluation Approach

• Performance-based approach
• Development of standards and indicators
• Measurement of indicators to determine level at
  which standards are being met
• Development of measurement matrices that contain
  standards, indicators, and performance levels
  (not developed limited development adequate
  development)
• Determination of overall performance at the
  national level

9
Evaluation Tools

• Measurement matrices
• - standards
• - indicators (structures, processes, outputs,
• outcomes)
• - performance levels
• Data collection instruments

10
INDEPENDENT EVALUATION
DD Act Administered by ADD Programs DD
Councils PAs UCEDDs Collaboration
KEY FUNCTIONS
FRAMEWORK OF INDICATORS
STANDARDS
PERFOR- MANCE LEVELS
Standards will be used as benchmarks to
evaluate the indicators, as measured.
DD Councils Outreach Informing
policymakers UCEDDs Training Comm. serv.
Research Dissemination PAs Individual
advocacy Outreach/public education Collab. Proj.
develop. Proj. implement.

• Structure
• Process
• Output
• Outcome

• Level
• at which a
• standard is
• met.
• Three
• performance
• levels for
• each
• standard.

MEASUREMENT MATRICES Will be developed to
organize each programs key functions, framework
of indicators, standards and corresponding
indicators, and performance levels for each
standard.
11
(No Transcript)
12
Project Tasks and Timing

• October, 2005 December, 2006
• Collect and review background information.
• Establish Advisory Panel, Working Groups, and
  Validation Panels.
• Develop draft performance standards, indicators,
  and data collection tools.
• Work with Validation Panels to finalize matrices
  for pilot study.
• Train pilot study staff.
• January, 2007 September, 2007
• Conduct pilot study in up to 5 states.
• Write report to ADD with recommendations.

13
Advisory Panel

• Self-advocates
• Family members
• Representatives from 3 programs - UCEDD Richard
  Carroll from Arizona
• Child/disability advocates
• Evaluation expert
• Federal representative (for PAIMI evaluation)

14
Working Groups

• 4 Working Groups (UCEDD, PA, DD Council,
  Collaboration)
• Role To assist Westat in developing draft
  measurement matrices that will be reviewed and
  endorsed by Validation Panels
• Process In-person and telephone meetings work
  offline

15
Criteria for Selection of UCEDD Working Group
Members

• Director/Associate Director
• At least one state/jurisdiction has more than one
  UCEDD program
• Rural/urban
• Geographic distribution
• Reflects different types of UCEDDS (e.g.,
  placement in medical school, school of education,
  standalone)

16
Working Group Activities

• Description of program
• Discussion and identification of key functions
• Identification of structures, processes, outputs,
  and outcomes for each key function
• Discussion of standards for structures,
  processes, outputs, and outcomes

17
What can UCEDDs do?

• Stay informed
• Open process - input welcome
• Participation in pilot study random selection

18
UCEDD Working Group Members
19
Westat Contact Information

• Lynn Elinson
• - LynnElinson_at_westat.com
• - 412 421-8610
• Cynthia Thomas
• - CynthiaThomas_at_westat.com
• - 301 251-4364
• Priyanthi Silva
• - PriyanthiSilva_at_westat.com
• - 301 610-5162
• Bill Frey
• - FreyW1_at_westat.com
• - 301 610-5198