Using Indicator 14 Data for Program Improvement

1
Using Indicator 14 Data for Program
Improvement OSEP Leadership Meeting August 19,
2009
2
Who we are

• NPSO is a national technical assistance
  dissemination center funded by the U.S.
  Department of Education Office of Special
  Education Programs.

3
NPSO Center Charge

• Help State Education Agencies develop practical,
  yet rigorous, data collection systems to describe
  the further education and competitive employment
  experiences of youth with disabilities as they
  transition from high school to adult life.

4
IDEA Purpose

• To ensure that all children with
• disabilities have available to them a free
• appropriate public education that emphasizes
• special education and related services
• designed to meet their unique needs and
• prepare them for further education, employment,
  and independent living.

IDEA Regulations 300.1(a)
5
Revised Indicator 14

• Percent of youth who are no longer in secondary
  school, had IEPs in effect at the time they left
  school, and were
• A. Enrolled in higher education within one year
  of leaving high school.
• B. Enrolled in higher education or competitively
  employed within one year of leaving high school.
• C. Enrolled in higher education or in some other
  postsecondary education or training or
  competitively employed or in some other
  employment within one year of leaving high
  school.

6
Highlight of Changes

• Feb. 2009, significant changes
• Now a New Indicator
• No reporting required FFY 2008 APR due Feb. 1,
  2010 (for students who left school in 2007-2008
  school year)
• Baseline will be reported in FFY 2009 SPP, due
  Feb. 1, 2011
• New baseline will represent students who left
  school during the 2008-09 school year
• New indicator requires reporting 3 percentages
  (A, B, C)
• Reporting categories are higher education,
  competitive employment, some other postsecondary
  education or training, and some other employment
• Report each student in only 1 of the 4 categories
  of the indicator-with higher education as the
  highest
• This presentation summarizes a new NPSO document
  entitled Frequently Asked Questions, Revised
  Part B Indicator, Post-School Outcomes, July 2009

7
SPP/APR Reporting Timelines
8
What is the definition ofhigher education for
the purposes of this collection?

• Engaged in Higher Education means a student who
  is enrolled in a 2 or 4-year degreed program
  provided by community or technical college (2
  year) and/or college/university (4 or more year
  program).

9
What is the definition ofother postsecondary
training?

• May include but are not are limited to
• Compensatory education programs
• High school completion document or certificate
  classes (e.g., Adult Basic Education, General
  Education Development (GED))
• Short-term education or employment training
  program (e.g., Workforce Investment Act (WIA)
• Job Corps
• Vocational technical school which is less than a
  two year program
• Adult education
• Workforce development

10
What does at least 90 days at any time in the
year since leaving high school mean?

• Ninety (90) cumulative days or 3 months of
  continuous work at an average of 20 hours per
  week.

11
What are the Indicator measure denominators?

• For the three measures
• (A, B, and C),
• the denominator equals the
• number of respondent leavers

12
What are the Indicator measure numerators?
First calculate the following 1 of
respondent leavers enrolled in higher
education 2 of respondent leavers in
competitive employment (and not counted in 1
above) 3 of respondent leavers enrolled in
some other postsecondary education
or training (and not counted in 1 and 2 above) 4
of respondent leavers in some other
employment (and not counted in 1, 2, and 3
above) Important Note For federal reporting
Leavers should be counted in only ONE of the
above categories. Count the leaver in the highest
category. For Example If an individual is
enrolled in higher education and competitively
employed, count the individual ONLY in higher
education and NOT as competitively employed AND
enrolled in higher education.
13
Calculations

• To calculate the indicator percentages, use the
  following
• A 1 divided by total respondents
• B 1 2 divided by total respondents
• C 1 2 3 4 divided by total respondents

14
Here is another way to visualize it

Note Remember that individuals are only counted
once are counted in the highest category.
15
Must states set a target for each of the measures
of the Indicator (e.g., A, B and C)?
Yes
16
Data Use Toolkit An Overview
17
Purpose of the Data Use Toolkit

• To help State and Local Education personnel use
  post-school outcomes data to improve in-school
  programs for youth with disabilities

18
Three Components
19
Purpose Audience - Materials
20
The Wisconsin Department of Public Instruction is
committed to identifying and responding to the
need of students with disabilities
Indicator 14

• To that end, it is necessary to document the post
  high school outcomes of youth with disabilities,
  and to use that information to make programming
  and planning decisions to improve education and
  transition services for students, and to
  ultimately improve their post high school
  outcomes.

21
Indicator 14 Wisconsin Post High School
Outcomes Survey
Survey Background

• Wisconsin has been collecting outcomes for the
  past nine years.
• In preparation for the first survey year (2001),
  an extensive review of the literature was
  conducted to examine similar methods in survey
  design with the NLTS and the NLTS2.
• Input on survey questions was obtained from
  multiple stakeholders, including the DPI, DWD,
  DVR, IHE, DHS, parents, advocates, general and
  special education administration, and school
  board representatives.

22
Indicator 14 Wisconsin Post High School
Outcomes Survey

• As other Indicators have taken on more meaning in
  transition planning, WI has worked closely
  connecting our work across indicators with
• NPSO
• NSTTAC
• NDPC-SwD
• HS Redesign Center
• NAESDE
• IDEA Partnership
• National Community of Practice on Transition
• Received (2007-08) an OSEP Supplementary
  Transition Grant that enabled state to bring in
  TA Center experts to State Transition Conference
  and individual districts

23
Indicator 14 Wisconsin Post High School
Outcomes Survey
The Survey

• Years 1-6 Statewide data was collected
• (state responsibility)
• 1st, 3rd, 5th years, a 40 random sample of
  exiters was interviewed.
• 2nd, 4th, 6th years, LEAs were instructed on in
  survey administration and data use.
• Two groups (1st and 3rd cohorts) were contacted 3
  and 5 years after exiting high school to assess
  long- term outcomes.
• The state outcomes data collected was
  representative of the state exiters, but there
  was never enough information to give back to
  participating districts.

24
Indicator 14 Wisconsin Post High School
Outcomes Survey

• The third year of Indictor 14 data collection is
  currently being collected (shift to LEA
  responsibility)
• For Indicator 14, WDPI uses the Procedural
  Compliance Self-Assessment Monitoring Cycle to
  identify LEAs for data collection.
• Wisconsins LEAs have been divided into five
  survey year groups of approximately 88 districts
  each year.

25
Indicator 14 Wisconsin Post High School
Outcomes Survey
Why is it important to look at post high school
outcomes?

• Increased emphasis on accountability in education
  requires states to report to the public and to
  the Office of Special Education Programs (OSEP)
  information on the 20 compliance and performance
  Indicators.
• The Indicators are used to determine how well the
  state is meeting its responsibility under IDEA
  2004.
• The Indicators are also to be used within the
  state for system improvement.
• LEAs can use indictor data for local
  decision-making and improvement planning.

26
Indicator 14 Wisconsin Post High School
Outcomes Survey

• The Special Education District Profile is used to
  analyze LEA performance on the SPP Indicators.
• The WDPI posts local indicator data on their
  website http//dpi.wi.gov/sped/lpp-profile.html
• LEAs use both compliance and improvement
  indicator data for local improvements planning.
• Improvement indicators, like Indicator 14, do not
  require LEAs to meet a performance level, but are
  reviewed by the WDPI.

27
Indicator 14 Wisconsin Post High School
Outcomes Survey

• WDPI consultants consider whether or not the
  improvement activities
• are aligned to the indicators
• if they reflect state priorities
• are actionable
• are realistic
• include measures of performance and timelines
• identify responsibility for implementation

28
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Wisconsin Department of Public Instruction (WDPI)
• Awarded a discretionary grant CESA 11 to assist
  with the data collection and reporting
  requirements of Indicator 14
• Developed a telephone survey all exiters within
  selected districts are attempted to be contacted
• WPHSOS Website www.posthighsurvey.org
• A web-based tool developed to assist with the
  requirements of Indicator 14
• Designed to ensure consistency in data collection
  and reporting in a time-efficient, cost-effective
  manner

29
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Website Survey
• Includes a 5 8 minute telephone interview with
  former students
• Interviews are completed by St. Norbert College
  Survey Center in De Pere, WI on behalf of the
  district, at no cost to the district
• Assesses former students participation in
• Employment
• Postsecondary education or training
• High school experiences
• IEP Planning
• Includes several open-ended questions

30
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Website Data
• Contact information (phone number/address)
• Survey responses (complete survey on-line)
• All interview questions, comments and Indicator
  14 can be sorted by gender, ethnicity, disability
  and exit reason
• Immediate retrieval of district level data
• Website Reports
• GEDE Table
• Summary Report (one page auto-filled)
• District Report Starter (auto-filled, place for
  LEA interpretation)
• SEA / LEA Indicator 14 Report
• Improvement Planning Forms

31
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Indicator 14 outcomes are shared in general
  information dissemination

• State Superintendents Special Education
  Leadership Conf.
• Wisconsin Council of Administrators of Special
  Services
• Rehabilitation Conference of Wisconsin
• Wisconsin School Psychology Association (WSPA)
• Department of Workforce Development
• Division of Vocational Rehabilitation
• DHS (Supported employment and sheltered work case
  managers)
• WDPI Stakeholder meetings
• WDPI consultant meetings

31
31
32
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Indicator 14 outcomes are shared in general
  information dissemination

• Annual Wisconsin State Transition Conference
• Transition Coordinators Summits
• Transition Hub e-newsletter and other electronic
  and print materials - posted on www.wsti.org
• Wisconsin Community on Transition (6 Community of
  Practice Groups) - minutes www.sharedwork.org
• Other national/state/regional/local meetings as
  requested
• http//dpi.wi.gov/sped/spp-posthigh.html
• www.posthighsurvey.org

32
32
33
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Indicator 14 outcomes are shared for specific
  improvement planning

• County Council Meetings
• Multiple schools in one county planned one year
  in advance and had all districts participate in
  an outcomes survey.
• That fall, the county council members got
  together to review both district and combined
  county results
• Used that information to plan transition
  activities
• Outcomes surveys for all exiters/specific
  populations (Wis. School for the Deaf/HI)
• Outcomes surveys in non-required survey years.

33
33
34
Indicator 14 Wisconsin Post High School
Outcomes Survey
NEXT STEPS 1. Information Dissemination to
Broader Audience

• Wisconsin School Board Association
• Wisconsin Association of District Administrators
• Wisconsin Secondary School Principals Association
• Wisconsin Council of Administrators of Special
  Services
• School-based, multiple Indicator review teams

34
34
35
Indicator 14 Wisconsin Post High School
Outcomes Survey
NEXT STEPS 2. Resources

• Create a web-based repository of high quality
  resources and evidence-based transition practices
• Ability to sort resources by media type, gender,
  ethnicity, disability, NSTTACs Predictors of
  Success, Kohlers Taxonomy, and other criteria
• Connect data to resources and improvement
  planning
• Professional development and improvement planning
  within Wisconsin Personnel Development System

35
35
36
Indicator 14 Wisconsin Post High School
Outcomes Survey
NEXT STEPS 3. Data Use

• Data used for improvement planning is key to
  increasing the percentage of youth competitively
  employed and/or participating in postsecondary
  education or training.

• Wisconsin is working with the PSO Center to pilot
  the Data Use Manual at both the LEA and SEA level
• Focused Review of Improvement Indicators (FRII)
• Across Indictors and state projects (SPDG)

36
36
37
Indicator 14 Wisconsin Post High School
Outcomes Survey
Districts review outcomes data for one of two
reasons

• 1. LEA is Required to
• Because of a performance level requirement
• WDPI is developing a process (FRII), but does not
  have a minimal Indicator 14 performance level for
  districts at this time
• 2. LEA Wants to
• More than 100 districts have used to post high
  survey in a non-required survey year
• Several districts have used for all exiters

37
37
38
Indicator 14 Wisconsin Post High School
Outcomes Survey

• When districts want to review/use outcomes
• Districts express a desire to know what their
  local outcomes are and what they can do to
  change/improve practices
• Staff and transition partners work together
• Data Use Manual will be on-line and
  user-friendly.
• Professional development will be provided to
  directors, principals, transition specialists and
  others with access to and/or interest in data on
  data use process.

38
38
39
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Possible data use process format
• Morning
• Use ppt to understand data collection and
  reporting responsibility of Indicator 14
• Review state and local outcomes and other
  indicator data relevant to the district
• Team identify strengths/needs
• Afternoon or Another Meeting
• Team decides/agrees where to put time, energy,
  resources (may be smaller or building teams)
• Resources are identified
• Districts begin using resources to develop
  improvement plan and next steps

40
Indicator 14 Wisconsin Post High School
Outcomes Survey

• Putting it altogether . . .
• Get more stakeholders informed and involved by
  presenting outcomes and other indicator data to
  general education audiences and community
  partners
• Work with special education leadership and
  districts to engage them in systems review of
  practices
• Provide a format for looking at state and local
  data across indicators
• Provide a format for districts to easily locate
  high quality resources
• Provide a format of improvement planning where
  districts use their data to make planning
  decisions, and return to their data to support
  efforts and changes

40
40
41
Indicator 14 Wisconsin Post High School
Outcomes Survey
Now we will talk about the outcomes of Hudsons
youth
42
(No Transcript)
43
Examining Local Post High School Outcomes What
Can We Learn and How Can We Shape Programs?

• A Guided Dialogue For Using Post High School
  Outcomes For Youth with Disabilities to Improve
  Transition Services Outcomes

44
Purposes of Todays Meeting

• Become familiar with the federal post-school
  outcomes (PSO) requirements and how Wisconsin
  meets those requirements.
• Review Wisconsin Post High School Outcomes Survey
  (WPHSOS) statewide results.
• Discuss outcomes results from our district.
• Identify our program strengths (positive
  outcomes) and areas in need of improvement (less
  positive outcomes).
• Next Steps

45
Materials we will use today

• Wisconsin Post High School Outcomes (WPHSOS)
  Specific Terminology List
• 1-page summary describing how WPHSOS/Indicator 14
  data are collected
• List of the 20 State Performance Plan (SPP)
  Indicators
• Hudsons Local Performance Plan (LPP) Report of
  Indicators

46
Critical Relationships
Drop-out rate (Indicator 2)

• Meaningful postsecondary goals/plans
• (Indicator 13)

Parent Survey (Indicator 8)
Disproportionality (Indicators 9 10)
Graduation with a diploma (Indicator 1)
Positive post-school outcomes (Indicator 14)
Kohler (NSTTAC), 2007
47
Why is it important for us to look at post high
school outcomes?

• To ensure that all children with disabilities
  have available to them a free appropriate public
  education that emphasizes special education and
  related services designed to meet their unique
  needs and prepare them for further education,
  employment, and independent living.

IDEA Regulations 300.1(a)
48
We believe post high school outcomes are
important to analyze because

• Student perspective is most important
• Enables us to do a more effective and efficient
  job
• Help prioritize our focus
• Identify weaknesses and strengths in our
  transition process
• Prioritize student needs (present and future)
• Reflects our practices
• Relationships between school and community
• Did we get the students involved and responsible?

48
48
49
Drilling into Hudsons engagement rate

• Lets look at the percent of youth engaged in
  post high school activities by

• Gender
• Ethnicity/Race
• Disability
• Exit Types
• Combinations of above or subcategories of the
  above

50
Thinking about Transition-Related
Program Improvement

• In what areas are youth with disabilities doing
  well?
• What areas need improvement?
• What is the district doing well?
• What does the district need to improve?

51
Thinking about Transition-Related Program
Improvement

• What does this mean to you as a teacher, a
  parent, an administrator, a youth, a community
  member?
• How can you/we use this information to connect
  with others?

51
52
Staff responses based on their observations of
the data...

• Areas we are doing well...
• Graduation rate
• Involving students in their IEP programming
• Preparing students for employment
• Social skills and friendships
• Social activities
• Positive school/community partnerships with
  Bridge
• Participating in the WPHSOS more than mandated
• District being flexible and creative
• District is looking for feedback from staff and
  involving staff in decision and planning

53
Staff responses based on their observations of
the data...

• Areas in need of improvement
• Life skills and transition curriculum staff
  training
• Engaging at-risk students in school/work
  programs
• Job coaching
• Help connect students to resources
• Tutoring programs
• Helping students understand their disability
  self determination skills
• More specific transition goals increase
  emphasis at the Middle School level
• Transition Skills scope and sequence that
  coordinates with state and district general
  education standards
• Use district employees to help train others

54
Next Steps Questions To Be Answered

• What cost effective strategies/resources are
  available to us now?
• Who are the teachers with these skills in our
  district?
• Present this data and engage in dialogue to a
  larger group incorporate into Summer Academy
• Reassemble in Sept./Oct.
• Develop manageable goals for self, programs,
  district
• Avoid feeling overwhelmed so we dont shut down
  and get nothing done

55
Contact us at