Enhancing WIA Performance Outcomes for Older Youth

1
Enhancing WIA Performance Outcomes for Older
Youth
March 1, 2005 Massachusetts
Social Policy Research Associates1330 Broadway,
Suite 1426 Oakland, CA 94612Phone (510)
763-1499 Fax [phone]www.spra.com
Public/Private Ventures2000 Market Street, Suite
600 Philadelphia, PA 19103Phone [phone]
/Fax [phone]www.ppv.org
2
What is the Performance Enhancement Project?

• Provide technical assistance to more than 30
  states to improve performance.
• Diagnose causes of poor performance.
• Assist states and local areas to address
  problems.
• Disseminate practices to help states and local
  areas improve performance.

3
Enhancing Performance For Older Youth Workshop
Goals

• Develop an understanding of the older youth
  performance measures.
• Consider the factors which influence performance
  on the earnings change and credentials measures.
• Explore strategies to enhance older youth
  performance.

4
What do you expect to learn from the training
today?
5
Why should I care about performance?
6
Why should I care about performance?

• Better service to customers.
• Makes sure programs are on on track helping
  customers attain good outcomes.
• Sanctions incentives.

7
Part I. OVERVIEW OF WIA PERFORMANCE
8
Key Definition EXIT

• Hard exit
• A customer formally completes program or
  withdraws from services.
• Soft exit
• Customers who do not receive a WIA funded or WIA
  partner service for 90 days and are not scheduled
  for future services except follow-up.
• Planned gaps in service over 90 days due to a
  health/medical condition or delay before entry
  into training will not result in a soft exit.
• The exit date is the date of last service.

9
Avoid Soft Exits
Avoiding soft exits whenever possible is
important for performance

• By the time counselors find out about soft exits,
  youth may be well into their quarter after exit.
• The case manager may have little time to verify
  whether a positive outcome has occurred or to
  offer further assistance if needed.
• Note A soft exit does NOT automatically mean
  that a participant will have a negative outcome.

10
Planned Gaps in Service

• Youth can be put on hold when they have a
  legitimate reason for not participating for a
  period of time, but plan to return.
• These holds are for health/medical reasons when
  a service or training is not immediately
  available or when a reservist is called to
  active duty, and has a return date.
• Documenting a hold can prevent a soft exit.

11
Key Definition QUARTERS

• Calendar quarters
• January to March
• April to June
• July to September
• October to December

• Keep in Mind
• The exit quarter is the quarter containing the
  exit date.
• Quarters after exit (i.e. 1st or 3rd quarter)
  mean quarters after the exit quarter.

12
Key Definition QUARTERS
Please refer to Handout A
13
Key Definition Point of Registration

• WIA Youth Program
• All youth must be registered for WIA to receive
  any WIA funded youth services.
• Once registered, all youth participants are
  included in performance measures

14
Unemployment Insurance (UI) Wage Data

• UI wage data is used to track unsubsidized
  employment and wages in all Older Youth measures
  and the Younger Youth Retention measure.
• NOTE Gathering UI wage data is a State
  responsibility.

15
Unemployment Insurance (UI) Wage Data

• Challenges with using UI wage data

• Typically there is a two quarter lag before this
  data can be analyzed by the state for WIA
  performance.
• This means data verifying employment and wages
  for performance measures is not reported for many
  months after the performance period.
• Not all employed workers show up in UI wage
  records.
• States can use Supplemental Data to verify
  employment for these workers.

16
Global Exclusions

• Exclusions from all Performance Measures
• Institutionalized/incarcerated at exit
• Health/medical exits
• Reservists called to active duty who choose not
  to return to WIA
• Deceased

17
Performance Measures

• Question What WIA performance measures are
  calculated for Older Youth?

18
Older and Younger Youth Performance Measures

• Older Youth (19-21 yrs.)
• Entered Employment Rate
• Employment Retention Rate
• Earnings Change
• Credential Attainment Rate
• Younger Youth (14-18 yrs.)
• Skill Attainment Rate
• Diploma or Equivalent Attainment Rate
• Retention Rate

19
Entered Employment Rate Older Youth

• Official definition Of those not employed at
  registration and who are not enrolled in
  post-secondary education or advanced training in
  the first quarter after exit, the number of Older
  Youth who have entered employment by the end of
  the first quarter after exit divided by the
  number of Older Youth who exit during the quarter.

20
Entered Employment Rate Older Youth

• Simplified definition Generally measures the
  percentage of youth unemployed at program entry
  who are employed after the program.

21
Entered Employment Rate Older Youth

• Positive Performance Outcome
• Youth are employed in the 1st quarter after exit.

• Youth employed at registration are excluded.

22
Entered Employment Rate Older Youth

• Other Good Results
• Youth enter advanced training or post secondary
  education in the first quarter after exit.
• This is a great result for youth.
• If not employed, these youth are excluded from
  the measure.
• This is neither a positive nor negative outcome.
• Does lower the number of youth included in the
  calculation which may already be small.

23
Entered Employment Rate Older Youth

• Key Factors Contributing to a Positive Outcome
• Service/program completion.
• Direct job placement.

24
Entered Employment Rate Older Youth

• Calculation
• Number Employed in Quarter after Exit
• divided by
• Number of Exiters

Example 99 youth employed 1st qtr after
exit 12 87 89 150 exiters - 40 - 12
98 These 40 youth are not employed and are
enrolled in advanced training or post-secondary
education. They are excluded from the
calculation. These 12 youth were excluded from
the entire calculation because they were employed
at registration.
25
Employment Retention Rate Older Youth
Official definition Of those who are employed
in the 1st quarter after exit, the number of
Older Youth who are employed in the 3rd quarter
after exit divided by the number of Older Youth
who exit during the quarter. Older youth
enrolled in postsecondary education or advanced
training in 3rd quarter after exit are excluded.
26
Employment Retention Rate Older Youth

• Simplified definition Generally measures the
  percentage of older youth who got a job soon
  after leaving the program and are employed many
  months later.

27
Employment Retention Rate Older Youth

• Positive Performance Outcome
• Youth are employed in the 3rd quarter after exit.

• Youth not employed in the 1st quarter after exit
  are excluded.

28
Employment Retention Rate Older Youth

• Other Good Results
• Youth are enrolled in advanced training or post
  secondary in 3rd quarter after exit.
• This is a great result for youth.
• If they are not employed, these youth are
  excluded from the measure.
• This is neither a positive nor negative outcome.
• Does lower the number of youth included in the
  calculation which may already be small.

29
Employment Retention Rate Older Youth

• Key Factors Contributing to a Positive Outcome
• Good initial job placement.
• Effective post placement services and follow up
• Job retention, including supports.
• Advancement services and supports.
• Re-employment services.

30
Employment Retention Rate Older Youth

• Calculation
• Number Employed in 3rd Quarter after Exit
• divided by
• Number Employed in Quarter after Exit

Example ___66 youth employed in 3rd
qtr. after exit_______ 78.5 99 youth
employed in 1st qtr. after exit 15
84 These 15 youth are excluded because they
are enrolled in postsecondary education or
advanced training and not employed in 3rd quarter
after exit. The 12 youth employed at
registration and employed in the 1st qtr. after
exit are included in the calculation.
31
Earnings Change Older Youth

• Official definition Of older youth employed in
  1st quarter after exit, total post-program
  earnings (in Q2 Q3 after exit) minus
  pre-program earnings (in Q2 Q3 prior to
  registration) divided by the number of older
  youth/adults who exit during the quarter.
  Individuals not employed in the 1st quarter after
  exit are excluded.

32
Earnings Change Older Youth/Adult

• Simplified definition Basically measures
  earnings of older youth who become employed after
  leaving the WIA program and compares the earnings
  to earnings prior to entering WIA.
• Earnings are expected to be higher by a
  negotiated amount.

33
Earnings Change Older Youth

• Positive Performance Outcome
• Increase in Older Youth earnings when comparing
  pre-program (quarters 2 3 before registration)
  and post program earnings (quarters 2 3 after
  exit).

• Participants not employed in the 1st qtr. after
  exit are excluded.
• Participants for whom supplemental data was used
  to verify employment are excluded.

34
Earnings Change Older Youth

• Other Good Results
• Youth are enrolled in advanced training or post
  secondary education in the 3rd quarter after
  exit.
• This is a great result for youth.
• If not employed, these youth are excluded from
  the measure.
• This is neither a positive nor negative outcome.
• Does lower the number of youth included in the
  calculation which may already be small.

35
Earnings Change Older Youth

• Key Factors Contributing to a Positive Outcome
• Primarily serving customers with a past history
  of little work/earnings may make attainment
  easier.
• Connect participants to good paying jobs.
• Connect participants to permanent and/or full
  time jobs help them stay employed.
• Provide effective post-placement services and
  follow up
• Job retention, including supports.
• Advancement services and supports.
• Re-employment services.

36
Earnings Change Older Youth

• Calculation
• Post-program Earnings Pre-program Earnings
  divided by
• Number of Exiters in Quarter

Example ____950,000_- 550,000 400,000
2,380.95 240 exiters - 60-12
168 These 60 customers were not employed in
the 1st quarter after exit, so they are
excluded. Supplemental data was used to verify
employment for these 12 customers, so they are
excluded.
37
Setting Earnings Targets for Older Youth

• Sample Individual calculation

1,950 goal for PY 2003 (for 2nd 3rd quarters
after exit) 6,240 estimated customer earnings
in the 2nd 3rd quarters before
registration 8,190 required customer earnings
in the 2nd and 3rd quarters after exit, to
contribute positively to local goal

38
Setting Earnings Targets

• Earnings change is an aggregate measure.
• Serving customers with previous high earnings can
  be challenging.
• Serving some customers with zero previous
  earnings can make achieving the earnings change
  measures a little easier.

39
Setting Earnings Targets
Other Helpful Information

• Hourly wages X hours worked per week weekly
  income
• Weekly income X 26 (weeks in 2 quarters)
  earnings in 2nd and 3rd quarters before
  registration or after exit
• Local or state goal / 26 / 40 Hourly wage, at
  full-time
• 2 Quarters income X 2 Annual Income
• Quarterly income / 26 / 40 Hourly Wage, at
  full-time

40
Older Youth Credential Rate

• Official definition The number of Older Youth
  who are employed or are in post-secondary
  education or advanced training in the 1st quarter
  after exit and receive a credential by the end of
  the 3rd quarter after exit divided by the number
  of Older Youth who exit during the quarter.

41
Older YouthCredential Rate

• Simplified definition Measures percentage of
  all Older Youth who earn a credential during the
  program or within several months leaving
  services. They must also be employed, or be in
  postsecondary education or advanced training soon
  after leaving the program.
• All Older Youth are included in the credential
  measure.

42
Older Youth Credential Rate
Positive Performance Outcome

• Older Youth receive a credential by the end of
  the the third quarter after exit AND are employed
  or in postsecondary or advanced training during
  the 1st quarter after exit.

• Not employed or in post secondary education or
  advanced training during the 1st quarter after
  exit results in a negative even if a credential
  is attained!

43
Older YouthCredential Rate

• Youth complete program services.
• A credential policy is clearly articulated.
• Training tied to viable credential.
• Effective follow up services and tracking are
  provided identifying, getting and recording
  credentials.

Key Factors Contributing to a Positive Outcome
44
Older YouthCredential Rate

• Calculation
• Number who attain a credential by 3rd qtr. after
  exit and are employed, in advanced training or
  post-secondary education in 1st qtr after exit
• divided by
• Number of Exiters during the Quarter

Example __120w/credential by 3rd qtr.
after exit 25 95__ 63 150
exiters These 25 were not employed or
enrolled in advanced training or postsecondary
education in the 1st qtr. after exit.
45
Older Youth Performance Measures Quiz
Please refer to Handout B
46
Part II. Focus on Earnings Change and Credentials
47
Factors Affecting Performance on Earnings Change

• Program Design and Policy
• Program Operations
• Program Analysis

48
Program Design and Policy

• Who Are the Participants?
• What Are their Prior Wages?
• Enrollment in Older Youth vs. Adult/Dislocated
  Program
• Approach to Training and Credentials
• Intentional Retention

49
Program Operations

• Assessment and Participant Goals
• Selection of Providers (OY vs. YY)
• Job Development
• Credential Achievement
• Specific Retention Activities

50
Program Analysis

• Pre and Post Wages
• Earnings by Post Quarters
• Earnings by Training vs. No Training
• Different Participant Earnings
• Different Provider Earnings

51
Credentials
52
What type of credentials did youth obtain?
HS/GED 41
Occupational Certificate 21
AA/BA 2
Occupational License 1
Others 35
53
Key Factors Contributing Positive Outcomes
Obtains Credential by 3d Quarter

• WIB has a clear credential policy
• Most youth are enrolled in training programs
• Most training programs lead to viable
  credentials
• Most youth successfully complete training
  programs
• All youth receive effective follow up services
• Outcomes are tracked and recorded

54
Key Factors Contributing Positive Outcomes
In Employment in 1st quarter

• Avoid drop-outs
• Avoid premature exits
• All youth receive effective follow up services
• Employment retention
• Re-employment and career advancement
• Needed supportive services
• All outcomes are tracked and recorded

55

• Labor Market Considerations

What available jobs are appropriate for young
workers?
Will employers hire young workers?
What skills are required for these jobs?
What credentials validate the skills and are
valued by local employers?
56

• System Design Considerations

Which options make the most sense for the youth?
Who offers the training that results in
credentials and employment?
Who offers other service supports?
What additional partnerships are needed?
Who will plan and manage record keeping, tracking
and ongoing performance management?
57
Selecting a Provider Who Can Best Support and
Train Older Youth

• Employ staff who form trusting relationships
  with
  youth
• Provide reliable information about careers
• Help youth make realistic career decisions
• Help youth meet prerequisites
• Provide ongoing support
• Have strong employer connections
• Provide strong placement support
• Offer ongoing retention and advancement services

58
Exit Readiness

• Keep older youth enrolled
• Until they are ready to be employed, stay
  employed, or to become re-employed if they leave
  their placement job
• Until they are equipped to earn higher wages than
  when they entered the program
• Until the goals set forth in their IEP have been
  attained.
• You may want to keep them registered until they
  have their credential and you have documentation.
  

59
Exercise
Please refer to Handout C
60
Part III. PROGRAM DESIGN
61
Program Design Considerations

• Make an Investment
• Provide Comprehensive Services
• Develop Needed Collaborations
• Ensure Effective Provider Capacity
• Manage the Overall Youth System

62
Make An Investment
63
Make An Investment

• Think about
• How are you adding value to youth participants so
  that they succeed in the labor market?
• What tangible services or products do you provide
  that equip youth for long term labor market
  success?

64
Make An Investment

• Add value to participants by
• Grounding all activities in the realities of the
  local labor market and real employers.
• Providing meaningful opportunities for job
  shadowing, work experience and internships.
• Developing both soft skills and occupational
  skillsnot one or the other.
• Developing a strong employer orientation for all
  programs for youth.

65
Provide ComprehensiveServices
66
Provide Comprehensive Services

• Think about
• Have you considered all components of a
  comprehensive program?
• Have you considered the difference between
  serving in and out of school youth and older and
  younger youth?

67
Provide Comprehensive Services

• Key Program Components
• Recruitment
• Enrollment and assessment
• Design and Delivery of the 10 Elements
• Meaningful and relevant activities
• In-program supports and retention activities
• Connection to measurable outcomes
• Post-exit retention services

68
Provide Comprehensive Services

• What are the key differences in program design
  between in school and out of school youth?

69
Provide Comprehensive Services

• What are the key differences in program design
  between younger and older youth?

70
Develop NeededCollaborations
71
Develop Needed Collaborations

• Think about
• How do you maximize the WIA infrastructure to
  best serve youth?
• How do you design the infrastructure to leverage
  other resources in the community available to
  youth?

72
Develop Needed Collaborations
Optimize the WIA Infrastructureby focusing on

• One stops
• Employment Security
• Youth providers
• Employer community
• Education and training system

73
Develop Needed Collaborations

• Optimize and leverage other systems
• Education System (ADA funding)
• TANF
• Social Service and Health
• Juvenile Justice System
• Literacy, ABE funds
• Vocational Rehab
• Community Organizations

74
Ensure Effective Provider Capacity
75
Ensure EffectiveProvider Capacity

• Think about
• Are you working with providers that have the
  capacity to serve youth effectively?
• Are you closely monitoring the timely and
  effective delivery of youth services?

76
Ensure EffectiveProvider Capacity

• Effective Capacity of Providers includes
• Ability to recruit youth.
• Ability to engage and keep youth (low program
  drop-outs).
• Ties to employment and education communities.
• Services and activities tied to measurable
  outcomes.
• Reliable and believable management information
  systems.
• Substantive follow up activities (post exit).

77
Ensure EffectiveProvider Capacity
Monitor the timely and effective delivery of
youth services by

• Making performance an explicit condition within
  contracts.
• Setting interim performance milestones/benchmarks
  and monitoring progress.
• Insisting on timely and reliable data submissions.

78
Manage the OverallYouth System
79
Manage the OverallYouth System

• Think about
• Do youth council members, staff and others fully
  understand the youth performance measures?
• Do youth council staff have the capacity to track
  and analyze activities and results?

80
Manage the OverallYouth System
Understanding the performance measures includes

• Youth council members understand content and
  consequences of measures.
• Management/administrative staff understand
  content and steps to achieving measures.
• Frontline staff understand content and
  implications of program actions (enrollment,
  service, exit, follow up).

81
Manage the OverallYouth System

• Capacity to track and analyze performance
• Use real time MIS.
• Dedicate analytical staff to
• Ensure good data keeping.
• Oversee achievement of provider milestones.
• Assess ongoing achievement of performance
  measures data.

82
Effective Follow-up/Retention/ and Advancement
StrategiesEffective RFP/Contracting Process
Part IV. Two Key Components of Successful Programs
83
Our Mantra What happens after exit is as
important as what happens before!
In Program
In Follow-Up
Pre-Program
Exit
Registration
84
What is Follow- up?
What is Retention?
85
Allowable and Required Follow-up Services

• According to USDOL WIA Youth Office Staff

• Types and duration of services must be based on
  needs of the individual.

All youth must receive some follow-up services
for a minimum of 12 months.
Follow-up services may include a wide range of
services that help retain a youth or lead the
youth back to a "positive outcome." For example,
this may include providing additional training to
help a youth retain or obtain a job.
86
What research shows
Job loss is very common in the first three to six
months
Problems with supportive services, particularly
child care are a major reason for job loss
Certain types of barriers such as health
problems, domestic violence, drug abuse, and
mental health problems keep people from working
steadily
Lack of soft skills is a major cause of job loss
87
Creating a Culture of Follow-Up/ Retention
Elements

• ACTIVITIES/SERVICES
• Effective Job Development and Replacement
  Services
• Active Labor Market/Post Secondary Connections
• Incentives
• Individual and Group Support
• Value-added Workshops
• Accessible Supportive Services

• QUALITY DATA
• Data Entry and Verification

• LOCAL STRUCTURES
• Contracting Options
• Smooth Transitions
• Caseload Considerations

88
Contracting Options
A. Add follow-up to existing youth contracts
B. Use a separate contract for follow-up
C. Add follow-up to an overall case management
contract
89
A. Add Follow-Up to Existing Youth Contract

• Possible benefits
• Youth have seamless relationship with staff
  through program services follow-up
• Deeper relationships with program staff may make
  youth more responsive in follow-up
• Possible drawbacks
• If in-program services contractor changes,
  follow-up relationship disrupted
• Many contractors providing follow-up services
  with none developing significant staff expertise

90
B. Use a Separate Contract for Follow-Up

• Possible benefits
• Could have just one follow-up provider
  specializing in follow-up services
• Fewer problems with youth left hanging if program
  contract changes
• Possible drawbacks
• Youth have little or no relationship with
  follow-up staff prior to placement

91
C. Add Follow-Up to an Overall Case Management
Contract

• Possible benefits
• Youth have seamless relationship with staff
  through entire WIA experience
• Possible drawbacks
• Relationship with case manager not as deep
  because case manager not involved in providing
  program services
• With short contracting periods, youth may lose
  their case manager before even entering follow-up

92
The RFP/Contracting Process
93
Select the Right Providers

• What is the product you are buying through a
  youth WIA contract?
• What are you paying for?

94

• Performance!
• (not just the 10 elements!)

95
Buyingthe RFP process

• Build RFP around outcomes (performance)
• Make RFP process short, simple, and fair
• Make evaluation criteria clear use them
• Avoid paper partnerships
• Ensure that older youth providers understand
  earnings change and credential requirements.
• Rate providers based on their ability to achieve
  outcomes.

96
Proposals and contracts

• Avoid one year contracts
• Ask bidders to spell out activities related to
  job development and retention
• Ask for descriptions of staff roles and
  responsibilities and budget information related
  to job development and retention
• Ask for information on case load management

97
Exercise
Please refer to Handout D
98
Remaining questions?
Carol Clymer Cclymer_at_ppv.org Public/Private
Ventures 2000 Market Street Suite
600 Philadelphia, PA 19103

• Brandon Roberts robert3_at_starpower.net