Texas%20Workforce%20Commission-%20Missing%20Link%20Project

1
Texas Workforce Commission- Missing Link Project

• Missing Link Project Update
• TAIR Annual Conference
• February 6, 2008
• James Loiselle
• Hiwot Berhane

2
Agenda

• Background
• ICF Approach
• Overview of Results
• Limitations
• Questions/Comments

3
Background

• Job creation and economic development a statewide
  priority
• Industry clusters facilitate this by
• Helping Texas efficiently monitor workforce
  conditions
• Bringing attention to industries and jobs key to
  the States growth
• Purpose of project
• Develop master database linking DWAs, CIPs, and
  WECM courses
• Provide a common set of linkages to bridge the
  gap between employers and workforce trainers

4
Aligning WECMs, CIPs, and DWAs A Stepwise
Approach(8 Total Tasks)
Step 1 Project Kick-off
Step 2 Edit and verify DWA to CIP links made by
TWC/LMCI staff
Step 3 Link DWA statements to WECM courses
Step 4 Gather SME Input to Refine and Validate
Ratings
5
Task 1 Project Kick-Off

• Gathered information regarding CIP, DWA, and WECM
  data sources
• TWC provided database of 411 technical CIP
  codes with unverified linkages to DWAs and SOC
  codes.

6
Database Structure of Mechanical Linkages
Provided By TWC

• CIP Code Related ONet Occupations
• DWA 1
• DWA 2
• DWA 3
• DWA 4

7
Task 2 Edit And Verify CIP and DWA Database
Structure

• ICF reviewed linkages made by TWC staff between
  the ONET occupations and CIP codes
• Remaining non-technical CIPs were removed from
  the original database
• Outcome Database of 384 technical CIP codes
  with unverified linkages to DWAs and SOC codes.

8
Task 3 Link DWA statements to WECM courses

• Decided Linking DWA to WECMs ? best way to
  demonstrate a link between a DWA and a CIP
• Developed one Access database for each of the 384
  CIPs
• Created a specialized Access form within each
  database to aid the rating process
• Conducted ratings of each DWA to WECM
  relationship for each CIP
• Outcome Complete ratings for all potential DWA
  to WECM relationships in every CIP-specific
  database

9
Task 3 Link DWA statements to WECM courses
Number of Elements in Rating Process and Estimated Ratings Made By ICF Team Number of Elements in Rating Process and Estimated Ratings Made By ICF Team
Total Number of CIPs 384
Number of CIPs with related WECM information 309
Number of Blank CIPs (i.e., no related WECMs) 75
Number of WECMs included in linkage process 4334
Number of DWAs included in linkage process 2165
Approximate number of DWA to WECM linkage ratings 350,000
Note 1 4461 WECMs were provided by TWC in
original database. After 127 duplicates were
removed, 4334 WECMs remained. Note 2
Individual DWAs, by nature, were typically linked
to multiple CIPs.
10
Task 3 Link DWA statements to WECM courses

• CIP Code Occupations
• Course 1 DWA 1
• Course 2 DWA 2
• Course 3 DWA 3
• Course 4 DWA 4

11
Task 3 Link DWA statements to WECM courses

Scale to Determine DWA to WECM linkage Scale to Determine DWA to WECM linkage
Is a student better prepared to perform the DWA after completing this WECM? Is a student better prepared to perform the DWA after completing this WECM?
Rating Option Description
Yes DWA covered in the WECM.
No DWA not covered in the WECM.
Maybe Unclear CIP/WECM WECM includes technical term requiring input from Subject Matter Experts (SMEs).
Maybe Unclear DWA DWA includes technical term requiring input from SMEs.
Maybe Don't Know DWA and WECM content are clear, but need SME input to make judgment.
Special Topic WECM is a Special Topics Course content subject to change over time and across schools. Unable to identify reliable linkage.
12
Task 3 Link DWA statements to WECM courses

• Intent was to identify WECMs where knowledge or
  skill related to DWA could be obtained.
• Distinguishing Yes/No ratings sometimes difficult
  so rules developed for raters/ SMEs

Summary of Rating Rules Used to Distinguish Yes/No Ratings Summary of Rating Rules Used to Distinguish Yes/No Ratings
Rating Situation
Yes Student is better prepared to perform the DWA after completing WECM than before the WECM.
Yes WECM introduced basic elements of DWA although student does not have complete mastery of the DWA.
Yes WECM introduces the theory behind a DWA although student has not performed the DWA in the WECM.
No DWA involves providing general supervision or performing a broad activity not related to the content of the WECM .
No Individual who has just completed WECM is no better at performing DWA than prior to WECM.
No Student may be better able to perform DWA after WECM but performance improvement is not related to WECM content
13
Task 4 Gather SME Input to Refine and Validate
Ratings

• Gathered input from SMEs to refine/validate the
  DWA to WECM ratings
• Assigned databases based on SME area of
  specialization.
• SMEs reviewed three types of Maybe ratings to
  change the Maybe ratings into Yes/No ratings.
• SMEs reviewed existing Yes/No ratings.
• SME selection, use, and affiliation described on
  subsequent slides.
• Outcome Verified Yes, No, or Special Topic
  rating for each DWA to WECM relationship.

14
Task 4 Gather SME Input to Refine and Validate
Ratings

• Selection and Use of Project SMEs
• SMEs selected based on knowledge of technical
  CIPs under study.
• CIPs clustered into 28 Program Areas (e.g.,
  Mathematics and Statistics, Visual and Performing
  Arts, Engineering, Construction Trades).
• SMEs asked to indicate areas of expertise using
  defined program clusters.
• SMEs used to resolve Maybes where raters had
  questions about CIP/ WECM content and/or
  technical aspects of work being performed.
• 32 SMEs were used to rectify Maybe ratings
  across all CIPs and verify Yes/No ratings
• SMEs included technical college faculty,
  university grad students, and education/work
  specialists from 10 organizations including ICF.

15
Task 4 Gather SME Input to Refine and Validate
Ratings
Summary of Participating SMEs Summary of Participating SMEs Summary of Participating SMEs
Number of SMEs Organization Location
1 Temple College Temple, TX
3 Angelina College Lufkin, TX
1 San Jacinto College Pasadena, TX
1 Dallas County Community College Dallas, TX
1 Lone Star Cy-Fair College Cypress, TX
3 Texas AM University College Station, TX
5 Angelo State University San Angelo, TX
1 Penn State University State College, PA
8 George Mason University Fairfax, VA
8 ICF International Fairfax, VA
32 SMEs 10 Organizations
16
Aligning WECMs, CIPs, and DWAs A Stepwise
Approach(8 Total Tasks)
Step 5 Identify and Add Missing DWAs for Each CIP
Step 6 Conduct roll-up for DWA to CIP linkages
Step 7 Link DWA statements to WECM courses
Step 8 Create master database
17
Task 5 Identify and Add Missing DWAs for Each
CIP

• Reviewed each CIP database to determine whether
  additional DWAs should be added
• Compared the original list of linked DWAs to the
  full list of DWAs provided in ONET, which were
  clustered into program (CIP) groups
• Outcome All relevant DWAs represented in CIP to
  DWA linkages.

18
Task 5 Identify and Add Missing DWAs for Each
CIP

• CIP Code Related ONet Occupations
• Course 1 DWA 1
• Course 2 DWA 2
• Course 3 DWA 3
• Course 4 DWA 4
• DWA 18

DWA 18Newly identified DWA currently addressed
through WECM but not originally linked to CIP
through mechanical linkages conducted by TWC.
19
Task 6 Conduct Roll-Up for DWA to CIP Linkages

• Used roll-up strategy to define relationships
  between DWAs and CIPs.
• DWAs considered linked to a CIP (i.e., assigned a
  Yes rating) if linked to one or more WECMs
  related to the CIP.
• CIPs considered not linked (i.e., assigned a No
  rating) if not linked to any WECMs related to the
  CIP
• Outcome Yes/ No rating indicating DWAs
  linked to each CIP

20
Task 7 Conduct Quality Control Check of
Databases

• Reviewed each CIP database to verify all ratings
  accurate, structure and content of the databases
  sound, and redundancies removed.
• Removed duplicate WECMs, verified variable/field
  name consistency, and updated forms to facilitate
  data read, edit, and present functions.
• Outcome Final databases for each CIP.

21
Task 8 Create Database of Identified Linkages

• Created a user-friendly, searchable master
  database that incorporates and reflects the new
  and edited relationships between WECM Courses,
  CIP codes, and DWAs.
• Outcome Final master database combining all
  linkages across CIPs, DWAs and WECMs while
  providing easy search capability.

22
Results- Deliverable Overview

23
Deliverable Category 1

• 309 CIP Databases with 3 tables and 2 forms each.
  Sample shown below
• Tables Forms
• CIP150305 CIP150305
• CIP150305_RollUp WECM_w_CIPinfo_listofsocs_Mast
  er
• WECM_w_CIPinfo_listofsocs_Master

24
Deliverable Category 2

• Consolidated Database with 309 tables each
  representing one CIP.

25
Navigating a CIP Database - 521904

• Select Forms on Objects menu gt Open
  WECM_w_CIPinfo_listofsocs_Master
• View CIP details, related WECMs with details, and
  related DWAs with ratings

26
Limitations of Deliverables

• ONET DWAs are sometimes vague, incomplete,
  and/or not aligned with the needs of Texas
  industries limiting the application of results
• WECMs do not cover the gamut of skills trained
  within CIPs across schools, just common ones.
  Course and program content varies across
  locations to meet local needs limiting the
  reliability of linkages
• SMEs used were from academic community rather
  than industry so results may be biased to
  educators point of view
• Accuracy of deliverables dependent on
  currency/validity of source crosswalks (e.g.,
  SOC-CIP, DWA-SOC, WECM-CIP crosswalks)

27
Contact
For any additional information or questions,
please contact James Loiselle Research
Specialist Texas Workforce Commission (512)
491-4974 james.loiselle_at_cdr.state.tx.us