Reporting and Data Validation Updates

1
U.S. Department of LaborEmployment and Training
Administration

• Reporting and Data Validation Updates
• Presenters Wes Day
• Barbara Strother
• Greg Wilson
• ETAs Office of Performance and Technology

2
Presentation Overview

• Discuss WIA and Wagner-Peyser reporting
  requirements and due dates
• Review ETAs performance reporting system
• Review ETAs data validation initiative and new
  software

3
Required Reports

• Annual Report required for WIA
• 9091
• Quarterly reports are required for WIA and
  Wagner-Peyser/VETS
• 9090 and 9002A-E, VETS 200A-C
• Monthly WIA and Wagner-Peyser reports (ARRA)
• 9147, 9148, 9149
• Workforce Investment Act Standardized Record Data
  (WIASRD)

4
Annual PY 2008 DRAFT Due Dates

• Monday, August 17, 2009 Validated PY 2008 4th
  quarter 9002 and VETS 200 reports
• Monday, August 17, 2009 Wagner-Peyser Report
  Validation and Data Element Validation summaries
  for PY 2008 4th quarter 9002 and VETS 200 report
• Thursday, October 1, 2009 Validated PY 2008 WIA
  Annual Report
• Thursday, October 1, 2009 WIA Report Validation
  summary
• Thursday, October 15, 2009 PY 2008 WIASRD file
• Tuesday, February 1, 2010 WIA Data Element
  Validation
• NOTE All files must be submitted by 1159 PM
  Eastern Daylight Time

5
Quarterly and Monthly Due Dates

• Quarterly Reports are due 45 days after the end
  of the quarter.
• Example 1/1/09 - 3/31/09 report due 5/15/09
• Monthly Reports are due 15 days after the end of
  the month.
• Example 7/1/09 - 7/31/09 report due 8/17/09
• First report due July 15th

6
Grantee Reporting System - EBSS

• States submit their reports to ETA using the
  Grantee Reporting System (aka EBSS)
• EBSS Enterprise Business Support System
• http//www.etareports.doleta.gov

7
Grantee Reporting Log In Screen
http//www.etareports.doleta.gov
8
Passwords

• One for Wagner-Peyser Reporting
• One for WIA Reporting
• One for Wagner-Peyser Data Validation
• One for WIA Data Validation
• For password assistance, contact
  ebss.help_at_dol.gov

9
The WIA Screen
10
Labor Exchange Screen
11
The WIA Screen - Monthly
12
Monthly Performance Reporting
13
WIA Adult Monthly Report
14
Monthly Reporting Guidance

• Training and Employment Guidance Letter No.
  24-08 includes
• Detailed descriptions of reports
• Report specifications
• Timelines
• Available at http//doleta.gov/directives/

15
The WIA Screen - Quarterly
16
WIA Quarterly Report
17
WIA Quarterly Report
18
WIA Quarterly Reporting Guidance

• Workforce Investment Act Quarterly Report
  General Reporting Instructions and ETA Form 9090
• Available at http//www.doleta.gov/performance/R
  eporting

19
The WIA Screen - Annual
20
WIA Annual
21
WIA Annual Table A
22
Table O Local Performance

• Table O contains information by Local Workforce
  Board
• States should review to see if ETA has the most
  current listing of local areas and their ETA
  assigned numbers
• States should send any updates to ETA no later
  than August 25
• send updates to ebss.help_at_dol.gov

23
Cross Form Edit Checks
24
WIA Annual Reporting Guidance

• Workforce Investment Act Annual Report General
  Reporting Instructions and ETA Form 9091
• Available at http//www.doleta.gov/performance/R
  eporting

25
The WIA Screen - WIASRD
26
WIASRD - Intro

• Workforce Investment Act Standardized Record
  Data (WIASRD)
• Individual records (not aggregate)
• Very important for ad-hoc analysis
• Type of training
• Demographics
• Outcomes

27
WIASRD - Updates

• Updates for 2008
• Adds valid values to Other Non-WIA Programs
  (WIASRD field 326) to allow states to code
  American Recovery and Reinvestment Act of 2009
  co-enrollments
• Designates WIASRD field 313c as a Special
  Projects field

28
WIASRD Submission Tips

• Uploading WIASRD files to EBSS can be very time
  consuming.
• ETA processes WIASRD files overnight and provides
  the state with a status the following day.
• Some tips to reduce WIASRD upload time include
• Zip your file
• Remove unnecessary characters
• Firefox is faster than IE

29
WIASRD Tips Continued

• The slow way 626 characters
• "000000000","18810118","2","3","2","2","2","3","3
  ","3","2","1","2","1","3","1","2","2","2","12","2"
  ,"","2","2","1","1","4","1","00000","19020503","19
  021010","2","2","1","2","2","2","2","","2","","","
  ","","","","","","","","","","","","","","","","",
  "1","2","","","2","2","2","2","999999999","0","2",
  "1","2","2","2","1","1","","","","1","1","2","1","
  ","1","1","02001.00","","01712.00","","03112.00","
  03094.00","01182.00","00000.00","00000.00","5","",
  "3","3","","","","","","","","","","","","","","",
  "","","","","","","","","","","","","","","","",""
  ,"","","","","","","","","","","","","","","","","
  ","","","","
• The fast way 283 characters
• 000000000,18810118,2,3,2,2,2,1,2,2,3,3,3,2,1,2,1,
  3,1,2,2,2,12,2,,2,2,1,1,4,1,00000,19020503,1902101
  0,2,2,1,2,2,2,2,,2,,,,,,,,,,,,,,,,,,,,1,2,,,2,2,2,
  2,999999999,0,2,1,2,2,2,1,1,,,,1,1,2,1,,1,1,2001,,
  1712,,3112,03094,1182,0,0,5,,3,3,,,,,,,,,,,,,,,,,,
  ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

30
WIASRD The Future

• Quarterly submissions beginning March 31, 2010.
• Includes participants and exiters.
• Now is the time to fix problems.

31
Dont Forget Wagner-Peyser!

• Monthly and Quarterly Reports
• Submitted to ETA via EBSS
• Quarterly Guidance ETA 9002 and VETS 200 Data
  Preparation Handbook No. 406
• Monthly Guidance Training and Employment
  Guidance Letter No. 24-08

32
Basic Reporting Review

• Monthly WIA and Wagner-Peyser Reports
• Quarterly WIA and Wagner-Peyser Reports
• Annual WIA Report
• WIASRD

33
Part 2 Data Validation

• ETAs data informs the Legislative and Executive
  branches of government as well as media,
  researchers, and the public.
• Without Data Validation, ETA is not able to
  assess the accuracy of its data.

34
How Does Data Validation Work?

• Two separate processes are required to ensure
  that performance data is reliable
• Report Validation ensures that aggregate reports
  are calculated correctly.
• Data Element Validation ensures that the data
  elements in the participant records used to
  calculate aggregate reports are accurate.

35
Data Validation Guidance

• TEN 14-02 To inform the workforce investment
  system of the data validation initiative for
  employment and training programs. Issued May 28,
  2003.
• TEGL 3-03 Data Validation Policy for Employment
  and Training Programs. Issued August 20, 2003.
• TEGL 3-03, Change 1. Issued August 20, 2004.
• TEGL 3-03, Change 2. Issued October 20, 2004.
• TEGL 3-03, Change 3. Issued July 15, 2005.
• TEN 9-06 Program Year 2005/Fiscal Year 2006
  Performance Reporting and Data Validation
  Timelines. Issued August 17, 2006.
• TEN 19-07 Program Year 2006/Fiscal Year 2007
  Performance Reporting and Data Validation
  Timelines. Issued December 11, 2007.
• TEN 09-08 Program Year 2007/Fiscal Year 2008
  Performance Reporting and Data Validation
  Timelines. Issued September 3, 2008

36
Data Reporting and Validation Software

• Data Reporting and Validation Software (DRVS)
  version 7.2 released May 6, 2009.
• Use for PY2008 Annual Report and 2009 Quarterly
  reports
• Adds valid values to Other Non-WIA Programs
  (WIASRD 326) to allow states to code American
  Recovery and Reinvestment Act of 2009
  co-enrollments
• Fully implements the Literacy/Numeracy
  calculation per TEGL 17-05, Change 2
• Designates WIASRD 313c as a Special Projects
  field

37
DRVS Available Here!
DRVS is available at http//www.doleta.gov/perfor
mance/Reporting/tools_datavalidation.cfm
38
DRVS is Essential to Reporting

• Processes the WIA Quarterly and Annual Reports,
  and the Wagner-Peyser/VETS quarterly performance
  reports
• Provides all functions necessary for validation
• Compares reports
• Sampling and data accuracy analysis
• Uploads performance reports and validation error
  reports to ETA

39
Reporting of Validation Results

• DRVS produces
• Report validation summary
• Data element validation summary and analytical
  reports
• WIA and LX DRVS creates files with the annual
  report validation values for upload to ETA

40
DRVS Reporting Options Screen

• This reporting option screen is for the WIA
  annual report covering Program Year 2008.

41
Creating RV Extract Files for WIA

• State IT staff produce extract file with
  participant records for all participants and
  exiters included on the appropriate report
• Extract file format is based on the WIASRD (with
  extra fields added to help locate participant
  files in one-stops sampling unit, WIB Name,
  Office Name, Case Manager, and User Field)

42
Loading Extract File into DRVS

• State staff import the extract file into the data
  validation software
• The software processes the file and builds
  performance outcome groups
• Multiple edit checks

43
Edit Check Error Report - Example
44
Reload Files if Necessary

• Review error and duplicate reports
• Determine whether the extract file must be
  regenerated or reformatted based on whether the
  number of errors is significant
• Load the entire modified extract file if
  necessary
• DRVS will not import additional records into a
  file that has already been imported.

45
Uploading the WIA Annual Report to DRVS

• To calculate the error rate, the software must
  have the states reported values
• Import the item totals from the WIA Annual Report
  into the reported count input form
• The validation software provides the capability
  to e-submit the report to ETA

46
Whew
47
Report Validation Summary

• After reported values from the WIA Annual Report
  have been imported the software produces the
  report validation summary
• Calculates the difference between the validation
  and reported values
• Displays an error rate for each reported value
• DRVS can e-submit the report to ETA

48
Report Validation Summary Example
49
How Does Data Validation Work? (again)

• Two separate processes are required to ensure
  that performance data is reliable
• Report Validation ensures that aggregate reports
  are calculated correctly.
• Data Element Validation ensures that the data
  elements in the participant records used to
  calculate aggregate reports are accurate.

50
Data Element Validation

• Report will not be accurate if the data being
  used by the software are wrong
• Requires checking data elements against source
  documentation to verify compliance with federal
  definitions
• States identify state-specific source
  documentation
• DRVS produces worksheets for state staff to enter
  source documentation review

51
Source Documents

• Source documents support the validity of a data
  element
• States should identify state-specific source
  documents

52
Location of Source Documents

• Most source documentation will be located in the
  One-Stop center
• Some source documentation will be stored at the
  state level
• Wage record information is checked against
  states wage record files
• There may be other central administrative data
  files

53
Sampling Methodology

• DRVS selects a sample of records from a sample of
  offices in the state
• Ensures that not every office needs to be visited
  for WIA
• DRVS selects a sample of records from each
  performance outcome
• Ensures that results will represent the overall
  quality of the states data

54
Overview of Tasks for Data Element Validation

• Determine how validation will be done
• Assemble worksheets and arrange visits to offices
• Gather supporting documentation at office and
  complete worksheets
• Upload result to DRVS and then submit to ETA

55
Required Reports - Revisited

• Annual Report required for WIA
• Quarterly reports are required for WIA and
  Wagner-Peyser/VETS
• Monthly WIA and Wagner-Peyser reports (ARRA)
• Workforce Investment Act Standardized Record Data
  (WIASRD)

and dont forget the Data Validation results!
56
Thats All Folks

• For additional information, please visit
• http//www.doleta.gov/performance/reporting
• Email
• etaperforms_at_dol.gov
• ebss.help_at_dol.gov