National Postschool Outcomes Center

1

• National Postschool Outcomes Center
• Floridas Experience in Accessing Administrative
  Records
• Teleconference
• May 24, 2005
• Jay Pfeiffer, Assistant Deputy Commissioner
• Division of Accountability, Research, and
  Measurement
• Florida Department of Education
• 850-245-0437
• Jay.Pfeiffer_at_FLDOE.org

2
K-20 Governance Organization Alignment of Data
Systems
Office of K-20 Education Information
Accountability
Additional role Department-wide conduit for LMS,
LED, Census, BEBR publications
Source Information Systems
Integrating Systems
Community College Technical Center
K-20 Education Data Warehouse
Public Schools
University
FETPIP
K-12
Employment Earnings Postsecondary
Ed. Welfare Incarceration Licensure
Adult Education Career/Technical Associate Degree
Baccalaureate Masters Ph.D./Professional
Source Systems Employment Assessment Financial
Aid Teacher Certification Facilities
Annual Administration Accountability
Continuous/Longitudinal
3
FETPIP The Florida Education and
Training Placement Information Program
4
The Unemployment Insurance Wage Report
represents the best potential comprehensive
source of occupational and wage information in
the state, with the addition of occupational
titles to the report. This would be useful in
labor market demand forecasting and analysis
economic analysisevaluation and
follow-up Polivka, Larry et al. Vocational
Education Study Labor Market Demand Information
in Florida Policy Issues and Alternatives
Office of the Governor, August 1982 Tallahassee
Florida
Key Background Studies and Recommendations
Priority should be given to examining the
addition of occupational information to the UI
Wage Report file. Herndon, John, et al. Project
to Coordinate Economic Development, Employment
and Training Report 1 Issues and
Recommendations, Report to the Governor 1 of
the EDIT Project, March 1982 Tallahassee Florida
Regarding the wage record system The
addition of a single data element, an
occupational title, would provide the possibility
for the most useful single data base in the State
of Florida. The Florida Occupational
Information Coordinating Committee A suggested
approach to developing a comprehensive
occupational information data base for Florida
from existing data resource. A concept paper,
Florida Department of Labor and Employment
Security, June 1981 Tallahassee Florida

• Instead of maintaining a separate program to
  collect industry occupational information, it
  would be more efficient in terms of resources
  expended to merge the data collection phase of
  the Occupational Employment Statistics (OES)
  program with the ES 202 program.
• Floridas Occupational Information and Delivery
  System, Technical Assistance Project, U.S.
  Department of Commerce, Project Number
  04-06-01109, May 1975 Tallahassee Florida

5
Initial Legislative Direction

• Funds provided in Specific Appropriations 337A
  shall be used to conduct a study of the
  feasibility of implementing an automated system
  that matches social security numbers of persons
  completing vocational programs with wage,
  geographic, and occupational title data available
  through the Unemployment Insurance Wage Report
  and the Workers Compensation system.
• Chapter 84-220 Laws of Florida, Specific
  Appropriation 337A Tallahassee Florida

• Specific Appropriation 337A established two
  important precedents
• Administrative records might have uses other than
  that which they were originally designed for
  and,
• Education money went to the DLES under the
  tutelage of the FCOVE and the SJTCC with the DOE
  reporting back results

6
DEVELOPMENTS LEADING TO FETPIP'S CURRENT STATUS
7
Florida Education and Training Placement
Information Program (FETPIP) Annual Collection
Cycle
February, 2003
FL UC Covered Employment
High School
Performance and Accountability Reports
Fed Government Postal Service
Vocational Center
Uniformed Military Service
Community College
Workforce Funding Formula
Out-of-State Employment
Public University
Out-of-State Postsecondary Ed.
Independent University
Ad Hoc Reports
Merge Files Unduplicate Edit Names SSN
System Exits
Public University Enrollment
Other Programs
Public Adult Ed. Vocational Enrollment
CHOICES Career Counseling Software
Federal Programs
Private College University Enrollment
Create Master Follow-up File
Welfare Transition Services
Public Community College Enrollment
FL Corrections Releases
Internet Education WIA Reports
Create Tapes Files for Record Links
FL Corrections Incarcerations
Special Projects, Longitudes
Public Assistance Receipt
Inputs
Outputs
Linked Files
8
Input Data
Examples

• Student/Participant - demographic,
  socio-economic, enrollment/services, educational
  programs, promotion, attendance, GPA, test
  scores
• Educational Institution - types, location,
  performance statistics, private/public
• Financial Aid - federal/state/local,
  loans/grants, amounts disbursed
• Courses - offerings, student teacher
  participation, instruction type, grades
• Educational Staff - demographic, awards,
  certification, teaching work activity
• Educational Awards - type, program, institution,
  date

9
Regular Publication of Annual Performance
Accountability Data
Annual performance data include employment rates,
earnings, full-time employment proxies,
continuing postsecondary education statistics by
educational sector, public assistance
participation, and incarceration

10
Most High School Grads Nation-wide Go On To
Postsecondary Within 2 Years
Source NELS 88, Second (1992) and Third (1994)
Follow up in, USDOE, NCES, Access to
Postsecondary Education for the 1992 High School
Graduates, 1998, Table 2.
11
(No Transcript)
12
Regular Publication of Longitudinal Information
for Select Cohorts


1991 Public High School Graduates in 2002
Standard Diplomas
Information may be aggregated over long periods
of time to show longer term results of continuing
to pursue educational opportunities, employment,
earnings, and interactions of various factors
Source FETPIP Longitudinal

13
Florida Public High School Class of 1991 Earnings
by Educational Credential in 2002
Source FETPIP Longitudinal
14
FETPIP Data
Major Uses

• Public Education Funding
• Performance-Based Budgeting
• Federal Performance Measurement WIA,
  Wagner-Peyser, Voc Rehab, Blind Services,
  Perkins, NRS (Adult Education)
• Private School Licensure
• School District Accountability Reports
• Community College Performance Reports
• University System Program Reviews
• ICUF Accountability Reporting
• Legislative Performance Reviews
• Workforce Estimating Conference
• Policy Research
• Career Counseling, Consumer and Guidance
  Information

15
FETPIP-Type Operations
Benefits

• Use existing data resources
• Cheaper, more accurate, more complete, and with
  more capabilities, than traditional surveys
• Data are regarded as objective- non-judgmental
• Provide a basis for common core sets of outcome
  measures, comparisons, discussions of issues
  across many programs operations

16
With such a system, what can you expect to hear?
Part 1

• Wow! You can do that?
• If I provide you with a set of student data can
  you?
• How much will this cost me?
• Why should I participate in this?

17
With such a system, what can you expect to hear?
Part 2

• The System
• is too slow
• is not done frequently enough
• doesnt cover students without correct SSNs
• misses types of employment
• misses types of postsecondary education
• doesnt cover the other, more important outcomes

18
Challenges (part 1)

• Administrative data are designed for purposes
  other than performance measurement
• Definitions, taxonomies may differ
• Time periods may differ
• What is measured may not be exactly what is
  desired
• Administrative data change over time
• Owners of Administrative Data have legal,
  ethical, proprietary obligations
• They wont naturally want to share
• Legal beliefs may be conjured up
• Program interests may be different from those of
  technical staff
• The owners change over time

19
Challenges (part 2)

• Confidentiality of Information
• Requirements may be different for each data
  source
• The issue is complicated by legal issues as well
  as beliefs
• Beliefs/requirements may change over time
• Weakest Links
• The strength of a system of linked data sets is
  determined by the weakest of the data resources
• Opponents will focus on the weaknesses
• The data lag actual performance
• Support
• On going, reliable funding
• On going, reliable political support
• Cooperative attitude among stake holders
• All involved will change

20
What has worked for FETPIP?
Support

• Statutory authority, provisions, including cross
  references
• Core funding from State General Revenue
  appropriated by the Legislature
• Persistently collaborate with legislative,
  executive staff
• Persistently collaborate with data owners,
  stakeholders in decisions about the mechanics of
  exchanges, reporting of results
• Constantly anticipate roles that the system can
  play in accountability, performance measurement,
  cost return analysis, evaluation
• Consistently work with data owners on changes,
  data anomalies, problems
• Steadfastly make it easy for stake holders to
  participate, reduce burdens, provide data in
  exchange
• Always facilitate discussions about common data
  elements, definitions, core performance measures,
  performance funding

21
What has worked for FETPIP?
Confidentiality/Security

• Location within the state education agency
• Always surpass all requirements, expectations
  regarding data security
• Faithfully limit uses to statistical purposes for
  evaluation, program improvement
• Persistently keep exchange agreements current and
  up-to-date, continually brief employees, stake
  holders on security and confidentiality issues
• Always refer inquiries about particular resources
  to the resource owners

22
What has worked for FETPIP?
Changes

• Strive to meet changes, challenges head-on
• Persistently refresh understanding of resources,
  requirements, limitations
• Persistently seek new data resources, expand
  coverage
• Always revise business rules as understanding,
  conditions change
• Establish short term and long term visions for
  the effort

23
What has worked for FETPIP?
Other

• Establish archival data resources
• Accommodate ad hoc interests and requirements
• Seek out opportunities to demonstrate your wares
• Always collaborate

24
K20 EDW Floridas K20 Education Data Warehouse
25
Floridas K20 Education Data Warehouse
26
K20 Data Warehouse Student-Centered
Demography
Educational Awards
Educational Institution

Educational Staff
Financial Aid
Student
Student Employment
Courses
Grades, Standardized Test Scores