Job Vacancy Survey Workshop

1
Job Vacancy Survey Workshop
April 22-23, 2003 St. Paul MN
2
Welcome
MDES Commissioner Harry Mares
3
Welcome

• Job vacancy information is required under WIA and
  the Perkins Act to plan workforce development
  policy and programs.
• A job vacancy survey is an innovative product
  that provides this information.

4
Welcome

• The Minnesota Job Vacancy Survey helps us
  understand the workforce needs in the state.
• It helps us to plan and implement an effective
  workforce development policy.
• This will help Minnesota remain competitive and
  economically vital.
• We want to learn from you!

5
Introductions
6
Presenters
MDES

• Oriane Casale
• Rachel Hillman
• Christina Chiu
• Jeff Willingham

• Curt Theis
• Annie Tietema

BLS
7
History of Minnesotas Survey
Oriane Casale
8
Minnesotas History

• Minnesota was one of 7 pilot states funded by the
  DOL ETA to conduct a job vacancy survey.
• In 1999, we partnered with the Metro Workforce
  Investment Board and the City of Minneapolis
  Employment and Training Program.
• Conducted two rounds of a Twin Cities survey.

9
History Continued

• Survey design based on the Milwaukee Survey of
  Job Openings conducted by the University of
  Wisconsin-Milwaukee since 1993.
• Used two rounds to refine survey instrument,
  sample design, data collection, and analysis.
• Findings are not comparable to current survey (or
  to each other).
• Results were well received by partners, media and
  Governors office.
• This created demand!

10
History Continued II

• MDES Commissioner saw a need for this information
  and was willing to fund statewide survey.
• We have since produced five rounds of a statewide
  survey.
• Success in Minnesota and other states including
  Colorado and Wisconsin has meant further ETA and
  BLS interest and support National Job Vacancy
  Statistics Workgroup.

11
Job Vacancy Statistics Workgroup
Oriane Casale Find more on this in About JVS,
Introduction.
12
Background

• The Workforce Information Council (WIC) has
  identified obtaining accurate job vacancy
  information from within the States and also
  nationwide as a vital goal.
• BLS ETA

13
Workgroup Goals

• Employ statistically rigorous methodology.
• Develop comparable statistics across States and
  local areas.
• Reduce startup and overhead costs.
• Promote the use of job vacancy statistics.

14
Participants

• Colorado
• Florida (2)
• Maine
• Maryland (2)
• Minnesota (2)
• New Mexico
• New York
• Wisconsin

• ETA
• BLS (4)

See Job Vacancy Statistics Project Brochure!
15
Progress

• Survey Instrument
• Sample Allocation and Selection System (BLS)
• Data Capture System (MDES)
• Estimates Production Systems (BLS)
• Job Vacancy Statistics Website
• Workshops

16
Progress Continued

• Access to latest news at www.JVSinfo.org
• or e-mail JVS.mail_at_state.mn.us
• JVS Software package
• Sample Allocation and Selection
• Data Capture System
• Estimates Production
• Ongoing technical support provided to the States
• How to Guide About JVS

17
Survey Timeline
Rachel Hillman Find more on this in About JVS,
Survey Timeline and Resources.
18
Survey Timeline
After the sample is drawn...

• Pre-Survey Resources
• Items in Mailings
• Pre-Mailing Activity
• Post-Survey Activity

See Survey Timeline handout!
and before the results are in.
19
Pre-Survey Resources

• Human Resources
• Communications
• Purchasing
• Follow-up Telephone Call
• Data Processing
• Mailing and Duplicating

20
Pre-Survey Activities

• Make Detailed Timeline
• Assignments

21
Pre-Mailing Activity

• Address Correction Process
• -Postcard
• Telephone for Contact Name
• Alternate Response Methods
• Supplies

22
Items in Mailing

• Postcard
• Cover Letter
• Follow-up Insert
• Survey Instrument
• (Well talk about that later.)

See Survey Insert Handouts!
23
Post-Survey Activity

• Data Entry
• Data Cleansing

24
Cost
Oriane Casale Find more on this in About JVS,
Cost.
25
Staff Costs

• Survey Coordinator Analyst
• Technical Support
• Administrative Staff
• Graphic Artist

See Cost Handouts!
26
Mail or Telephone

• A mail survey incurs costs for printing and
  mailing postcards, letters and surveys and for
  calls for sample refinement and survey follow-up.

• The alternative is to conduct the survey by
  phone or contract with someone who can, like the
  Colorado LMI Office.

27
Methods of Funding

• One-Stop LMI funds (MN, NJ, FL, WA)
• Reed Act (NJ)
• WIC Boards (MN)
• County/City (MN)
• Chamber of Commerce (KS)
• State/Private (CO)

28
Paper Survey
Rachel Hillman Find more on this in About JVS,
Data Collection Survey instrument.
29
Paper Survey

• One page
• Easily Faxable
• Field-tested (Minnesota) and cognitively tested
  (BLS)

Survey Instrument
See Survey Instrument handouts!
30
Paper Survey Continued
Survey Asks

• Job title Required
• Number of vacancies Required
• Full-time or part-time Required
• Temporary or permanent

31
Paper Survey II
Survey Asks

• Number of Days Open 4 categories
• Education 6 categories
• Experience 3 categories
• Certificate/License
• Wage Hourly Wage or Monthly, Annual Salary
• Benefits Up to 5 unique benefits

32
Cognitive Testing of Survey Instrument
Oriane Casale Find more on this in About JVS,
Data Collection Survey instrument.
33
Cognitive Testing

• Purpose
• Is format easy to follow?
• Do respondents understand survey questions and do
  they fill it out as intended?
• Can respondents access information requested and
  if so, how easily?

34
Cognitive Testing Continued

• 23 completed interviews
• 18 in-person
• 5 over the phone
• Respondents were asked to
• Comment in general on the form
• Express their impressions while filling out the
  form
• Explain their reasoning for the answers given on
  the form

35
Results of Cognitive Testing

• Overall respondents understood the form, provided
  the information we wanted and had easy access to
  the information.
• Made changes to instructions and several columns
  of information based on results
• Changed education/experience categories
• Wording of seasonal/temporary question
• Changed compensation question

36
Survey Instrument Modifications
States who have modified the JVS Workgroup survey
instrument

• Kansas
• Louisiana
• Minnesota
• Washington

Their surveys are available through JVS Helpdesk
JVS.mail_at_state.mn.us
37
Minnesotas Web Survey
Rachel Hillman Find more on this in About JVS,
Data Collection Alternative response methods.
38
Web Survey
Components of Website

• Fill out the Survey on-line
• E-mail file containing job vacancy data
• Instructions
• Questions
• Uses and Benefits
• Data

39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
Web Survey Response Rate

• 3 percent of the firms who responded filled out
  the survey on the Internet

49
Minnesota Alternative Response Methods
Rachel Hillman Find more on this in About JVS,
Data Collection Alternative response methods.
50
MN Alternative Response Methods

• We offer the following response options
• Mail
• Fax
• Phone
• Internet
• Website
• E-mail

51
Alternative Response Methods Response Rates

• Phone 8 percent

• Website 9 percent

• Fax 9 percent

• Other 5 percent

52
Report Writing Analysis
Workforce Supply Demand Measures
53
Job Vacancy Rate
Rachel Hillman Find more on this in About JVS,
Workforce Supply and Demand Indicators.
54
Job Vacancy Rate
Job Vacancies
Total Employment
55
Job Vacancy Rate Continued

• The job vacancy rate is the number of jobs open
  for hire as a percent of total employment.
• Job vacancy rate of 2.5 percent or fewer than
  three jobs for every 100 filled jobs in Minnesota

56
Job Vacancy Rate Continued II

• Job vacancy rates by major occupational group
  (2-digit SOC)
• Job vacancy rates by occupation (6-digit SOC)
• Job vacancy rates by firm size
• Job vacancy rates by industry

Estimates Delivery System (EDS)
ES-202
57
Estimates Delivery System (EDS)
Annie Tietema Find more on this in About JVS,
Workforce Supply and Demand Indicators
Occupational employment estimates.
58
Estimates Delivery System 2000
As developed by the North Carolina Employment
Security Commission, in conjunction with the OES
Policy Council Technical Committee
59
Overview

• EDS 2000 is a system for producing occupational
  wage and employment estimates and publications
  from the results of the OES survey
• Employment and wage statistics produced by the
  BLS National Office can be imported into EDS for
  inclusion in publications
• EDS allows states the flexibility to define and
  generate estimates for any sub-state geography
  based on county-level building blocks (townships
  in New England states)

60
Overview Continued

• EDS exports estimates in a variety of formats,
  including one format suitable for Internet
  publications and two database table formats
• EDS will follow the OES bi-annual data
  collection, which began with the November 2002
  panel.
• While EDS has been SIC-based, the new version
  due out in fall will operate on the NAICS
  industry coding convention.

61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
Where do I get access to EDS?

• Contact your states OES (Occupational Employment
  Statistics) division for information about your
  states usage of Estimate Delivery System 2000.

80
Hiring Demand Index
Rachel Hillman Find more on this in About JVS,
Workforce Supply and Demand Indicators.
81
Workforce Supply and Demand Revisited

• Job vacancy rates do not provide conclusive proof
  that a workforce shortage exists

82
What about Turnover?

• Turnover the rate at which workers cycle in and
  out of jobs distorts interpretation of job
  vacancy rates

83
Hiring Demand Index

• Compare job vacancy rates while controlling for
  turnover
• Assumptions
• 1. Fields with greatest vacancy rates often have
  proportionate hiring activity -- positions get
  filled!
• 2. Number of job vacancies meaningful when
  compared with number of positions filled in an
  average month

84
Hiring Demand Index Continued
Job Vacancy Rate Turnover Rate
For the Occupational Group
Job Vacancy Rate Turnover Rate
For All Occupations
85
(No Transcript)
86
MN Help Wanted!

• healthcare practitioners and technical
  occupations (including registered nurses and
  licensed practical nurses)
• community and social service occupations
  (including social workers and counselors)
• personal care and service occupations (including
  childcare workers, hairstylists, and personal and
  home care aides)
• healthcare support occupations (including
  nursing aides, orderlies and attendants)

87
Hiring Demand Index III

• Similar to U.S. BLSs fill rate from 1991 pilot
  study
• Index allows identification of occupation groups
  where worker shortages are more serious

88
Turnover Rates

• CPS Job Tenure Supplement microdata
• Nationwide sample with data elements for
  occupation and tenure (n 50,000)
• Turnover rates were calculated
• Turnover P P2 P3
• where P is the share of employment with lt1 year
  tenure

89
Turnover Rates Continued

• Assumption Jobs can turn over multiple times, so
  turnover will be greater than number of new hires
  that persist

90
Alternate measures of turnover?

• Assume that distribution of tenure in a field
  will remain consistent over time.
• Question How many workers at every level of
  tenure must leave to maintain the distribution?
• A matrix algebra calculation of turnover but the
  resulting estimates arent very different

91
Other Ideas for Analysis

• Comparison to Unemployment
• Job vacancies vs. number of unemployed
• Job vacancy rate vs. insured unemployment rate
• Others?

92
4Q 2000 4Q 2001 4Q 2002
Job Vacancies Unemployed Workers
93
Beveridge Curve

• Supply Insured unemployment is a proxy indicator
  of workforce availability
• Many unemployed lots of competition for jobs
• Demand Job vacancies as a share of all jobs
• Few job vacancies lots of competition for jobs

94
Beveridge Curve in the Twin Cities, Fourth
Quarter 2002
95
Marketing
Statewide Marketing Substate Regional Marketing
96
Statewide Marketing
Rachel Hillman Find more on this in About JVS,
Marketing.
97
What Information Do Job Vacancy Statistics
Provide?

• Identify occupations in demand
• Determine education and experience needed for job
  vacancies by region
• Describe job vacancies in terms of
• Wages
• Benefits
• Part-time/full-time
• Temporary/seasonal or permanent

98
Steps for Marketing Your JVS

• Five steps to building your JVS customer base
• Identify the potential audiences in your state
  and around the nation.
• 2. Create user-friendly paper and web
  publications oriented toward these audiences and
  work to make sure they receive them.

99
Who Uses Job Vacancy Survey Data
Policy Makers
Media
Job Seekers
Job Vacancy Survey Data
Workforce Investment Boards
Job Counselors
Education and Training Planners
Industry
Human Resource Professionals
And Others!
100
Possible Publications

• Executive Summary/Report
• Regional
• Brochure
• Regional
• Industry
• Web Publication
• Others?

101
Statewide Users

• Policy Makers
• General findings
• Educational and Training Planners
• Program possibilities
• Training needs
• Human Resource Professionals
• General Findings
• Job Seekers
• General Findings on ISEEK/Career publications

102
Statewide Users Continued

• Job Counselors
• General findings
• Specialized requests
• Workforce Investment Boards
• General findings
• Specialized requests
• Industry
• General findings
• Specialized requests
• Media
• General Findings

Other Users?
103
Steps for Marketing Your JVS Continued

• 3. Incorporate findings from the job vacancy
  survey into the range of information disseminated
  through your LMI Office (publications, articles,
  interviews, presentations, training and website).
• 4. Press release new data and findings to the
  media and provide interviews to help the media
  interpret and write about this new information.
• 5. Collect and use customer feedback to
  continuously improve publications and web
  presence.

104
How MN Use Job Vacancy Survey Data
Executive Summary
Technical Report
Presentations
Job Vacancy Survey Data
Regional Reports
Specialized Requests
Review Supplements and Regional Spotlights
Media Requests
Trends Articles
105
JVS Release

• Prepare Website Executive Summary/Report
• Give the radio/newspaper media a little preview
• Press Release
• Prepare paper Executive Summary/Report
• Prepare regional Brochures

106
Reader Questionnaire

• A variety of users
• Information meet needs presented in a clear and
  understandable manner
• A lot of very specific suggestions

Partially Yes
Good Bad
Please see pages 27-28 of 4Q 2002 Minnesota Job
Vacancy Survey for a copy of a Reader
Questionnaire!
107
MN Regional Marketing
Rachel Hillman Find more on this in About JVS,
Marketing Regional brochures/reports.
108
MN Regional Users
Regional Analysts

• WIB Boards
• Job Service Employer Committees (JSEC)
• City/County Boards
• Dislocated Workers
• State healthcare groups
• Educational Administrators/Planners

Research Statistics Office
109
Not Users?

• Chamber of Commerce
• And Others?

110
Data Capture System
Curt Theis Rachel Hillman Find more on this in
About JVS, Data Capture System.
111
System Requirements

• Windows 98 operating system or higher (Windows
  2000)
• Pentium 133 MHz or higher processor
• 8 MB of RAM in addition to operating system
  requirements
• 170 MB of available hard disk space
• CD drive

112
Installation

• Insert CD
• Start setup.exe program
• Follow steps
• Lets give it a try!

113
(No Transcript)
114
(No Transcript)
115
Switchboard

• Four sections of Switchboard
• Data Entry Forms
• All data entry and SOC coding are done here.
• Error Reports
• A series or queries indicating where possible
  errors occur in data entry.
• Tracking Reports
• Daily accounts of surveys received by various
  grouping characteristics.
• Management
• Add or delete users, create mailing and contact
  lists, and link to or import data tables may be
  done here.

116
(No Transcript)
117
Data Entry Forms

• Four main Data Entry Forms
• Survey Tracking
• Daily record of surveys received and the method
  of response.
• Part A
• Part A of the survey form (contact information)
  is entered here.
• Part B
• Part B of the survey form (job vacancies) are
  entered here.
• SOC Code
• Job titles are SOC coded in this form

118
(No Transcript)
119
Survey Tracking Form

• Enter Survey ID
• Firm mailing address will be displayed at the
  bottom of the form. Double check survey for
  consistency.
• Enter User Name
• Response Status
• Method of response
• Date Received
• Round Received
• What mailing was this survey?

Dont forget to correct addresses.
120
(No Transcript)
121
Part A Form

• Go to Survey ID
• Firm mailing address and strata will be displayed
  on the right hand side. Only surveys that have
  already been tracked may be entered.
• Respondent, title, phone, employment
• All information on Part A of the survey form is
  entered here.
• No Vacancies
• Comments

122
(No Transcript)
123
Part B Form

• Survey ID
• The Press to Enter Survey ID button will
  automatically enter the last survey ID entered on
  the Part A form. Firm mailing address and strata
  will be displayed on the right hand side.
• This field is required.
• Full- or Part-time
• All information on Part A of the survey form is
  entered here.
• This field is required.
• Job Title
• Job title is entered as it appears on the survey
  form.
• Permanent or Seasonal
• Number of Job Vacancies
• This field is required.

124
Part B Form Continued

• Length
• The length of time the job vacancies has been
  open is entered in days.
• Education
• Education requirement selected from a drop down
  menu.
• 8. Experience
• Experience requirement selected from a drop down
  menu.
• Wage/Wage Period
• Benefits
• User defined benefits may be entered using the
  Update Benefit Label button.

125
(No Transcript)
126
SOC Code Form

• Coded SOC Title
• Reported Job Title
• The job title comes from the Part B form.
• Number Open/Wage/Education/Experience
• This information comes from the Part B form.

127
SOC Code Form Continued

• Also included
• Firm mailing address and strata information
• SOC code, title and definition
• Measure of education for that SOC code
• National and State wages for that SOC code

128
(No Transcript)
129
SOC Coding

• Drop down SOC Code List of 30,000 plus job titles
  and SOC codes comes from
• Dictionary of Occupational Titles (DOT)
• Official SOC titles
• Other sources
• Census titles up to 10,000 more titles coming!

130
(No Transcript)
131
Error Reports

• Benefit Error 1
• At least one benefit and no benefits is
  checked.
• Benefit Error 2
• At least one benefit and no response is
  checked.
• No Vacancies, Yet Part B
• No vacancies box is checked, but there are jobs
  recorded in the Part B form for that survey ID
  number.
• Part-time with Salary
• The job vacancies is recorded as part-time and
  has a salary or large wage.

132
Error Reports Continued

• Received, Yet No Part A
• The survey has been tracked, but no Part A
  exists.
• This is the most common error.
• Uncoded Openings
• The job vacancy has not been SOC coded.
• Unusable Part B
• One of the three required fields (full- or
  part-time, job title, number of openings) is
  missing.
• Vacancies, Yet No Part B
• No vacancies has not been checked and no Part B
  records exist for this survey ID number.
• Zero Openings
• A zero has been entered into the number of
  openings field.

133
(No Transcript)
134
Tracking Reports
A report of how many surveys have been received
by

• Area
• Date
• Size
• Round
• Industry
• Ownership
• Vacancy Response

135
(No Transcript)
136
Management

• Add/Delete Users
• Generate Contact List
• Generate Mailing List
• Setup Data Tables
• Link Data Tables
• Import Optional Table
• Generate Results Mailing List

137
(No Transcript)
138
(No Transcript)
139
(No Transcript)
140
(No Transcript)
141
(No Transcript)
142
(No Transcript)
143
(No Transcript)
144
JVS Sample Allocation and Selection

• Christina Chiu
• Bureau of Labor Statistics
• Statistical Methods Staff

Find more on this in About JVS, Sampling
Methodology.
145
Sampling

• Sampling Frame
• Scope
• Stratification
• Neyman Allocation
• Allocation
• Population variance
• Sampling variance
• Relative standard error
• Software
• Validation of popsamp.txt

146
Sampling Frame

• Your ES-202 File
• one quarter of data
• one record per establishment
• 21F EQUI format (see 3.1 of users guide)
• record length of 725 or 801

147
Limiting Your Scope

• Definition of Scope
• The target population or universe that you want
  to make an inference or draw conclusions.
• If you do not limit your scope, your universe is
  your ES-202 file.
• If you do limit your scope, your universe is a
  subset of your ES-202 file.

148
Limiting Your Scope

• Purpose of Scope
• To include or exclude establishments in the
  sample based on certain criteria
• Example You may want your sample to only include
  private establishments, but you want to exclude
  temp agencies. Your scope would include private
  establishments. Temp agencies would be out of
  scope.

149
Limiting Your Scope

• Limiting your scope options
• What if I do not limit my scope?
• All establishments on the ES-202 file will be
  considered.
• You may define your scope by OES Scope or CES
  Scope - see 1.1.6 of users guide
• In addition to defining your scope by OES or
  CES, you may further define your scope by your
  own constraints.
• You may define your scope by your own
  constraints.

150
Limiting Your Scope

• Scope constraints
• Employment size of establishment
• Industry
• Ownership
• Geography

151
Limiting Your Scope

• Scope constraints by Employment

Lower Bound Upper Bound Scope
Not Selected Not Selected All establishments regardless of employment size can be included in the sample.
5 Not Selected Only establishments with at least 5 employees can be included in the sample.
Not Selected 250 Only establishments with 250 or less employees can be included in the sample.
5 250 Only establishments with at least 5 employees and 250 or less employees can be included in the sample.
152
Limiting Your Scope

• Scope Constraints by Industry
• 2-digit SIC
• 3-digit SIC
• 4-digit SIC
• 2-digit NAICS
• 3-digit NAICS
• 4-digit NAICS
• 5-digit NAICS
• 6-digit NAICS
• Combined sector NAICS

153
Limiting Your Scope

• Scope Constraints by Ownership
• Private sector
• Federal government
• State government
• Local and municipal

154
Limiting Your Scope

• Scope Constraints by Geography
• County
• Town
• User-defined area
• you must supply your own area file
• see 1.1.3 of users guide

155
Defining Your Strata

• Definition of Strata
• The universe or population is divided into
  nonoverlapping groups or strata.
• Purpose of Strata
• To reduce the error that comes from sampling a
  population
• Can produce more reliable estimates with
  homogeneity

156
Defining Your Strata

• Strata constraints
• Employment
• Industry
• Ownership
• Geography
• Stratifying Dos
• Do stratify by employment size and industry.
• Do stratify by area if you are using an area
  file.
• Stratifying Donts
• Dont stratify by industry alone.
• Dont stratify by employment size alone.
• Dont overstratify - consequence is empty cells
• Example Avoid stratifying by many size classes
• Example Avoid stratifying by very specific
  industries

157
Defining Your Strata

• Strata constraints by employment
• 0-19, 20-249, 250
• 0-4, 5-49, 50, 249, 250
• 0-9, 10-49, 50-249, 250,999, 1000-5999, 5000
• 0-4, 5-9, 10-19, 20-49, 50-99, 100-249, 250-499,
  500-999, 1000
• 0-9, 10-19, 20-49, 50-99, 100-249, 250-499,
  500-999, 1000
• 0-9, 10-19, 20-49, 50-99, 100-249, 250
• 0-4, 5-9, 10-19, 20-49, 50-99, 100-249, 250
• If you want to sample establishments with at
  least one employee, limit your scope to have a
  lower bound of employment of 1.

158
Defining Your Strata

• Strata constraints by employment
• Example If you plan on sampling 2000
  establishments, and your strata is only
  constrained by 4-size employment, then your 2000
  units will be distributed among 4 strata.

Size
Strata 1 1
Strata 2 2
Strata 3 3
Strata 4 4
159
Defining Your Strata
Size Industry
Strata 1 1 Ag
Strata 2 2 Ag
Strata 3 3 Ag
Strata 4 4 Ag
Strata 5 1 Mining
Strata 6 2 Mining
Strata 7 3 Mining
Strata 8 4 Mining
Strata 9 1 Construction

• Strata constraints by industry
• Example If you plan on sampling 2000
  establishments, and your strata is only
  constrained by employment (4-size) and 24
  industries (2-digit NAICS sectors), your 2000
  units will be distributed among 96 strata.

and so onup to Strata 96
160
Defining Your Strata

• Strata constraints by ownership
• Strata constraints by geography
• Recommended if you are using an area file

161
Allocation

• How are units allocated across strata?
• Neyman allocation
• A type of stratified random sampling
• Puts more units in strata that have large Nh or
  where Sh is more variable.

nh is the sample size per stratum. n is the
total sample size. Nh is the population within a
stratum. Sh is the population standard deviation
per stratum.
162
Allocation
Neyman allocation example

• Say you want to take a sample across 5
  strata, and you want the total sample size to be
  n60.

Stratum Nh Sh nh Wh
A 44 3.02
B 168 3.77
C 56 5.29
D 16 7.26
E 27 4.58
N 311
How do I distribute the sample of n60 across the
five strata?
163
Allocation
Neyman allocation example
164
Allocation

• Neyman allocation example

165
Allocation
Neyman allocation example Results
Stratum Nh Sh nh Wh Nh/nh
A 44 3.02 6 7.333
B 168 3.77 29 5.793
C 56 5.29 14 4.000
D 16 7.26 5 3.200
E 27 4.58 6 4.500
166
Population Variance or Sh

• Definition
• The extent to which the population units or Nh
  differ from each other.
• We do not have the true population variance or
  Sh, so we estimate Sh by using the estimated
  population variance or sh.
• In most cases, we do not have the estimated
  population variance of job vacancies because we
  do not have any estimates of job vacancies.

167
Population Variance

• What does the software use for the estimated
  population variance of job vacancies?
• User defined population variance file
• The software assumes that the estimate of job
  vacancies is 3 of the employment. The software
  calculates the estimated variance of job
  vacancies based on this assumption.

168
Population Variance

• Minimizing sampling error options
• By variance in employment
• When you select this option, you are using
  variance in employment as a proxy for the
  estimated population variance of job vacancies.
• By mean in employment
• When you select this option, you are using the
  mean in employment as a proxy for the estimated
  population variance of job vacancies.
• The variance in employment and mean in employment
  options provide very similar results.
• By user provided file
• Select this option if you provided your own
  population variance file. See 1.1.3 of users
  guide.

169
Sampling Variance

• Sampling variance
• Definition The extent to which the population
  units or Nh differ from each other.
• Different from population variance
• Minimum of 2 units per strata required to
  calculate sampling variance
• Recommend a minimum of 3 units per strata to
  account for non-response.

170
Sampling Variance

• Sampling Variance formula
• Sampling error is equal to the square root of the
  sampling variance
• A component of the relative standard error

Nh-nh is the finite population correction factor.
When Nhnh, the estimated sampling variance is
zero because all units in the population are
included in the sample.
171
Relative Standard Error

• What is relative standard error (rse)?
• RSE is your overall sampling error divided by
  your estimate of job vacancies.
• The lower the sampling error, the more reliable
  your estimate.
• Example Lets say that 5,000 job vacancies have
  been estimated, and the relative standard error
  is 5. We can say, with 95 confidence, that the
  true number of job vacancies lie in the
  interval (4500, 5500).
• C.I. estimate 2(sampling error)
• 5000 2(5000.05)

172
Relative Standard Error

• Relative standard error formula

173
Allocation

• Two methods of sample selection
• Sample size
• This option is more common since you usually know
  how much you can afford to sample.
• Software will not increase sample size due to
  response rate.
• Example If you can afford to collect 3000 units
  and you expect your response rate to be 50, then
  select a sample size of 6000.
• Target relative standard error
• If this option is chosen, a target relative
  standard error of 15 or less is recommended.
• Can result in high sample size

174
Allocation

• Target relative standard error by strata
• By employment size strata
• By ownership strata
• By industry strata
• By geography strata
• This is different from setting an overall target
  relative standard error.
• This option is not recommended because we are
  setting the desired precision for each strata.

175
Allocation

• Certainty units
• By Employment
• If you select this option, all establishments in
  your largest size class will be sampled.
• Example If you select this option and you
  selected 4-size classes, then all establishments
  with 250 or more employees will be sampled.
• By Ownership
• Example If you select this option and choose
  state, then all state government establishments
  will be sampled.

176
Allocation

• Response Rate
• Enter your expected response rate.
• The software assumes the same response rate
  across all strata.
• If you selected the sample size option, response
  rate does not greatly affect your allocation.
• You must calculate your sample size outside of
  the system.
• Example If you can afford to collect 3000 units
  and you expect your response rate to be 50, then
  select a sample size of 6000.
• If you selected the target rse option, a low
  response rate will generate a larger sample than
  would a high response rate.

177
Software

• Required software
• JVS Sample Allocation and Selection (JVSSASS)
  software created by BLS
• SAS Version 6 or higher
• Required input files
• ES-202 EQUI file
• Optional input files
• User-provided area file
• User-provided population variance file

178
Software

• Output Files
• popsamp.txt
• verify sample size and scope
• Data Capture System input
• popdata.txt
• Data Capture System input
• jvasel.log
• check for errors
• tblMacroVar.txt
• your picks
• jvasel.lst

179
Validation of popsamp.txt

• Open Excel. View files of all type.
• Popsamp.txt is in the job vacancy folder.
• Use comma delimiter
• View and manipulate values up to three decimal
  places.
• Verify that the overall sample size, or sum of
  Column K, is the sample size that you specified
  in the software.
• Verify that the certainty units you specified
  have been selected.
• Verify that you all your sample units are in the
  scope you defined.

180
Validation of popsamp.txt
Column A own 0 means that you did not stratify by ownership
Column B industry Industry identifier
Column C size Size identifier
Column D geo 0 means that you did not stratify by geography
Column E strata Strata identifier
Column F popctNh Population per strata
Column G semp Sum of employment per strata
Column H memp Mean of employment
Column I vemp Variance in employment
Column J sd_emp Standard deviation in employment
Column K samplenh Sample size per strata
Column L weightNh/nh Sampling weight per strata
181
Job Vacancy SurveyEstimates Production System

• Jeffrey T. Willingham
• U.S. Bureau of Labor Statistics

Find more on this in About JVS, Job Vacancy
Survey Estimates Production System.
182
Contents

• System Requirements
• Graphical User Interface
• Data
• Screening of Data
• Adjustment of Data
• Estimation
• Confidentiality
• Future Versions
• Demonstration

183
System Requirements

• PC with Microsoft Windows or NT
• SAS v.8

184
Graphical User Interface
185
Data
186
Screening of DataBasic Data Edits

• Input detail and estab data sets.
• Variables All.
• Identifies any discrepancies between the reported
  data and their expected characteristics.
• If discrepancies are present, then they will be
  flagged and shown via an outputted report.
• The user will be required to modify any
  discrepancies using the Data Capture System.

187
Screening of DataOutlier Detection

• Input detail and estab data sets.
• Variables Wgt_Diff, Emp_Diff, Min_Wage,
  Max_Wage, and Vacancies.
• Wgt_Diff OriWgt - CurWgt
• Emp_Diff Ori_Emp - Rpt_Emp
• Utilizes the Quartile Method.

188
Adjustment of DataUnit Non-Response

• Input estab data set.
• Variables Status.
• Utilizes the Non-Response Adjustment Factor
• Viable units are defined as those whose Status
  code ranges from 20-29 and 60-99 20-29
  refusals, 60-79 non-response and data
  unusable.
• Useable units are defined as those whose Status
  code ranges from 80-99 80-99 usable
  partial response and usable complete response.
• V_Emp_i the weighted employment of Viable
  records in Size i.
• U_Emp_i the weighted employment of Useable
  records in Size i.
• V_Emp_0 the weighted, aggregate employment of
  Viable records.
• U_Emp_0 the weighted, aggregate employment of
  Useable records.

189
Adjustment of DataItem Non-Response

• Input detail data set.
• Variables How_Long, Perm_TS, Edu_Lvl, Exp_Lvl,
  Min_Wage, Max_Wage, Benefit1-Benefit6, and
  Cert_License.
• Utilizes Nearest-Neighbor Hot-Deck Imputation
• A distance function, specific to the variables,
  is defined to locate the nearest possible
  neighbor.
• Once a donor is found, it is used to impute for
  the missing item.
• For this imputation process, successively larger
  cells are defined until a donor is located, i.e.,
  imputation begins at the very specific and
  continues to the broad.

190
EstimationEstimation Levels
191
EstimationEstimates

• Total Employment
• Total Vacancies
• Total Vacancy Rate
• Population Units
• Reporting Units

• Occupational Employment ¹
• Estimated Occupational Vacancies
• Occupational Vacancy Rate ¹
• Turnover Adjusted Demand ¹
• Number of Sampled Establishments Reporting the
  Occupation ²

¹ These statistics will only be available if
other data sources are used to supply estimates
of occupational employment and turnover. ² That
possess the characteristic at the estimating
level -- not weighted.
192
EstimationEstimates (continued)

• Job Type
• Proportion of Vacancies that are Permanent
• Proportion of Vacancies that are
  Temporary/Seasonal

• Time Open
• Proportion of Vacancies that are Open Less Than
  30 Days
• Proportion of Vacancies that are Between 30 Days
  and 60 Days
• Proportion of Vacancies Open More Than 60 Days
• Proportion of Vacancies Always Open

193
EstimationEstimates (continued)

• Education
• Proportion of Vacancies Requiring No Diploma
• Proportion of Vacancies Requiring HS/GED
• Proportion of Vacancies Requiring Vocational
  Education
• Proportion of Vacancies Requiring a Two-year
  Degree
• Proportion of Vacancies Requiring a Bachelors
  Degree
• Proportion of Vacancies Requiring an Advanced
  Degree

• Experience
• Proportion of Vacancies Requiring No Experience
• Proportion of Vacancies Requiring Work Experience
• Proportion of Vacancies Requiring Related
  Experience
• Proportion of Vacancies Requiring Occupational
  Experience

194
EstimationEstimates (continued)

• Benefits
• Proportion of Vacancies with benefit 1 (health
  insurance)
• Proportion of Vacancies with benefit 2 (paid sick
  leave)
• Proportion of Vacancies with benefit 3 (paid
  vacation)
• Proportion of Vacancies with benefit 4
  (retirement savings plan or pension)
• Proportion of Vacancies with benefit 5 (no
  benefits offered)
• Proportion of Vacancies with benefit 6 (no
  response)

• Hourly Wage
• Mean Wage for Vacancies
• Minimum Wage for Vacancies
• Maximum Wage for Vacancies

195
EstimationSample Formulas
196
Confidentiality

• Similar to the OES Confidentiality guidelines
• Flag with 0 if there are less than 3
  respondents.
• Flag with 0 if the unweighted employment of the
  largest respondent is greater than 50 of the
  weighted employment.
• Flag with 0 if the unweighted employment of the
  two largest respondents is greater than 75 of
  the weighted employment.
• Flag with 1 otherwise.

197
Future Versions

• Atypical Reporter Review and Adjustment
• This will implement weight adjustments to account
  for consolidated and disaggregated reports.
• A consolidated report occurs when more than one
  unit is sampled and the employer provides a
  combined report for all of them.
• A disaggregated report occurs when one unit is
  sampled but the employer provides multiple
  reports that constitute this or a larger unit.
• Variance Estimation
• Variance estimation provides a measure of the
  degree to which an estimate varies about the mean
  of all possible estimates.