Non-Response Bias Analyses of the Survey of Workplace Violence Prevention

1
Non-Response Bias Analyses of the Survey of
Workplace Violence Prevention

• Andrew Kato, Kathy Downey, William McCarthy, and
  Samantha Cruz
• U.S. Bureau of Labor Statistics
• The opinions expressed here are those of the
  authors and do not represent official policy of
  the Bureau of Labor Statistics.

2
Survey of Workplace Violence Prevention

• Special 2005 study conducted for NIOSH (National
  Institute for Occupational Safety and Health)
• Workplace Violence Prevention (WVP)
• Prevalence of security features,
• The risks facing employees,
• Employer policies and training, and
• Related topics associated with maintaining a safe
  work environment

3
WVP Sampling

• Sample taken from respondents to 2003 SOII
  (Survey of Occupational Injuries and Illnesses)
• SOII - Private industry, State and local
  government
• public sector units from the fourth Quarter 2003
  Longitudinal Database (LDB) and mining and
  railroad establishments
• Total of 39,998 units
• Randomly selected units proportional to size and
  oversampled within specific industries
• Used respondents so have prior relationship

4
WVP Methodology

• Hardcopy was 12 pages, envelope and insert also
  available to non-respondents in Word via e-mail
• Voluntary
• Protocol
• Initial mailing to SOII respondent (Sept 05),
• Follow-up mailing to non-respondents,
• Address corrections for post office returns, and
• Telephone follow-ups to non-respondents
• Close-out June 06
• Final response rate was 61

5
Purpose of Non-Response Analyses

• OMB requirement
• Examine potential bias due to non-response since
  predicted response rate might be low

6
Predominant Approaches to Conducting Non-Response
(Olson, 2006)

• Comparing characteristics with a benchmark survey
  
• Comparing frame information between respondents
  and non-respondents
• Simulating statistics based on restricted
  protocol (level of effort analyses)
• Mounting experiments to produce varying response
  rates across groups

7
Data Available for Non-Response Analyses

• 2003 SOII frame data
• Size class
• 1 1-10
• 2 11-49
• 3 50-249
• 4 250-999
• 5 1000
• Sector (industry) combined private and public
• 2003 SOII
• Rate (per 10,000 FTE hours) of job transfer or
  restriction
• Rate (per 10,000 FTE hours) of days-away-from-work
  cases

8
Data Analyses

• Compare non-respondents and respondents on 2003
  frame data and survey responses
• Level of effort analyses response propensity
  models

9
Comparing WPV Respondents and Non-Respondents
Size Class, by NR
plt.0001 for overall
10
Comparing WPV Respondents and Non-Respondents
Size Class, by Average Days Away from Work Rate
plt.003 for class 1, plt.0001 for class 3
11
Comparing WPV Respondents and Non-Respondents
Size Class, by Average Job Transfer or
Restriction Rate
plt.0001 for classes 3, 4, and 5
12
Comparing WPV Respondents and Non-Respondents
Selected Sectors, by NR
plt.0001 for selected sectors above
13
Comparing WPV Respondents and Non-Respondents
Selected Sectors, by Average Days Away from Work
Rate
plt.02 transport, plt.004 rest
14
Comparing WPV Respondents and Non-Respondents
Selected Sectors, by Average Job Transfer or
Restriction Rate
varying significant p values
15
Comparing WPV Respondents and Non-Respondents
Conclusions

• Dealing with four projects when discussing
• 2003 SOII sample and analyzed 2003 SOII estimates
  of DAW and JTR (injuries/illness)
• WPV survey existence of violence prevention
  programs
• This NR project
• 2001 one-time Respirator Survey used 1999 SOII
  sample

16
Comparing WPV Respondents and Non-Respondents
Conclusions

• Non-respondents versus respondents
• Higher size class showed curvilinear effect with
  size 3 highest
• Industry had some variation
• possibly those with more public units

17
Comparing WPV Respondents and Non-Respondents
Conclusions
plt.0001
18
Comparing WPV Respondents and Non-Respondents
Conclusions Days Away from Work, Job Transfer

• Higher rates are responding more
• Size
• Days away from work rate highest class 3,
  lowest 1
• Job transfer or restriction rate highest 3, 4,
  5
• Industry
• Days away from work rate wholesale, retail,
  real estate, health care, transportation
• Job transfer or restriction rate same plus
  public admin, admin support, manufacturing

19
Level of Effort Response Propensity Models for
Contact and Cooperation
Predicting Contact Predicting Contact Predicting Cooperation Predicting Cooperation
Coeff SE Coeff SE
Intercept 5.09 0.16 5.59 0.21
Size 1 0.07 0.17 -0.07 0.20
Size 2 -0.04 0.08 -0.04 0.09
Size 3 0.09 0.05 0.14 0.06
Size 4 -0.01 0.05 0.01 0.06
Hours (FTE worked) 0.00 0.00 0.00 0.00
DART case rate 0.01 0.00 0.01 0.00
1st NR mail Nov 05 -0.19 0.21 -0.82 0.26
1st round calling 0.29 0.15 -0.83 0.18
2nd NR mail Apr 06 -5.10 0.17 -3.88 0.13
2nd round calling 12.46 163.1 -5.53 0.15
plt.05 plt.0001
20
Level of Effort Response Propensity Strata for
Contact and Cooperation
Response Propensity Strata Response Propensity Strata Response Propensity Strata Response Propensity Strata Response Propensity Strata
Low Grp 2 Grp 3 Grp 4 High
Contact Contact Contact Contact Contact Contact
Actual rate (n) 51 (5021) 55 (5021) 88 (5021) 99 (5021) 97 (5021)
Est. Non-contacts 51 54 58 99 99
Est. Contacts 51 54 92 99 100
Cooperation Cooperation Cooperation Cooperation Cooperation Cooperation
Actual rate 36 (1410) 55 (2179) 98 (3883) 99 (3921) 100 (3935)
Est. Ref. 32 55 98 100 100
Est. Coop 43 56 99 99 100
21
Conclusions

• Some differences between WPV respondents and
  non-respondents
• Size
• Industry
• SOII estimates
• Respondents to WPV have more programs and more
  SOII incidents
• Not sure what impact to WPV, possibly respondents
  have more programs and more risks
  (over-reporting?)

22
Limitations

• Limitation only as good as phone logs from
  vendor (contact/non-contact)
• Learned over surveys to not have subcontractors
  so can have more detail in phone logs, manage
  mailings/contacts

23
Future Research

• Level of effort analyses tied into data estimates
• How data might change for key WPV estimates at
  different levels of effort (truncation)
• Huge work to re-weight, though
• More WPV analyses 3-digit NAICS, like
  transportation