The Safety Health of Maintenance Engineering (SHoMe) Tool

1
The Safety Health of Maintenance Engineering
(SHoMe) Tool

• Steve Mason
• Principal Human Factors Consultant
• HSEC Ltd

2
Error Types Action Strategies

• Cannot physically achieve goal
• - improved ergonomics
• Slips Lapses Distractions
• - design interlocks
• Mistakes
• - training, job aids
• Violations
• - safety culture organisational factors

3
Existing Guidelines/Tools

• Maintainability Design Handbook
• Bretby Maintainability Index
• HFRG Guide to reducing violations
• HFRG Guide to reducing maintenance error
• HSEC Human Factors Solutions
• SHoMe

4
Reactive, Proactive, or Both?
INCIDENT REVIEWS QUESTIONNAIRES
Strengths Information available from incident investigations Based on factual evidence Cost of accidents/incidents can be determined Strengths Quick to administer Findings reflect current attitudes Involves staff identifying areas of concern Can promote ownership
Weaknesses Root issues not always well documented Often insufficient incidents for analysis Data relates to the past may not be relevant to the future Weaknesses Findings based on perceptions may be influenced by range of factors unrelated to maintenance No factual information to gauge spend justified on expected improvements
5
SHoMe Basic StructureSafety Health of
Maintenance Engineering

• Evidence of Non-Compliance
• E.g risk taking, errors through tiredness,
  supervisors condoning unapproved actions, etc

• Potential Root Issues affecting Compliance
• Factors known to have caused problems
• Factors that could cause problems

Part(s) of Job Causing Most Difficulties
6
SHoMe Staff Groups
Technical Certifying Staff Technical
Non-Certifying Staff Management Technical
Support Staff (e.g. planners, )
7
Questionnaire Administration
Staff Group Generic Q Job Difficulty Org Q
Tech Cert ?version C ? ?
Tech Non-Cert ?version NC ? ?
Support Staff ?version NT N/A N/A
8
Questionnaire Administration

• Avoid issuing questionnaires collecting them in
  at the end of a shift response by committee!
• Preferably organise small groups, lead with
  introduction my management ( unions if possible)
• Stress anonymity and purpose of the surveys
• Emphasise that feedback will be given acted
  upon (if appropriate)
• Please ensure correct question set is issued

9
Example Generic Questions
Please indicate the extent to which you agree or disagree with the following statements in relation to your work over the last 6 months of so SA A ? D SD
Management and staff communicate well with each other
Managers always let us know of important safety findings
I am always properly briefed by those giving me a job
Before I start a job I am always given the necessary information
I know exactly what I am expected to do and my responsibilities
10
Extract Job Difficulty Questionnaire
A Was this part of your jobs? B No Problems C Some Problems D Major Problems
PLANNING Y/N? ?
PREPARATION Y/N? ?
INSPECTION Y/N? ?
ROUTINE WORK Y/N? ?
11
Extract from Organisational Questionnaire
The ease with which general written procedures can be understood
The amount of jargon and ambiguity contained in procedures
Ambiguity as a result of different layouts of different types of forms you use
Systems for implementing temporary revisions to maintenance manuals
Previously encountered problems not highlighted
The ability of planners to minimise jobs running over other shifts
The amount of work your immediate boss expects you to do in a shift
The way management treat individuals who report mistakes or safety concerns
12
Prioritising Results
Ranked Evidence of Non-Compliance
Ranked Major Root Issues 19
Ranked Human Factors Issues Generic Q
Ranked Issues Organisational Q
13
19 MAJOR H.F. ISSUES
Design Maintenance Interface Provision of Resources Training Fatigue Complacency Planning Communications Commercial Pressures Maintenance Procedures Roles Responsibility Management Attitudes Safety Commitment of Staff Job Pressure Working Conditions Fair-Blame Culture Management of Change Supervisor Effectiveness Supervisor Attitudes Competence
14
Results Presentation
Level 1 Non-Compliance Generic Q Summary of Major Root Issues only
Level 2 Non-Compliance Generic Q Expanded Major Root Issues Generic Q
Level 3 Expanded Non-Compliance Generic Org Q Expanded Major Root Issues Generic Org Q
MEDA Results presented against relevant MEDA structure
15
HSEC SHoMe ToolVersion 1.0September 2003

• This tool was developed by HSEC Ltd, on behalf of
  the UK CAA
• The tool may be freely copied, distributed and
  used for flight safety purposes, but not resold
  for commercial purposes
• The SHoMe tool must be used in conjunction with
  the user manual
• The software is not supported

16
Scoring Criteria
GENERIC QUESTIONNAIRE Good lt 35 Worthy of
Attention gt 55 Concern gt 65
ORGANISATIONAL QUESTIONNAIRE Probably
Acceptable lt 15 Concern gt 30
17
EVIDENCE OF NON-COMPLIANCE Mean Score
Completion of job despite the non-availability of appropriate tools and equipment 71
Regular non-compliance by the workforce 61
Aircraft released with work not done due to parts shortages 50
Direct pressure to deviate from procedures 43
Reports of colleagues making errors due to tiredness with last month 41
Direct pressure from the supervisor to deviate from procedures 36
Pride on getting aircraft back on time, even if this needs some non-compliance 34
Supervisors taking risks in releasing aircraft when maintenance incomplete 26
Supervisors condoning unapproved actions to get an aircraft away 22

18
POTENTIAL ROOT ISSUES Score
Design Maintenance Interface 57
Planning 51
Fatigue 51
Provision of Resources 50
Commercial Pressures 49
Working Conditions 49
Training 47
Roles Responsibility 41

19
POSITIVE FEATURES Scores
Supervisor Attitudes 29
Competence 30
Supervisor Effectiveness 31
No-Blame / Fair-Blame Culture 34
Management of Change 36
Complacency 38
Safety Commitment of the Engineers / Staff 39
Procedures Accuracy, Relevance Practicality 39
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Job Difficulty
24
(No Transcript)
25
Using SHoMe data
Show Differences between Cert, Non-Cert, Support
Staff Big Picture are there any major
concerns?
26
Typical Questions
Are there any significant differences between
those who have worked in the Industry for gt 20
years and those new to the Industry?
Are the problems with contract staff
significantly different from those with permanent
staff?
Differences within one company with staff at two
sites.
Differences between those working long hours.
27
SUMMARY
Human Factors Issues are important need to be
managed
SHoMe does not rely on incidents/near misses
SHoMe requires investment in time by all staff
Hopefully we agree it complements existing tools
Freely available option for Industry
benchmarking