SUBPART O TESTING PUBLIC WORKSHOP David Dykes Joe Levine presentation

About This Presentation

Transcript and Presenter's Notes

Title: SUBPART O TESTING PUBLIC WORKSHOP David Dykes Joe Levine

1
SUBPART O TESTING PUBLIC WORKSHOPDavid
DykesJoe Levine

February 19, 2009
New Orleans

2
WORKSHOP AGENDA

Subpart O Testing Regulations
Subpart O Testing History
Subpart O Pilot Testing Process
Subpart O Pilot Test Results
Testing Beyond Pilot
Discussion, Q A

3
WORKSHOP GOAL

To explain
What Subpart O testing is all about
Eliminate unwarranted concerns
How MMS industry can use testing to help raise
the level of OCS safety

4
Can Testing Benefit Your Company?

An extra set of eyes evaluating your personnel
Additional assessment of employees between
regularly scheduled training
If an area of concern is uncovered by testing you
have the opportunity to address it

5
What Subpart O Testing Is Not

A quantitative measure of an employees
Competency or knowledge of job
Ability to perform their job
Passing a test does not mean an employee knows
their job
Failing a test does not mean an employee can not
do their job

6
What Subpart O Testing Is

Another data point for MMS industry to use to
assess the effectiveness of a companys Subpart O
training program and overall safety management
system (SEMS, SEMP.)
An opportunity to gain additional feedback on how
well you are getting critical job related
information across to your most important asset
your people

7
Subpart O Overriding Principles

Testing, as with other aspects of Subpart O is
aimed primarily at the lessee
Testing will impact not only lessees, but
contractors as well
It is the lessees responsibility to ensure their
contractors can properly perform their assigned
duties

8
Subpart O Testing Regulations
9
30 CFR 250.1507 (c)

c) Employee or contract personnel testing
MMS or its authorized representative may conduct
testing at either onshore or offshore locations
for the purpose of evaluating an individual's
knowledge and skills in performing well control
and production safety duties.

10
30 CFR 250.1507 (d)

(d) Hands-on production safety, simulator, or
live well testing
MMS or its authorized representative may conduct
tests at either onshore or offshore locations.
Tests will be designed to evaluate the competency
of your employees or contract personnel in
performing their assigned well control and
production safety duties. You are responsible for
the costs associated with this testing, excluding
salary and travel costs for MMS personnel.

11
30 CFR 250.1508

What must I do when MMS administers written or
oral tests?
MMS or its authorized representative may test
your employees or contract personnel at your
worksite or at an onshore location. You and your
contractors must
(a) Allow MMS or its authorized representative to
administer written or oral tests and
(b) Identify personnel by current position, years
of experience in present position, years of total
oil field experience, and employer's name (e.g.,
operator, contractor, or sub-contractor company
name).

12
30 CFR 250.1509

What must I do when MMS administers or
requires hands-on, simulator, or other types of
testing?
If MMS or its authorized representative
conducts, or requires you or your contractor to
conduct hands-on, simulator, or other types of
testing, you must
(a) Allow MMS or its authorized
representative to administer or witness the
testing
(b) Identify personnel by current position,
years of experience in present position, years of
total oil field experience, and employer's name
(e.g., operator, contractor, or sub-contractor
company name) and
(c) Pay for all costs associated with the
testing, excluding salary and travel costs for
MMS personnel

13
Subpart O Testing History
14
At This Point, Testing Should Not be a Surprise
to You or Your Company

Testing has been included in our regulations
since the year 2000
We have voiced our intention to test at a variety
of conferences meetings in recent years
Before initiating the 2008 Subpart O pilot the
agency began exploring testing in 2007

15
January, 2007 IADC Well Control Work Group

MMS introduced idea to IADC that agency was
interested in implementing Subpart O testing
MMS presented hands-on well control testing
options to IADC for discussion purposes
Live well tests
School based testing
simulators
computers
30 CFR 250.462 (d) Well Control Drill

16
January, 2007 IADC Well Control Work Group

Why Was MMS Interested in Testing?
Increase in Subpart O INCs
40 INCs as of 1/2007
G 846, G851, G862, G842
Loss of experienced personnel
Up to this point MMS had conducted 6 written
tests, several oral tests, no hands-on tests
MMS requested IADC/Industry input on testing

17
NTL No. 2008-N03 - Well Control Production
Safety Training

Effective March 31, 2008
Clarified terms used in Subpart O rule
Internal audit, production safety, periodic,
contractor
Discussed lessee and contractor responsibilities,
program evaluations, employee verifications
Announced MMS intent to conduct hands-on testing
in 2008

18
April, 2008 - Baker Energy

MMS met with Baker to review their production
training program
Baker provided MMS detailed information on
program highlights to aid us in developing a
successful Subpart O test program
Training components
Assessment criteria
Test questions
Grading issues
Program statistics

19
June, 2008 IADC Well Control Work Group

MMS updated IADC continued to press for input
into testing program
Training INCs continue to increase
71 INCs as of 12/2007
Up to this point MMS had conducted 10 written
tests, 36 oral tests, no hands-on tests
IADC agreed to form internal group to provide MMS
testing feedback

20
August, 2008 IADC Well Control Work Group

Since June, 2008 IADC/Chevron provided MMS a
series of hands-on well control scenarios
Operational based
Position specific
Discussed IADC/Chevron suggestions for hands-on
well control scenarios received by agency
MMS provided work group overview of how pilot
Subpart O testing program could work moving
forward

21
October, 2008Diamond Offshore Drilling

Diamond volunteers rigs for offshore written /or
hands-on well control testing
Diamond/Devon
Ocean Endeavour semisubmersible
Pre MMS pilot
Received valuable real world input and comments
on specific test questions, test process, grading
policies

22
Testing

Since publication of Subpart O in 2000 MMS has
had a variety of tools at our disposal to monitor
industry compliance
Employee interviews
Audits
Tests
Majority of MMS efforts to date have focused on
interviews and audits

23

24
(No Transcript)
25
Testing

After evaluating how interviews and audits
work/dont work MMS began to explore testing
Provides MMS another tool in our audit tool box
to evaluate employee competency
Provides MMS and company more of a real world
evaluation of individual skills as related to
their job

26
SUBPART OPILOT TESTING PROCESS
27
Testing Pilot Program

Started 11/1/08
No enforcement (INC)
Goal
Issue as many tests as possible to gain a better
understanding of test administration
Receive lessee contractor input
Areas of concern
Grading
70
pass/fail
Consistency
test administration
Level of difficulty

28
Testing Pilot Program

Open book
All material acceptable
Charts, guides, books, tables.
No time limit
Testing location
Onshore (office, school), offshore
Passing grade
Written 70 and above
Hands-on pass/fail, all steps deemed critical

29
Written Well Control Pilot Tests

Three tests A, B, C
Each test includes three levels based upon rig
position
Level L1
floorhand, derrickman
Level L1L2
driller, assistant driller
Level L1L2L3
toolpusher, company man

30
Written Well Control Pilot Tests - Grading

Level L1 (floorhand, derrickman)
5 questions
Passing 70 on this section only
Level L1L2 (driller, assistant driller)
11 questions
5 questions from L1 6 new questions
Passing 70 on 11 questions
Level L1L2L3 (toolpusher, company man)
20 questions
5 questions from L1 6 questions from L1L2
9 new questions
Passing 70 on 20 questions (entire test)

31
Written Well Control Pilot Tests - Questions

Level L1 questions
Basic well control concepts, equipment practices
Level L1L2 questions
More advanced well control theory, equipment
concepts
Level L1L2L3 questions
Deeper knowledge of well control techniques,
operational systems, calculations
Only level to contain calculations

32
Hands-On Pilot Well Control Tests

Five scenarios
Each scenario includes a series of Yes/No skill
based questions
5 to 8 questions per scenario
Employee/team needs to show competency by
demonstrating or answering all questions
correctly
Each question in a scenario deemed to be critical
Scenarios focus on either an individual or a team

33
Written Production Pilot Tests

Five tests A, B, C, D, E
Each test includes three levels based upon
position on facility
Level L1
C or D operator
Level L1L2
A or B operator
Level L1L2L3
Lead operator

34
Written Production Pilot Tests - Grading

Level L1 (C or D operator)
35 questions
Passing 70 on this section only
Level L1L2 (A or B operator)
44 questions total
35 questions from L1 9 new questions
Passing 70 on 44 questions
Level L1L2L3 (lead operator)
50 questions total
35 questions from L1 9 questions from L1L2
6 new questions
Passing 70 on 50 questions (entire test)

35
Written Production Pilot Tests - Questions

Level L1 questions
Basic wellhead equipment design, safety device
testing and reporting requirements
Level L1L2 questions
More advanced safety analysis concepts, system
design theory and practices
Level L1L2L3 questions
Deeper knowledge of analysis, design,
installation and testing of production surface
safety systems and of government codes, rules and
regulations

36
Hands-On Production Pilot Tests

Ten scenarios
Each scenario includes a series of Yes/No skill
based questions
10 to 14 questions per scenario
Employee/team needs to show competency by
demonstrating or answering all questions in a
correct manner
Each question in a scenario deemed to be critical
Scenarios focus on either an individual or a team

37
PILOT TESTING PROGRAM RESULTS
38
Type of Tests Conducted During Pilot

Majority of written tests were offshore
announced
Small percentage of written tests were
unannounced
18 well control written tests conducted at school
compared against schools test
All hands-on pilot tests were conducted offshore

39
Testing Pilot Program

Lessee contractors either volunteered to
participate or were picked at random
Testing was conducted during normal MMS
inspection activities
Tests were mostly graded onsite
Employee comments were solicited
Often times an individual was given multiple
tests for feedback and comparison purposes
Test data was entered into an MMS data base

40
Companies Tested During Pilot

Devon/Diamond
ExxonMobil/HP
ExxonMobil/Sam Jones
Energy XX1/Hercules
BP/Rowan
Helis/Rowan
Hunt/Ensco
Badger/Pride

Mariner/Noble
El Paso/Wood Group
Mariner/Prosper
Chevron/Parker
Venoco
PXP
Shell
Stone
Apache

41
Pilot Test Breakdown by MMS Region

Total tests conducted during pilot
Gulf of Mexico - 129
Pacific - 7
Alaska - 0

42
Subpart O Pilot Tests Conducted

Written Tests
Well control 86
37 L1
23 L1L2
26 L1L2L3
Production 31
4 L1
7 L1L2
20 L1L2L3
Hands-On Tests -19
Well control - 4
Production - 15

43
Pilot Written Well Control Tests
44
PILOT WRITTEN WELL CONTROL TEST AGRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
85 91 100
95 73 80
85 64 60
90 91 60
93 73 80
65 64 80
100 60
60
Failures (less than 70 noted in red) 60
40
80
Tests Given 7 6 11
Average Grade 87.6 76 69.1
45
PILOT WRITTEN WELL CONTROL TEST B GRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
95 64 100
75 73 100
80 64 80
90 91 100
85 91 60
90 91 100
80 100 80
85 73 100
Failures (less than 70 noted in red) 90 64 80
70 80
60
60
100
100
Tests Given 10 9 14
Average Grade 84 79 85.7
46
PILOT WRITTEN WELL CONTROL TEST C GRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
65 64 40
90 64 80
60 73 80
45 64 20
95 64 40
75 64 40
Failures (less than 70 noted in red) 90 37 40
70 55 40
80 40
40
0
0
Tests Given 9 8 12
Average Grade 74 61 42
47
PILOT WRITTEN WELL CONTROL TEST A
48
PILOT WRITTEN WELL CONTROL TEST B
49
PILOT WRITTEN WELL CONTROL TEST C
50
PILOT WELL CONTROL WRITTEN TEST vs. SCHOOL TESTS

18 MMS well control written tests conducted at
school compared against schools test
Tested 6 people 2 toolpushers, 1 driller, 1
derrickman, 2 floorhands
School WellCAP test averages
Introductory 78
Fundamental 87
Supervisory 89
MMS test averages
L1 70
L1L2 81.7
L1L2L3 83.3
Limited data shows consistently higher school
scores than MMS scores for comparable tests

51
WELL CONTROL WRITTEN PILOT TESTS WHAT DID WE
LEARN ?

Well Control Test C Appears to be the Most
Difficult
Test C has more calculations than Test A or Test
B
Test C has lowest average test scores in each
category
Most Test Failures are for the L1 level
Floorhand, derrickman
As expected, L1 personnel are least experienced
6 yrs. avg. oilfield experience
Majority of L1 personnel appear to be trained
in accordance with IADC WellCAP Introductory
level
some MMS L1 questions were outside scope of
IADC WellCAP Introductory level
BOP testing and MMS regulations

52
WELL CONTROL WRITTEN PILOT TESTS AREAS OF CONCERN

L1 (floorhand, derrickman)
Causes of kick
not keeping hole full
insufficient mud weight
lost circulation
First step to control kick
shut well in ASAP
Location of wrench for manual valves (MMS
regulations)
ready access
Annular test pressure or test frequency
14 day
70 RWP or per MMS (not in WEllCAP, MMS
regulations)

53
WELL CONTROL WRITTEN PILOT TESTS AREAS OF CONCERN

L1L2 (driller, assistant driller)
Hydrostatic pressure concept
gradient or density of fluid x height of fluid
column
Ways to circulate out kick
drillers, wait/weight, volumetric
Ram test pressure
RWP or per MMS (MMS regulations)
With well shut in what happens to BHP as gas
bubble rises
Quantities of drilling fluid and drilling fluid
materials on rig to ensure well control (MMS
regulations)
Safe drilling margin

54
WELL CONTROL WRITTEN PILOT TESTS AREAS OF CONCERN

L1L2L3 (toolpusher, company man)
Calculations (only test w/ calculations)
P1V1P2V2
sacks Barite to increase MW
circulating pressures
equivalent mud weight
MMS regulations
diverter vent line OD (bottom founded, floater)
directional requirements (inclination, azimuth)

55
Written Well Control Tests Lessons Learned

Evaluate increasing number of questions in L1
section of test from 5 to 10 for easier/better
grading
Evaluate IADC WellCAP course curriculum in
developing questions for tests and use as
appropriate
Introductory (floorhand, derrickman)
Fundamental (derrickman, assistant driller,
driller)
Supervisory (tool pusher, company man)
Evaluate question specific comments received by
MMS industry personnel

56
Pilot Written Production Tests
57
PILOT WRITTEN PRODUCTION TEST AGRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
74 75 80
88 82 92
86
92
Tests Given 4 2 2
Average Grade 85 78.5 86
58
PILOT WRITTEN PRODUCTION TEST B GRADE
DISTRIBUITION
L1,L2,L3 L1,L2 L1
92 77
90
90
76
76
86
92
Tests Given 7 1 0
Average Grade 86 77 NA
59
PILOT WRITTEN PRODUCTION TEST C GRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
80 91 86
92
94
Tests Given 3 1 1
Average Grade 88.7 91 86
60
PILOT WRITTEN PRODUCTION TEST D GRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
86 84 83
96
88
Tests Given 3 1 1
Average Grade 90 84 83
61
PILOT WRITTEN PRODUCTION TEST E GRADE
DISTRIBUTION
L1,L2,L3 L1,L2 L1
80 91
92 89
94
Tests Given 3 2 0
Average Grade 88.7 90 NA
62
PILOT WRITTEN PRODUCTION TEST A
63
PILOT WRITTEN PRODUCTION TEST B
64
PILOT WRITTEN PRODUCTION TEST C
65
PILOT WRITTEN PRODUCTION TEST D
66
PILOT WRITTEN PRODUCTION TEST E
67
Written Pilot Production Tests What Did We Learn?

MMS would have liked to have conducted more
production written tests however, based upon the
data
Production tests A E appear to be of equivalent
difficulty
No failures out of 31 tests
74 lowest recorded grade (Test A-L1L2L3)

68
Written Pilot Production Tests Areas of Concern

L1 (C, D operators)
Wellhead component identification
casing valves (intermediate, production)
casing heads (surface, intermediate, production)
Equipment testing interval
TSH-compressor (six months)
TSH-fired component (annual)

69
Written Pilot Production Tests Areas of Concern

L1L2 (A, B operators)
Wellhead component identification
casing valves (intermediate, production)
casing heads (surface, intermediate, production
Separator pressure valve settings
Equipment testing interval
BSL (annual)
tubing plug (six months)
What does LACT stand for?

70
Written Pilot Production Tests Areas of Concern

L1L2L3 (lead operators)
DOI pipeline out of service requirements
of LEL gas detector should alarm at
Separator pressure valve settings
What does LACT stand for?
Equipment testing interval
tubing plug (annual)

71
Pilot Hands On Test Results

Well Control 4 (P, P, P, P)
Production 15
(P, F-13/14)
(P)
(P)
(P, P, F-9/10)
(P, P)
(P, F-9/10)
(P, F-7/10)
(P, P)

72
Pilot Hands On Tests What Did We Learn

Grading
Test taker could be competent even though they
did not answer all questions correctly or perform
task properly
Subjectivity of MMS
Should we continue with P/F hands on grading
system or evaluate use of a grading system
similar to written testing?

73
Test Taker Comments

Well Control Written
Good program
Tests put together well, each level fair, good
learning tool, good refresher, fair test
Tests should lead to increased safety
L1 test adequate, tests L1L2 and L1L2L3 should be
more detailed
Straightforward, easy to understand
Good for basic knowledge, good mix of easy/hard
questions
Good for rig tests
Will help improve skills between school based
training
L1 test easy, L1L2 test confusing
Do not test on crew change days (2 times)

74
Test Taker Comments

Well Control Written
Some questions I have seen before others I have
not
Strong test, good test (7 times)
Questions did not clearly show choices, I knew
the answer but it was not in the choices
Need MMS regulation book for test
Hard test
Testing is good, allows you to see who
understands what
Dont test on first day back after days off, keep
testing
Not enough time, felt rushed
Use one big test
Good questions (2 times)
Too many repeat questions
Tests shouldnt be announced
Tests identify items I am weak in

75
Test Taker Comments

Production Written
Some questions geared to management personnel not
operational personnel
Test A question 7 confusing
Test A question 2 misleading
Test B question 19 could not find in CFR (2
times)
We need a list of publications showing where
questions come from
Good general test on equipment and regulations
Good test
Test B is best, more real world then other test
versions

76
Test Taker Comments

Production Hands On
Scenario went over all procedures to troubleshoot
this event (2 times)
Evaluation went over all steps to troubleshoot
equipment and went through all components
Scenario went over all components operations
associated with performing task
Well Control
None

77
What's Next?Testing Beyond Pilot
78
Major Points of Interest

Post-pilot testing program will go into effect on
4/1/09
Post-pilot program will include enforcement
Lessons learned from pilot program will be
integrated into program whenever feasible

79
Who Will MMS Test ?

Possible test candidates
Operator
Contractor
Companies picked _at_ random
Companies experiencing
problems with Subpart O audit
loss of well control
lower than average OSI (poor performers)
When training is identified as a cause in an
accident

80
Test Triggers

Testing is one tool in the audit toolbox
Triggers for testing include
Failure of the operator to verify employee
knowledge and skill
Failure of the operator to verify contractor
employee knowledge and skill
Failure of the operator to evaluate contractors
training program

81
Post-Pilot Testing Procedure - Basics

Tests will be announced or unannounced
Announced
phone call or letter from MMS
Tests will be conducted at a variety of
locations onshore (school, office), offshore
Offshore
MMS to discuss w/ field management to determine
if it is safe appropriate to conduct a test at
that time (written or hands-on)
Open book
All material acceptable
Computers OK with MMS monitoring
No emails, no IMs
No time limit for test

82
Post-Pilot Testing Procedure - Basics

No set number of tests to be conducted by MMS
during a year
If we conduct test _at_ your rig/facility/school
goal is to test 10 of personnel on location
No retests
No help from others
Grading
70 passing for both written and hands-on tests

83
Post-Pilot Testing Procedure - Basics

Well control written tests
4 new tests instead of 3
10 questions in L1 (floorhand, derrickman)
section of test instead of 5
Questions to focus more on IADC WellCAP
Introductory level and less on MMS regulations
Production written tests
More or less same questions as pilot tests
however questions will be renumbered for easier
grading
Addition of an H2S supplement operational
specific testing

84
Post-Pilot Testing Procedure

Enforcement
If employee fails written test (lt 70) a G-856
INC may be issued
MMS will consider a hands-on test of employee to
validate written test prior to deciding on INC
Based on test(s) results MMS briefs lessee with
general assessment of employees performance
if MMS feels strongly that INC should be issued
lessee will be informed to expect INC in the mail
from District within 7 business days
no INC issued on-site
grading will not be conducted on-site, but in
District
lessee needs to identify to MMS how they will
address deficient employee before MMS leaves
location (school, rig/facility)

85
G 856 INC Information

INC with explanatory letter to lessee is issued
by District office and will include
Date time of test
Employer
lessee, contractor
Location of test
onshore (school, office)
offshore (rig/facility, lease/block)
Position of employee
Lead operator, driller ..
Score and result
numeric grade
pass/ fail
specific questions wrong will not be released
Overview of concerns

86
Post-Pilot Testing Procedure

G-856 Possible Corrective Actions
Retrain employee
Retest employee w/ company test
Retrain and retest employee
Put employee under direct supervision of another
employee
Transfer employee to another job
Others ??

87
New G-856 PINC

G-856 DOES EACH EMPLOYEE UNDERSTAND AND PERFORM
THE ASSIGNED WELL-CONTROL OR PRODUCTION SAFETY
DUTIES?
Authority 30 CFR 250.1503(a) 30 CFR
250.1507(c)
Enforcement Action W/C 30 CFR 250.1507(d)
INSPECTION PROCEDURE Verify that lessees
training plan provides a process to ensure that
employees understand and can perform their
assigned well-control or production safety
duties. Verify (either through written,
hands-on, or oral testing) that employees
understand and can perform their assigned
well-control or production safety duties.
IF NONCOMPLIANCE EXISTS Issue one warning (W)
INC for an audit of one or more employees if the
training plan fails to provide a process to
ensure that employees understand and can perform
assigned well-control or production safety
duties. Issue one component shut-in (C) INC for
one or more employees that fail to demonstrate
either through written, hands-on, or oral
testing, their ability to perform the assigned
well-control or production duties.
Note The employee is the component.
INSPECTION COUNT/INC COUNT Enter one item
checked/issue on INC per training plan audited.