How to Interpret Performance in PTEQA Schemes

1
How to Interpret Performance in PT/EQA Schemes

• Hong Kong Accreditation Service
• Hong Kong
• 11 September, 2009
• Dan Tholen, M.S.

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
PT/EQA Training

• Accreditation Requirements
• Responsibilities of Participants
• Selection of PT/EQA
• Running PT
• Interpreting reports
• Nonconformance process
• Communications with PT/EQA provider

9
Summary responsibilities

• PT provider
• Laboratory
• Accreditation Body

10
PT Provider Responsibilities

• Assure competence and quality
• Comply with accrediting requirements
• Cooperate with stakeholders on program design
• Establish open communication policy
• Document all procedures and complaints
• Maintain confidentiality

11
Accrediting Body Responsibilities(ISO/IEC 17043)

• Assist design of appropriate PT
• Select suitable PT schemes
• Establish policies for participation in PT
• Establish policies for use of PT data
• Maintain oversight of approved PT schemes using
  experienced personnel
• Train assessors on proper use of PT.

12
Laboratory Responsibilities

• Choose competent PT/EQA provider
• Follow instructions from the coordinator and the
  accrediting body
• Use routine methods, avoid collusion
• Act on unsatisfactory results
• Use data for self-improvement
• Retain results and document actions
• Participate in PT Scheme improvement.

13
Requirements for Participation

• ILAC/APLAC
• Current
• Proposed
• HKAS
• Other

14
Requirements ILAC P9

• Current (2005)
• One activity prior to gaining accreditation
• One activity relating to each major sub
  discipline within four years
• Proposed (Rev P9, V6)
• evidence of satisfactory participation prior to
  gaining accreditation
• further and ongoing activity that is fit for
  purpose and appropriate to the technical
  discipline(s) covered by the scope of
  accreditation.

15
Requirements ILAC P9

• Proposed Revision includes policies from
  accrediting body that consider
• Regulatory requirements
• Needs within a sector of activity
• Past performance on PT
• Other sources of information
• Use of reference materials
• Other Interlaboratory Comparisons

16
APLAC PT 006 Benchmarks

• Frequencies for different areas of testing and
  calibration
• Medical 10/year
• Chemical (environmental and mineral) 2/yr
• Biological 2/yr
• Forensics, chemical, veterinary 1/yr
• Specific Calibration fields 1 per 2 years

17
APLAC PT 006 Benchmarks

• The accreditation bodys PT policy should focus
  on more than just meeting the requirements as
  stated in ILAC-P9. The accreditation body should
  ensure participation when PT is available,
  appropriate and at a frequency that reflects best
  practices and/or best local norms. When
  determining the appropriate frequency of PT, the
  accreditation body should take into account the
  test range, method capabilities and regulatory
  limits,where available.

18
HKAS Frequency Requirements

• To be discussed

19
Varieties of Proficiency Testing (from ISO/IEC
FDIS 17043)

• Sequential measurement
• Simultaneous (bulk) testing
• Single time programs
• Continuous programs
• Measurable (quantitative) properties
• Nominal and ordinal properties
• Data transformation or interpretation
• Inspection and sampling

20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Selection of PT Scheme

• Approved by Accrediting Body and/or Regulatory
  Agency
• Accredited by recognized ILAC Member
• Appropriate analytes or measurements, ranges,
  frequencies for laboratory needs
• Affordable
• Appropriate assigned values and performance
  evaluation criteria
• Good customer relations

24
Selection of PT Scheme

• NOTE Some proficiency testing schemes can include
  tests which are not an exact match for the tests
  performed by the participant (for example, the
  use of a different national standard for the same
  determination) but it can still be technically
  justified to participate in the proficiency
  testing scheme if the treatment of the data
  allows for consideration of any significant
  differences in test methodology or other factors.

25
Running PT

• Follow PT providers instructions
• Use routine methods, unless.
• Dont cheat (ask other participants, refer PT
  item, test or measure in special ways)
• Use all operators and all equipment
• Be careful in reporting results
• Report on time

26
Interpreting Reports

• Make sure lab results are correct
• Look at warning and action signals
• Monitor acceptable results also for bias
• Review results from other participants
• Review results from other methods
• Review technical comments
• Review results from previous rounds

27
ISO 17043 Annex C

• C.4.1 Participants should draw their own
  conclusions about their performance from an
  evaluation of the organization and design of the
  proficiency testing scheme. Reviews should
  consider the relation between the proficiency
  testing scheme and the needs of the participant's
  customers.

28
ISO 17043 Annex C

• C.4.2 Participants should maintain their own
  records of performance in proficiency testing,
  including the outcomes of investigations of any
  unsatisfactory results and any subsequent
  corrective or preventive actions.

29
Preventive Action

• Better than non-conformance in PT
• Better for laboratory customer results
• Monitor PT results evaluated as acceptable

30
Investigating non-conformance

• Appropriate for severity of non-conformance
• Time since last check (number of customer
  results)
• Severity of potential errors
• Consider correction (remedial action)
• Need to notify customers?
• Need to stop processes?

31
Investigating non-conformance

• Consider Procedures
• Operator error
• Equipment
• Consider PT item and evaluation limits
• Matrix
• Range of measurand
• Evaluation criteria
• Consider PT provider operations

32
Root cause analysis

• Rarely personnel problem
• Training
• Documented procedures
• Undue Pressure
• Equipment

33
Corrective Action

• Appropriate for problem
• Likely to prevent recurrence
• Change to Quality Manual
• Test or Measurement Procedures
• Training
• Change to Management
• Supervision
• Inadequate resources

34
Monitor corrective actions

• Corrections might not be correct!
• Corrections might not be complete
• Follow-up monitoring is essential
• Next PT is not sufficient monitor alone
• Make additional corrections if needed
• Include root cause on why the initial correction
  was not sufficient
• Include in Internal Audits
• Include in Management Review

35
Communications with Participants

• PT provider has obligation to communicate
  information in advance
• Scheme purpose, design, content
• Reporting
• Dates of shipments and reports
• Requirements for participation, including cost
• Participants need to provide feedback
• Complaints
• Questions
• Suggestions

36
Confidentiality

• Obtain PT provider policy on confidentiality
• Strictly confidential?
• Shared with accrediting body?
• Shared with regulatory agency?
• Shared with other participants?

37
Questions?

• For me or for HKAS
• ISO or other standards
• Participation requirements
• Reviewing reports
• Investigations of unacceptable evaluations
• Other?

38
The End

• Thank you!