Link Between RiskInformed, InService Inspection and Inspection Qualification

1
Link Between Risk-Informed, In-Service Inspection
and Inspection Qualification

• IAEA 3rd Qualification Workshop
• Vienna
• October 2008

2
Background

• General aim
• To investigate the possibility of using expert
  judgement toestimate the Probability of
  Detection (POD)of an inspection qualified by the
  ENIQ methodology

3
Background

• Motivation
• Probabilistic risk assessment requires POD as a
  function of defect size the POD curve
• ENIQ qualification provides assurance of
  detecting defects with size greater than a
  specified amount
• Probability of detection is not quantified
• No curve is generated

4
Background

• Strategy
• Investigate
• what information is needed from the POD curve
  how much detail?
• how can the Technical Justification (TJ) required
  by the ENIQ methodology provide information about
  POD

5
Previous work

• The project was a development of proposals in the
  paper
• Framework for the quantitative modelling of the
  European methodology for qualification of
  non-destructive testing
• by Luca Gandossi Kaisa Simola
• (International Journal of Pressure Vessels
  Piping 82 (2005) pp 814-824)

6
Outcome of work on POD curve shape

• Work carried out on the sensitivity of Risk
  Reduction to POD curve shape showed that in many
  cases (not all), the plateau level of the POD
  curve is the most important factor
• (This is subject to the threshold detection size
  being small compared with the size at which
  failure probability starts to rise sharply)

POD
Plateau level
Detection threshold
Defect size
7
Scope of presentation

• Here, I concentrate on the work done on
  guidelines for assessing the plateau level of the
  POD from the ENIQ TJ
• A) General introduction to the Bayesian
  methodology proposed by Gandossi Simola
• B) A worked example of the methodology developed
  during the project
• Note that
• the guidelines result from the experience of two
  pilot studies
• they are preliminary there are some issues
  still to resolve

8

• Bayesian statistical model

9
General statistical model

• For simplicity, we assume that defect detection
  is a binomial statistical process
• A defect can either be detected or not detected
• The probability of detection has an unknown
  value, q
• In the present context, we mean the probability
  of detecting a defect which should be detected
  according to the Input Data specification of the
  ENIQ qualification.

10
General statistical model

• We are assuming the detection threshold (size) is
  low enough that the details of the curve dont
  matter
• We are focusing on the level of the POD plateau
  is it 80, 90, 95 ??

POD
Plateau level
Detection threshold
Defect size
11
General statistical model

• Hence we can take as a simple model of the
  detection process that in n trials the
  distribution of the number, s of detections is

(i.e. the Binomial distribution)
12
Bayesian statistics

• Bayes theorem tells us that
• We say that on the basis of
• the assumption of a prior distribution, p(q)
• the knowledge of the relation between q and s,
  pltsqgt (the so called likelihood function)
• We can find the posterior distribution pltqsgt,
  of q corresponding to an observed value s, this
  being the number of successes in the trial

13
Bayesian statistics

• It turns out that if we assume a prior
  distribution of the form
• then the posterior distribution is of the form
• The prior form is a standard probability density,
  Beta(a, ß)
• the posterior has the same form, it is Beta(a
  s, ß f)
• (where we now use f n s number of failures)

14
Bayesian statistics

• Beta probability densities

Be(21,3)
Be(11,2)
Be(1,1)
15
Bayesian statistics

• A key feature of the Bayesian approach is that
  when there is sufficient evidence (i.e. s, the
  number of successes) then the prior distribution
  assumed does not much affect the resulting
  posterior distribution
• In the present context the posterior distribution
  is the distribution of the parameter we seek
  namely the POD, q
• On the assumption that nothing is know about q
  before considering the inspection, we choose a
  prior distribution, Be(1,1)
• If we then conduct n trials and get s successes
  and f failures we say that a better estimate of
  the distribution of q is Be (1 s, 1 f)

16
Bayesian statistics

• An Illustration
• Initial assumption (i.e. prior distribution)
  is that the probability of detection could have
  any value with equal probability

17
Bayesian statistics

• Suppose we have 10 successes and 1 failure in a
  trial
• Using this data, we calculate the posterior
  distribution of the success probability to be
  Be(1 10, 1 2) Be(11, 2)

18
Bayesian statistics

• Note we are calculating a distribution for the
  parameter we seek, the probability of detection,
  q
• The most likely value is 9 out of 10 0.9 but
  the true value could be more or less

19
Bayesian statistics

• Now suppose we do more 10 more trials and again
  get 1 failure (now we have 2 failures out of 20
  trials).
• We get an updated posterior distribution
  Be(11010,111) Be(21,3)

20
Bayesian statistics

• This greater amount of evidence now gives us a
  sharper distribution
• We have increased confidence that the POD is near
  9 out of 10

21

• Applying the Bayesian model in the ENIQ context

22
Application of Bayesian model to assessing POD

• ENIQ context
• Clearly, we could get more confidence in the POD
  value if we carried out more trials
• In the ENIQ methodology, we usually focus on a
  limited scope
• specific inspection of a specific component
• more trials is an expensive solution
• An alternative, explored in the project, is to
  establish a prior distribution for POD based on
  our prior knowledge of the effectiveness of the
  inspection
• This knowledge is formalised in the technical
  justification

23
Application of Bayesian model to assessing POD

• To take advantage of the Bayesian methodology we
  must
• Think of the TJ as a kind of virtual trial of the
  inspection
• Assign a number of successes to the TJ where it
  provides convincing evidence in favour of defect
  detection
• Assign a number of failures to the TJ where the
  evidence is considered to be weak

24

• A worked example using guidelines developed in
  the project

25
Worked example

• Preliminary remarks
• Following the draft guidelines developed in the
  project, the POD assessment would be carried out
  by the Qualification Body (QB)
• A group of at least 3 experts is proposed so as
  to minimise individual bias
• The group needs
• Some training in the method
• The help of a facilitator with expertise in
  similar processes of expert elicitation
  (including its statistical aspects)

26
Worked example step 1

• Breakdown TJ into its parts
• Identify stand-alone elements
• These are independent aspects of the TJ
• for example
• 1 modelling
• 2 extrapolation from experimental evidence
  (other than the qualification trials)
• 3 physical reasoning
• Identify any limiting factors
• These are factors which can place an upper limit
  on POD
• For example
• 1 Coverage limitations
• 2 Equipment failure
• 3 Human factors

27
Worked example step 2

• Determine cumulative effect of limiting factors
• Suppose that the coverage is limited to 95 of
  the examination volume, - hence the POD cannot
  be greater than 95
• Similarly suppose the evidence suggests detection
  failure rates of
• 1 for equipment
• 2 for human factors
• Then the probability of missing a defect is 1
  (0.95 x 0.99 x 0.98) 0.078
• The limiting POD for the inspection is 92.2

28
Worked example step 3

• Decide the relative weights of the TJ stand-alone
  elements
• Suppose we have the following the scenario
• The modelling covers most aspects of the
  inspection
• A substantial number of experimental trials have
  been carried out for technique development the
  testpieces are similar to the real plant
  extrapolation is required for wall thickness
• For some defects, the modelling is not applicable
  but simple physical reasoning shows the signal
  levels must be high

29
Worked example step 3 (cont.)

• The qualification body could then (e.g.) decide
  that the relative importance of these elements
  was
• Modelling 30
• Experiments 50
• Physical Reasoning 20

30
Worked example step 4

• Scoring of the TJ elements
• The weights give the relative importance of the
  different elements in the TJ
• The score must reflect how well the evidence
  supports the detectability of the defects
• Here the scenario could be (e.g.)
• Modelling is well validated score 100
• Experiments are convincing but the justification
  for the extrapolation has some weakness maybe 1
  defect in 20 could be missed - score 95
• Physical reasoning convincing argument that it
  is conservative score 100

31
Worked example step 5

• Calculate the TJ total weighted score
• Convert the percentages into fractions and we get
  an overall TJ score of 0.975 as shown below

32
Worked example step 6

• Decide the TJ equivalent sample size
• This is most difficult step!
• Effectively, the experts are being asked to judge
  what the TJ is worth in terms of equivalent
  formal qualification trials
• Suppose, for the purpose of the example, that the
  QB believe that the TJ has equivalent value to a
  further number of trials, NTJ 20
• (i.e. a trial involving 20 more defects)

33
Worked example step 7

• Calculation of the distribution of the POD value
  on the basis of the evidence from the TJ
• Recall that we always start by assuming a prior
  distribution Be(aprior, ßprior)
• By choosing aprior 1 and ßprior 1, this
  prior distribution is uniform
• The impact of the TJ is given by the updated
  distribution
• Be(1 s, 1 f)
• Where
• s is the number of successes attributed to the
  TJ
• f is the number of failures attributed to the TJ

34
Worked example step 7 (cont.)

• The number of TJ successes is taken as
• NTJ x wTJ 20 x 0.975 19.5
• The number of TJ failures is taken as
• NTJ x (1 wTJ) 20 x 0.025 0.5
• Hence the distribution of the POD (strictly its
  probability density) is
• Be(1 19.5, 1 0.5) Be(20.5, 1.5)

35
Worked example step 7 (cont.)

• Resulting POD distribution
• Note peak value corresponds to the TJ score NTJ
  x wTJ 0.975

36
Worked example step 8

• Updating with evidence from practical trials
• Suppose we carry out an open trial on 10 defects
  and get 10 successes
• The POD distribution is then updated to
  Be(1 19.5 10, 1 0.5 0) Be(30.5, 1.5)

37
Worked example step 9

• Optional step take account of blind trials
• Suppose there are 15 defects in the blind trial
  and one is missed then the Beta distribution is
  updated once more to
• Be(30.5 14, 1.5 1) Be(44.5, 2.5)

38
Worked example step 10

• Combination of stand alone elements and limiting
  factors
• The POD distribution derived from the stand-alone
  elements has to be scaled to match the limiting
  factors
• Hence the maximum POD is set at 0.922 (step 2).

39
Worked example step 11

• Final step reporting POD
• It is convenient to report the result as a
  cumulative distribution of the POD

40
Worked example step 11 (cont.)

• We can conclude, for example, that there is 90
  confidence that the POD for this case is at least
  0.83

0.83
41
Some conclusions

• Pilot studies carried out suggest that the method
  can be made to work in practice
• Perhaps the key issue to resolve is how to weight
  the TJ against practical trials
• TJ s generally concentrate on worst-case
  defects it is important to avoid an
  excessively pessimistic assessment of POD
  based only on such defects

42
Some conclusions (cont.)

• As a general principle, it may be best to assess
  POD separately for different defect classes it
  is then the job of the structural integrity
  analyst to combine these PODs with the
  probabilities of occurrence of the different
  defect types
• For this and other reasons the TJ should ideally
  be constructed with this process in mind

43
Acknowledgements

• The work described here has been led by my
  colleague Barrie Shepherd
• The technical work has mainly been inspired and
  carried out by Luca Gandossi (JRC) and Kaisa
  Simola (VTT)
• Pilot studies also involved the following
  Qualification Body
• John Whittle (chairman independent consultant)
• Russ Booler (Serco Assurance)
• Håkan Söderstrand (SQC)
• Barry Dikstra (Doosan Babcock)

44
Further information

• The sponsors
• Swedish utilities, TVO, VGB PowerTech Service,
  Iberdrola
• have kindly given permission for the work to be
  made public and the various reports are available
  via the JRC/ENIQ web pages