General procedures QAQC and uncertainty management

1
General procedures QA/QC and uncertainty
management

• Finnish Russian Joint Seminar on Establishing
  the Institutional Framework for the
  Implementation of the Kyoto Protocol 14
  15.6.2005

2
Outline of the presentation

• Why quality assurance and control?
• What is quality?
• The quality management process
• The inventory process
• QC checks an example related to data quality
• Uncertainty and sensitivity analysis (very)
  little theory and some numerical examples
• How certain are the results of the Finnish
  inventory?

3
Why QA/QC and uncertainty management?

• Decision 20/CP.7 Each Annex I Party shall
  (should)
• establish quality objectives
• prepare an inventory QA/QC plan
• do annual general QC checks
• estimate uncertainties of the inventory
• do annual source category specific QC checks
• But also uncertainty and sensitivity analysis
  are important for inventory credibility
  (alongside documentation) and can be used to
• strengthen the knowledge base
• optimise inventory development

4
Quality of information, inventory quality

• Inventory is a package of information,
  intentionally conceived, designed and produced,
  the result of goal-oriented action
• Inventory quality would then be the relation
  between what is required and the actual output
  that is, quality is conformance to requirements
• Quality objectives are set to meet the
  requirements
• Such quality can - at least in principle - be
  characterised in terms of deviance from the set
  target

5
Quality Management process (Source Leena
Raittinen)
6
Inventory Process(Source Leena Raittinen)
7
An example source-specific QC of fuel
consumption data

• Some 2/3 of fuel data come from the VAHTI data
  base
• VAHTI contains plant-specific fuel consumption
  data reported on mass as well as energy-basis,
  net calorific values (NCVs) can be calculated
  based on this data
• Calculated NCVs are then compared to default
  NCVs, and discrepancies are used as an indicator
  of potential data quality problems
• Potential problem data are then classified based
  on the quantity (in terms of energy content) of
  fuel combusted
• Corrective measures are thus focused on those
  data that have the greatest potential impact on
  inventory accuracy

8
Uncertainty and sensitivity analysis
Source Saltelli, Chan Scott (2000)
9
Uncertainty and sensitivity analysis

• Consider inventory as a system consisting of a
  vector of inputs x x1, x2, , xnx (e.g.
  activity data, emission factor, other variables),
  a vector of outputs y y1, y2, , yny (e.g.
  the level and trend of emissions), and some model
  y(x)
• Uncertainty analysis addresses the question of
  uncertainty in y(x), given the uncertainty in x
• Sensitivity analysis addresses the importance of
  each element of x with respect to the uncertainty
  in y(x)

10
Sampling-based uncertainty and sensitivity
analysis

• Take a sample xk xk1, xk2, , xknx, k 1,
  2, , nsand evaluate the modely(xk) y1(xk),
  y2(xk), , yny(xk), k 1, 2, , ns
• Then the subject of sensitivity analysis are the
  pairsxk,y(xk), k 1, 2, , ns(there are
  many approaches to sensitivity analysis simple
  graphical one is illustrated later, rank
  correlation is used in the inventory)
• Uncertainty analysis consists of calculating
  summary statistics for y(xk) (e.g. mean,
  confidence interval)

11
Uncertainty analysis N2O from nitric acid
production 1/3
N lt- 1000 x lt- matrix(cbind(rnorm(N,477,12),
rlnorm(N,-4.74,0.4)),N,2) y lt-
matrix(c(rep(0,2N)),N,1) for (i in 1N) yi lt-
xi,1xi,2
1. Set sample size to 1000 2. Take independent
random samples and place the results in the
input matrix 3. Create a matrix for holding the
results 4. Evaluate the model and populate the
output matrix
Input matrix, xk (first 10 rows)
Output matrix, y(xk) (first 10 rows)
,1 ,2 1, 476.9727 0.008808214
2, 459.5567 0.011982220 3, 469.4009
0.005585713 4, 493.3419 0.010048255 5,
468.2163 0.010018177 6, 491.0041 0.005537194
7, 464.7267 0.015609256 8, 496.8499
0.006662455 9, 484.1871 0.005037490 10,
485.5212 0.004995754
1 4.201277 2 5.506510 3 2.621939 4
4.957225 5 4.690674 6 2.718785 7
7.254038 8 3.310240 9 2.439088 10
2.425545
Mean 4.5 Gg N2OC.V. 42.5
12
Sensitivity analysis N2O from nitric acid
production
(1) Uncertainty in y(x) is determined by
uncertainty in the emission factor
(2) One could hold activity level fixed (i.e.
treat it certain) without changing uncertainty
in y(x) significantly doing so actually changes
C.V. from 42.5 to 41.3
13
Simple example N2O from nitric acid production
2/3
The curves are virtuallysuperimposed
treatingactivity level certaindoes not change
the overall picture ofuncertainty in the
level of N2O emissions from nitric acid
productionIn other words efforts toreduce
uncertainty should be focused on
reducingemission factor uncertainty
14
Error propagation CO2 emissions from solid fuels

• Same simple model y x1x2, where x1 activity
  2.44105 TJ, and x2 emission factor 9.3210-2
  Gg/TJ
• Uncertainties in x1 and x2 are ?x1 2.4103 TJ
  and ?x1 1.410-3 Gg/TJ
• Uncertainty in y(x) is given by the error
  propagation equation (calculations based on error
  propagation are referred to as Tier 1
  uncertainty analysis in the Good Practice
  Guidance)

15
Error propagation CO2 emissions from solid fuels
An alternative way of reporting the uncertainty
is 22700 400 Gg (best estimate standard
deviation) Good Practice Guidance recommends
(best estimate 2 standard deviations)
16
Key source analysis

• Two methods to prioritise inventory improvement
  efforts
• The methodology is described in the Good Practice
  Guidance
• UNFCCC Secretariat prepares their own analysis,
  and countries should prepare their own these are
  then compared during inventory review
• Tier 1 considers source categorys contribution
  to the overall level and trend of emissions key
  sources are those that contribute 95 of
  emissions (trend of emissions)
• Tier 2 variant of the method incorporates
  uncertainties in this version, key sources
  contribute 90 of overall uncertainty

17
Uncertainty in the Finnish level and trend of
emissions
Level 2003 Mean 95 confidence interval -
excl. LULUCF 86 Mt CO2-eq. 82 92 Mt CO2-eq. -
incl. LULUCF 68 Mt CO2-eq. 58 78 Mt
CO2-eq. Trend 1990 - 2003 Mean 95
confidence interval - excl. LULUCF 22 16
26 - incl. LULUCF 42 25 65
18
Sensitivity of the 2003 emission level (incl.
LULUCF)
19
Sensitivity of the 2003 emissions level (excl.
LULUCF)
20
What is information? (We need this to discuss the
assumed property of this entity, namely quality.)
Finnish emissions of CO2 from solid fuels
22.7
Mt.
in 2003 were equal to
They increased by 30 from previous year,
and were 45 above the 1990 level.
21
Information - a conceptual model (Lillrank 2002)
Data D are the content of information. Meaning M
is a function of D and its context C M
f(D,C). Data turns into information when
somebody possessing thenecessary knowledge f
about how the world works(i) has a reliable
piece of data D, (ii) is informed about the
context C of that data, and (ii) is therefore
capable of attaching meaning to data.Different
people may contribute by adding their own
functionsand contexts to the process of
negotiation of meaning. And ofcourse, the
reliability of the piece of data can be disputed.