Best practices in application of sampling under PoA framework: DOEs perspective

1
'Best practices in application of sampling under
PoA framework DOEs perspective'
Workshop on Programme of Activities (PoA) under
the CDM Challenges and Road Ahead 07th - 8th
May 2011, Bonn
UNFCCC Secretariat
2
Contents

• Data sampling Approach (general)
• Data sampling Verification - Best practice
  example
• CDM Methodologies (general)
• Sampling plan- issues to be addressed

3
Data sampling Approach

• Sampling determines the reliability of the
  parameter value estimate expressed in terms of
  probability of a parameter falling within a
  specified interval around a parameters true
  value
• Tools/guidance (sampling guidance EB50, annex 30)
• Methodological guidance, otherwise 90/10
  confidence/precision as the criteria for
  reliability of sampling
• Over estimation of the mean value should be
  avoided, unbiased and reliable results ensured
• Simple random sample
• most commonly used approach
• each observation chosen randomly and entirely by
  chance
• presents unbiased estimate of true population,
• suitable for relatively homogeneous population
• Systematic sampling, Stratified random sample,
  Cluster sampling, Multi stage sampling
• Explanation of why the sample size is reliable
  and accurately reflects the population
• Statistically sound sampling
• 95/5 precision (95 confidence interval and 5
  margin of error)
• 90/10 precision (90 confidence interval and 10
  margin of error)
• 90/30 precision (for eg. leakage)
• Unique IDs- double counting avoided, verification
  status can be determined at all times

4
Real Projects- Verification Approach

• Gold Standard methodology for Improved
  Cook-stoves and Kitchen Regimes V.01 ( CDM Meth
  AMS IIG)
• 6627 stoves were installed during the first year
  of the crediting period
• Scalable to 1 Million plus installations
• Verification challenges (MP1)
• Access, Travel time
• Limited annual ERs (15000) Costs
• On site - Man days
• Population characteristics, record and
  installation database of every stove constructed
  incl. family name, ID number, location, and date
  of construction for all the households that
  receive a stove
• 1,788 monitoring surveys that include leakage,
  sustainability and qualitative fuel use data

Province Municipalities Approximate population
Santa Barbara 28 342,054
Copan 23 288,766
Lempira 28 288,766
Intibucá 17 179,862
5
Verification approach

• Statistical justification, Reasonable Assurance
• Key monitored parameters
• Number of stoves installed (350 samples)
• Continued use of stoves over time Drop off rate
  (alteration from original configuration)
• PDD states that this shall be done through a
  survey of first 50 beneficiaries who had stoves
  installed within the first 12 months of the start
  crediting period
• MR1 stated that a total of 1787 households were
  surveyed in 2009 and 2010 of these only 28
  stoves were found to be out of use - drop off
  rate of 1.57
• Does the sample selected represent the entire
  population (spatial and temporal) - first 50
  beneficiaries only?
• Verification of 30 households in one locality
  (1/2 man day) random sample indicated 3 drop off
  (indicating 10 drop off)
• 3 more localities selected, 30 h/h each
• Sample size increased to 120 households across
  three randomly selected locations (samples)
• Drop off rate found constant (7.5) across the
  selected samples
• (90 confidence level, 5-10 error margin)

6
Verification Approach

• Statistical Justification, Reasonable Assurance
• Emission reduction achieved per stove per year
• Registered PDD - 2.23 mtCO2e/year per stove
  (qualification attached- paired sample test)
• MR1 - 2.73 mtCO2e/year per stove
• Paired sample Kitchen Test for annual emission
  savings per stove (2010 Paired Fuelwood
  Consumption Study)
• Measure daily fuel consumption over a 4-day
  period in 50 households stoves, random selection,
  monitored before adoption of the La Justa 2x3
  vis-a vis traditional stove (fogon)
•  Actual sample size taken by PP was larger
  (reaching n55) and wood was weighed over a
  5-day period (not just a 4-day period as
  required), resulting in four 24-hour periods of
  fuelwood consumption data rather than 3
• Confidence level- 90, SD of population- 0.168
• Reliability directly proportional to numerical
  sample size
• Large sample size and paired design, sampling
  approach, assumptions justification of approach
  transparently reported
• Eliminated systematic underestimation
• Original datasets available, Yale university
  staff (Third Party) - on site

7
The sampling plan submitted by project
proponents is reviewed using to assess a range of
issues and questions, such as

• Does the sampling plan present a reasonable
  approach for obtaining unbiased, reliable
  estimates of the variables?
• Is the data collection/measurement method likely
  to provide reliable data given the nature of the
  parameters of interest and project, or is it
  subject to measurement errors?
• Is the population clearly defined and how well
  does the proposed approach to developing the
  sampling frame represent that population? Does
  the frame contain the information necessary to
  implement the sampling approach?
• Is the sampling approach suitable, given the
  nature of the parameters, the data collection
  method, and the information in the sampling
  frame?
• Is the proposed sample size adequate to achieve
  the minimum confidence/precision requirements? Is
  the ex ante estimate of the population variance
  needed for the calculation of the sample size
  adequately justified?
• Are the procedures for the data measurements well
  defined and do they adequately provide for
  minimizing non-sampling errors? Is the quality
  control and assurance strategy adequate? Are
  there mechanisms for avoiding bias in the answer,
  including possible fraud?
• Are the persons conducting the sampling
  activities qualified?
• (Ref. EB50, annexure 30)

8
Methodologies (CDM Portfolio)

• AMS ID Hydro Projects, AMS IIIF (Municipal
  Waste), AMS IIG (Efficient cook stoves), AMS IC
  (eg. Solar thermal), AMS II J (CFL Lighting)
• Sampling approach justified, consistent
  monitoring plan applied across all CPAs,
  technical specifications are similar, operational
  criteria/equipment provider same
• Statistically representative
• Issues temporal and spatial aspects
• AMS ID AMS IF (Hydro, Wind, Biomass,
  Photovoltaic) other methodological combinations
• More complex
• Sampling approach for each category

9

• Employs 59,000 people and operate a network of
  more than 1,000 offices and laboratories around
  the world
• Largest verifier of CERs issued under CDM

Questions?
Siddharth.Yadav_at_sgs.com Contact 44
7712785772
Thank You