F' Baret, J' Nightingale, S' Garrigues, J' Nickeson

1
LAND PRODUCT VALIDATION update

• F. Baret, J. Nightingale, S. Garrigues, J.
  Nickeson
• Missoula, 17 June 2009

2
Outline

• CEOS LPV context and structure
• Guidelines for best practices
• The way forward

3
CEOS/WGCV/LPV

• CEOS (committee on Earth Observation Satellite)
  group of space agencies aiming at harmonizing
  their activities at the international level.
• Composed of several working groups (WG) including
  WG on Calibration and Validation (WGCV).
• WGCV is made of several subgroups
• - IVOS (Infrared and Visible Optical Sensors)
• - Microwave
• - SAR
• - Terrain Mapping
• - Atmospheric Chemistry
• - Land Product Validation (LPV)

4
Mission Statement Goals

• To foster quantitative validation of higher level
  global land products derived from remote sensing
  data
• To relay results to users
• To develop and promote international standards
  and protocols for field sampling, scaling, error
  budgeting, data exchange
• To provide feed-back to international structures
  (GEO/GEOSS, GCOS, GTOS, IGBP ) for
• product definition, accuracy and quality
  assurance
• Requirements for future missions

5
Role of LPV within ECVs

• Essential Climate Variables (ECVs) are recognized
  to play a key role
• within international scientific structures (GCOS,
  GTOS)
• international convention for verification
  (UNFCCC) to complement/compare with national
  (official) figures
• Strong requirements on the evaluation of product
  uncertainties need coordinated and consensus
  validation efforts mandatory for all the ECVs

6
Implementation tools

• CEOS has no proper funding mechanisms
• Actions based on best efforts by space agencies
• CEOS provides recommendations to space agencies
• Possible direct actions on satellite data access
• Direct actions within the community
• Synergizing existing projects
• Initiating new projects
• Mostly a bottom-up approach

7
Products targeted

• Land cover with GOFC-GOLD
• Fire (mainly burnt area) with GOFC-GOLD
• Biophysical variables (LAI, fAPAR)
• Albedo, BRDF, surface reflectance
• Land Surface Temperature and emissivity (with
  IVOS)
• Soil moisture
• Biomass? In discussion with GOFC-GOLD
• Snow?

8
LPV proposed new structure

• sub-groups by product family with leads from the
  community if possible from different continents
• Albedo G. Schaepman C. Schaaf
• Land-Cover M. Herold M. Friedl
• Fire K. Tansey L. Boschetti
• Vegetation R. Fernandes, S. Plummer, J.
  Nightingale
• Land Surface Temperature emissivity S. Hook
  J. Sobrino
• Soil W. Wagner T. Jackson

9
Role of sub-group leads

• Coordinate the validation activities at the
  global level
• Guideline for best practices
• Lead the writing of a guideline document for best
  practices on validation
• Ensures community consensus and
  publication/distribution
• Ensures that the document is updated when new
  data / methods are available
• Validation activities promote their
  development/implementation
• Data sharing
• Data compilation
• Implementation of validation exercise
• Publication/distribution of results
• Convey information to and from the community
• Plans/Status/Results of the validation towards
  CEOS/international organizations/community

10
LPV Web site http//lpvs.gsfc.nasa.gov
LPV Wiki http//lpvs.pbworks.com/ LPV
Listservs
Communication tools
11
Outline

• CEOS LPV context and structure
• Guidelines for best practices
• The way forward

12
Best practices guidelines document

• The best practices guidelines should be
• Based on current knowledge, tools, data and
  methods
• Tested and easily repeatable
• The Best Practices guidelines document should
• Define the best practices including data and
  methods to conduct validation of a
  satellite-derived land product.
• Be a living document that is updated as tools,
  data, methods are improved through scientific
  endeavour
• Process for Endorsement by CEOS/ community
• Peer review process

13
Proposed common structure
14
Validation and intercomparison
validation is the process of quantitatively
defining the system response to known, controlled
signal inputs.
Validation refers to assessing the uncertainty
of higher level, satellite sensor derived
products by analytical comparison to reference
data, which is presumed to represent the target
value. Intercomparison of data products
provides an initial indication of gross
differences and possibly insights into the
reasons for the differences. (Justice et
al. 2000)
Validation and intercomparison are mandatory and
complementary
Validation allows absolute quantification of
uncertainties but often limited by the number
and quality of available reference
data Intercomparison provides a more exhaustive
evaluation of consistencies/differences Required
by users required when combining several
products
15
Products definition and uncertainties

• Products definition ECVs not always very clearly
  defined (GCOS/GTOS documents).
• Need more process model related defintions
• Feedback to GCOS/GTOS
• Required uncertainties attached to ECVs not
  defined in a traceable way
• Need more process related uncertainties
  evaluation
• Threshold / Optimal / Target ?
• Feedback to GCOS/GTOS

16
Data sets for validation sites

• Distribution of sites must be representative of
  surface types, state/conditions
• systematic (FRA 2010)
• stratified

Existing data sets Albedo 19 BSRN sites
Need to capitalize the information
17
Data sets for validation measurements
Reference measurements - interpretation by
experts based on HSR images (land-cover, fire)
- quantitative measurements (albedo,
LAI-fAPAR, LSTE , moisture)
Need for high spatial resolution images
(Landsat/SPOT)
18
Global validation
Metrics used Land cover accuracy
/user-producer Confusion matrix LAI-fAPAR
RMSE, weighed RMSE, biases
Users need more information on the structure of
uncertainties
19
Satellite data for intercomparison
- Definition of variables LCCS
- Spatial sampling exhaustive (Land
cover) systematic stratified (LAI, fAPAR,
Albedo )
- Spatial support area need to get same
projection / resolution - Temporal support
period need to get synchronous / same resolution
Degradation oforiginal characteristics
20
Global intercomparison
Land cover
21
Global intercomparison LAI

• Temporal continuity
• Temporal consistency

22

• Smoothness of temporal evolution
• Statistical distributions

23

• Scatterplots

24
Current status
Level 4 operational validation
Land cover
Level 3
Biophysical
Moisture
Level 2
Fire
Albedo
LST E
Level 1
25
Operational Validation land cover
26
Operational Validation Biophysical
27
Concluding remarks

• Very strong ECV context
• Need funding mechanisms for sustainable
  validation activities validation costs!!
• Importance of reference measurements
• data sharing
• improved cooperation with existing/developing
  networks
• Availability of high spatial resolution images
• A lot of new products and not all at the
  ultimate validation level . very challenging
• Mandatory for product improvement / combination