The Land, Atmosphere Near-real-time Capability for EOS (LANCE)

1
The Land, Atmosphere Near-real-time Capability
for EOS(LANCE)

• Ed Masuoka
• Code 614.5
• GESDIS Combined Branch Meeting
• 2-2-2010

2
What is LANCE?

• Building on existing EOSDIS elements
  (MODAPS/LAADS OMI SIPS, GES DISC), provides data
  from MODIS, OMI, AIRS, MLS, and AMSR-E
  instruments in near real-time (lt 3 hours from
  observation)
• Utilizes PGE Code from Standard Science Products,
  but
• All products generated with a nrt extension to
  distinguish them from standard products
• Requirements for ancillary data inputs have been
  relaxed (e.g., MODIS cloud mask, land surface
  reflectance, aerosols, and clouds)
• High operational availability achieved by
• Extra compute and ftp servers
• Redundant power sources
• Redundant production strings on separate networks
  (in progress)
• Applications of LANCE data include
• Numerical weather climate prediction/forecasting
  
• Monitoring of Natural Hazards
• Disaster Relief

• Agriculture
• Air quality
• Homeland Security

3
LANCE Architecture
LANCE Architecture
Protocols HTTP/FTP
Users
GES DISC
EDOS Level Zero Processing Facility
Data Available -AIRS L1 and L2 -MLS L2
Format RBD
AIRS
GMAO
Format RBD
MLS
NRL
Protocols HTTP/FTP
MODAPS
SPoRT
Data Available -MODIS L1, L2, L2G and selected
L3 -AMSR-E L1, L2 and L3
Format S-PDS
MODIS
UMBC
Format RBD
AMSR-E
USGS
Protocols HTTP/FTP
Others
OMI SIPS
Data Available -OMI L1B and L2
Format RBD
OMI
Observation to availability latency 30-180 min.
Transfer latency bandwidth dependent
LANCE Element Under Development
Operating LANCE Element
RBD Rate Buffered Data S-PDS Session Based
Production Data Set
4
LANCE Product Categories
Instrument Product Categories Average Latency Status
AIRS Radiances, Temperature and Moisture Profiles, Clouds and Trace Gases, Ephemeris/Attitude 75 140 minutes Operating
AMSR-E L1A Raw Data, Soil Moisture, Snow Water Equivalent, Temperature N/A Under Development
MLS Ozone, Temperature, Ephemeris/Attitude Data 75 140 minutes Operating
MODIS Radiances, Cloud/Aerosols, Water Vapor, Fire, Snow Cover, Sea Ice, Land Surface Reflectance, Ephemeris/attitude 90 145 minutes (L1 and L2 products) Operating
OMI Ozone, Clouds, Aerosols, Trace Gases 100 165 minutes Operating
MODIS also generates daily L2G and L3 Climate Modeling Grid for surface reflectance these products have a latency of 27-28 hours MODIS also generates daily L2G and L3 Climate Modeling Grid for surface reflectance these products have a latency of 27-28 hours MODIS also generates daily L2G and L3 Climate Modeling Grid for surface reflectance these products have a latency of 27-28 hours MODIS also generates daily L2G and L3 Climate Modeling Grid for surface reflectance these products have a latency of 27-28 hours
5
LANCE-MODIS Architecture
DOORS
EMSnet Mission Network
Science and Engineering Network
nrt2
PDR server
PDR server modpdr01
Browse Server

• High Operational Availability achieved by
• 1. Redundant power
• 2 sources of conditioned power (UPS 9 and 10)
  for each component
• 2. Extra compute and ftp servers (Dell 2950)
• 15 servers (2 can fail w/o impact to production)
• 2nd ftp server (currently manual fail-over)
• 3. Redundant production strings on separate
  networks

6
LANCE-MODIS Operational Data Flow
6
7
Example Latencies for LANCE-MODIS and Standard
Forward Products
Standard Processing Standard Processing NRT Processing (typical)
Product Category Terra(hrs) Aqua(hrs) Terra/Aqua (hrs)
L1/Cloud Mask 8 25 1.3 - 2.1
L2 Snow 8 25 1.5 - 2.2
L2 Sea Ice 8 25 1.5 - 2.2
L2 Fire 8 25 1.4 - 2.2
L2 Clouds 32 32 1.5 - 2.3
L2 Aerosol 32 32 1.5 - 2.3
L2 LSR 40 41 1.5 - 2.4
8
Comparison of MODIS Standard and NRT Products
Standard Processing
Near-Real-Time Processing

• Land Surface Reflectance over Midwest. Slightly
  more haze is visible in NRT view West of Great
  Lakes.

9
Comparison of MODIS Standard and NRT Products
Standard Processing
Near-Real-Time Processing

• Cloud top temperature over the Midwest. Thin
  clouds over lake Superior show sensitivity to
  GDAS ancillary data

10
Comparison of MODIS Standard and NRT Products
Short Name Science Data Match ( Global) pixel (Global) Omission Error pixel () Commission Error pixel ()
MOD09 LSR-B1 98.94 N/A N/A N/A
MOD09 LSR-B2 99.12 N/A N/A N/A
MOD09 LSR-B3 99.32 N/A N/A N/A
MOD09 LSR-B4 99.16 N/A N/A N/A
MOD10 L2 Snow 99.97 31831094 (2.1) 39751 (0.13) 44255 (0.14)
MOD29 L2 Sea Ice 99.95 14471848 (5.5) 8383 (0.06) 14812 (0.1)
MOD14 L2 Fire 99.97 3207 (0) 2 (0.06) 3 (0.09)

• For LSR the Match is the percentage of NRT data
  with lt1 error margin when compared to the
  operational Collection 5 codes
• For Snow, Sea Ice, and Fire, the Match is the
  exact pixel to pixel match between NRT and
  operational Collection 5 codes
• Omission and Commission errors are computed as a
  percentage of the snow,sea ice, fire in the
  operational Collection 5 products

11
AIRS L1 Product Quality
AIRS Level1-B VIS/NIR Quick Browse Image
Near-Real Time
Standard Product
11
12
AIRS NRT Data Product Quality

• Same PGEs used to produce Standard Data Products
  used in NRT processing
• NRT L1 data nearly identical to L1 Standard
  Product
• Some minor differences in geolocation due to use
  of predicted ephemeris
• Occasional minor differences in granules along
  contact session boundaries (0.1 K)
• Processing option retain data at session
  boundaries to ensure data overlap when generating
  L1B
• To date data latency has superseded minor quality
  improvement
• NRT L2 products compare favorably to the L2
  Standard Data Products.
• NRT system uses surface climatology in place of
  dynamic ancillary data (GFS) as input to science
  data processing
• Surface Air Temperature differences within 1 K
  for areas where sfc pressure is approximated by
  climatology, otherwise differences can be as much
  as 10 K in selected areas
• Total Water Vapor differences within 5 over most
  of the globe as much as 15 -20 where large
  surface pressure differences are observed,
  particularly at high latitudes

13
LANCE-MODIS Data Access and Services

• LANCE-MODIS Web Site
• URL http//LANCE-MODIS.eosdis.nasa.gov
• Provides access to a wide variety of information
  including MODIS ftp site, registration,
  latencies, metrics, PGE versions and production
  rules, data outages, and access to other LANCE
  systems
• Provides access to land and atmospheres browse
  products
• 7 days of browse products are maintained in
  rolling archive
• User Registration
• Registration is required before data may be
  accessed
• Present registration process is temporary pending
  the development of an EOSDIS-wide user
  registration system
• URL https//LANCE-MODIS.eosdis.nasa.gov8080
• User Data Access
• Subscriptions
• Provide data directly as available (push)
• Direct access to public
• Download from directories (pull)
• Hostname nrt1.modaps.eosdis.nasa.gov Directory
  Path allData/1/esdtx/year/dataday
• 7 days of data products are archived in a rolling
  archive

14
LANCE-AIRS/MLS User Access and Data Services

• 5 day rolling archive available
• FTP
• Pull (anonymous, subscription)
• Push (subscription)
• Near Real Time Data Web Site
• http//disc.gsfc.nasa.gov/nrt
• Currently no format conversion or subsetting
  offered for NRT data products
• Spatially qualified subscriptions available
• Web Map Service (WMS) offered for AIRS NRT
  products
• L1 visible radiances
• CO concentration
• SO2 indicator

15
Present Users of the LANCE-MODIS System
MODIS Rapid Response System, NASA/GSFC
Ocean Color Data Processing System, NASA/GSFC
Climate and Radiation Branch, NASA/GSFC
Foreign Agricultural Service
University of Wisconsin
Direct Broadcast Laboratory, NASA/GSFC
Naval Research Laboratory, Monterey
GES DISC, NASA/GSFC
FNMOC, Monterey
USGS, Sioux Falls
University of Maryland, Baltimore
University of Maryland, College Park
Battelle Laboratory
16
Example Application eMODIS Africa Expedited
Product Flow
T12hrs
USGS FEWS NET Processing System (stack, smooth,
create products)
Terra MODIS
T9hrs
EWEM NDVI and Anomaly Maps
EDOS
FEWS NET Decision Support System (USAID FFP)
MODIS L0 Data
T3hrs
MODIS L2 Data
Input Data Deadline Tuesday 1000 a.m.
FEWS Impact Assessment Maps
LANCE
EWEM/FEWS NET Desired/required 24 hour or less
data delivery schedule
November 2009
17
Proposed Modifications to LANCE-MODIS

• Following the LANCE Workshop in December, the
  following changes to LANCE-MODIS are being
  planned/considered
• Addition of a redundant system using different
  power and on a different network
• Adding a Land Surface Reflectance product
• Adding the AMSR-E products
• Adding a variety of sub-setting capabilities
• Adding re-projection and mosaicing capabilities
• Adding a variety of data formats (e.g. GeoTiff)
• Integration of the Rapid Response System with
  LANCE-MODIS
• Generate products derived from Land Surface
  Temperature to support the Flood Mapping program

17
18
Backup
19
NRT Products from LANCE-MODIS
Short-name PGE Description Product Description
MxD01 L1A Raw Radiances and Geolocation MODIS Raw Radiances in Counts 5-Min L1A Swath
MxD03 L1A Raw Radiances and Geolocation MODIS Geolocation Fields 5-Min Swath, 1km
MxD021KM L1B Calibration MODIS Calibrated Radiances 5-Min L1B Swath 1km
MxD02HKM L1B Calibration MODIS Calibrated Radiances 5-Min L1B Swath 500m
MxD02QKM L1B Calibration MODIS Calibrated Radiances 5-Min L1B Swath 250m
MxD07_L2 L2 Cloud Masks/Atmospheric Profiles MODIS Temperature and Water Vapor Profiles 5-Min L2
MxD35_L2 L2 Cloud Masks/Atmospheric Profiles MODIS Cloud Mask and Spectral Test Results 5-Min L2
MxD04_L2 L2 Atmosphere MODIS Aerosol 5-Min L2 Swath 10km
MxD05_L2 L2 Atmosphere MODIS Total Precipitable Water Vapor 5-Min L2 Swath 1km
MxD06_L2 L2 Clouds MODIS Clouds 5-Min L2 Swath 1km and 5km
MxD10_L2 L2 Snow Cover MODIS Snow Cover 5-Min L2 Swath 500m
MxD10L2C L2 Snow Cover MODIS Coarse Snow Cover 5-Min L2 Swath 5km
MxD29 L2 Sea Ice MODIS Sea Ice Extent 5-Min L2 Swath 1km
MxD29L2C L2 Sea Ice MODIS Coarse Sea Ice Extent 5-Min L2 Swath 5km
20
NRT Products from LANCE-MODIS
Short-name PGE Description Product Description
MxD09 Land Surface Reflectance MODIS Surface Reflectance 5-Min L2 Swath 250m, 500m and 1km
MxD09CMA/MxD09CMG L3 Daily Land Surface Reflectance MODIS Surface Reflectance L3 0.05 deg Tile Climate Modeling Grid
MxD09GST/GHK/GQK L2G Land Surface Reflectance MODIS L2G Daily, tiled 1km/500m/250m surface reflectance
MxD09GA/GQ L2G Land Surface Reflectance MODIS Light L2G Daily, tiled 500m and 1km/250m
MxDTBGD L2G Land Surface Reflectance MODIS L2G Daily, tiled daytime thermal bands 1km
MxD14 L2 Thermal Anomalies/Fire MODIS Thermal Anomalies/Fire 5-Min L2 Swath 1km
MxD00F Session-Based L0 PDS file splitter Session-Based L0 PDS file, 5-min Swath
MxD00S Session-Based L0 PDS Original Session-Based L0 file from EDOS
MxD02SSH L1B Calibration 5 km subsampled L1B
MYDGB0 Aqua Attitude and Ephemeris Aqua Attitude and Ephemeris in Session L0 format
21
NRT Datatypes available from GES DISC
AIRABRAD_NRT Aqua/AIRS Level 1B AMSU-A1 and AMSU-A2 Combined Geolocated and Calibrated Brightness Temperatures
AIRIBQAP_NRT Aqua/AIRS Level 1B Infrared (IR) Quality Assurance Subset
AIRIBRAD_NRT Aqua/AIRS Level 1B Infrared (IR) Geolocated and Calibrated Radiances
AIRVBQAP_NRT Aqua/AIRS Level 1B Visible/Near Infrared (VIS/NIR) Quality Assurance Subset
AIRVBRAD_NRT Aqua/AIRS Level 1B Visible/Near Infrared (VIS/NIR) Geolocated and Calibrated Radiances
AIRI2CCF_NRT Aqua/AIRS FINAL AIRS Level 2 Cloud Clear Radiance Product
AIRX2RET_NRT Aqua/AIRS Level 2 Standard Final Retrieval Product
AIRX2SUP_NRT Aqua/AIRS Level 2 Support Product
PM1EPHND_NRT Preprocessed Aqua Platform Definitive Ephemeris Data from FDS in Native format
PM1ATTNR_NRT Preprocessed Aqua Platform Refined Attitude Data in Native format
ML2O3_NRT Aura/MLS L2 Ozone (O3) Mixing Ratio
ML2T_NRT Aura/MLS L2 Temperature
AURATTN_NRT Preprocessed Aura platform attitude product
AUREPHMN_NRT Preprocessed Aura platform ephemeris product
1All datatypes appended with _NRT to denote
difference from production datatypes
22
Product Dependencies on Ancillary Data

• The L2 codes search back for ancillary data a
  maximum of the times indicated on the arrows
• The L2 Snow, Sea Ice, and Fire do not use
  ancillary data products

Cloud Mask
L2 Aerosols
84h
120h(optional)
14d
72h
21d
36h
GFS
REYNSST
SeaIce
NISE
GDAS
TOAST
120h
14d
14d
8d
7d
60h
L2 LSR
L2 Cloud
Ancillary Data
UnchangedFrom Standard
PGE