Large Data Set Issues in Remote Sensing

1
Large Data Set Issues in Remote Sensing

• Mark Berman
• CSIRO Mathematical and Information Sciences
  (CMIS), Sydney
• June 12, 2009

2
Some Remote Sensors
NIR near infrared SWIR shortwave infrared
TIR thermal infrared pan panchromatic
3
Sizes of Remotely Sensed Data Sets

• An example Mapping environmental change over
  Australia using Landsat
• Area of Australia 7.6 106 km2
• No. of pixels required 7.6 106/(0.0032)2
  1010
• Total no. of observations 7 1010
• Time series analysis over 14 time slices
  (1972 2004) 7 14 1010 2 1012 (2
  terabytes)

4
Remote Sensing Processing and Analysis Issues

• Large data sets require large amounts of
  automation
• Speed is often an important issue (not
  necessarily real time)
• Parallel computing can often help
• Nevertheless speed requirement discourages
  complex models
• All models are wrong and in large data sets it
  shows!
• Continuous nature of remote sensing data (both
  spatial and spectral aspects) means that model
  errors are often obvious
• Incorporating spatial context can help, but
  harder to parallelise
• Storage, transfer issues
• What is delivered to customers raw data, maps,
  statistical summaries?