Multidimensional Data and GIS

1
Multidimensional Data and GIS

• Steve Kopp
• Nawajish Noman
• ESRI

2
Overview

• Introduction to Multidimensional Data
• What is netCDF?
• Some netCDF data sources
• Working with netCDF in ArcGIS

3
Multidimensional Data

• Data cube (3D) or hypercube (4D,5D)
• Temperature varying with time
• Temperature varying with time and altitude

T
Y
X
4
Gridded Data
Raster
Point Features
5
Multidimensional Data
Time 3
Time 2
Time 1
6
Multidimensional Data
7
Multidimensional Data
Time 3
Time 2
Time 1
8
Multidimensional Data
Time 1
Time 2
Data Cube
Time 3
Time Slices
9
Multidimensional Data
10
Multidimensional Data Formats

• NetCDF (network Common Data Form)
• HDF (4.x and previous releases), HDF-EOS, HDF5
  (Hierarchical Data Format)
• GRIB, GRIB II (GRIdded Binary)

11
What is NetCDF?

• NetCDF (network Common Data Form)
• A platform independent format for representing
  multi-dimensional array-orientated scientific
  data.
• Self Describing - a netCDF file includes
  information about the data it contains.
• Direct Access - a small subset of a large dataset
  may be accessed efficiently, without first
  reading through all the preceding data.
• Sharable - one writer and multiple readers may
  simultaneously access the same netCDF file.
• NetCDF is new to the GIS community but widely
  used by scientific communities for around many
  years

12
Why NetCDF?

• Most commonly used format in the oceanographic
  and atmospheric science for observational data
  and numerical modeling
• The National Center for Atmospheric Research
  (NCAR)
• University Corporation for Atmospheric Research
  (UCAR) NOAA's Climate Diagnostics Center (CDC)
• Los Alamos National Laboratory (LANL)
• The National Center for Supercomputing
  Applications
• US Air Force and Navy
• Atmospheric Research in Australia
• Australia Defense
• UK Hydrographic Office
• NATO
• ...

13
What is a NetCDF file?

• NetCDF is a binary file
• A NetCDF file consists of
• Global Attributes Describe the contents of
  the file
• Dimensions Define the structure of the data
• (e.g Time, Depth, Latitude, Longitude)
• Variables Holds the data in arrays shaped
  by Dimensions
• Variable Attributes Describes the contents of
  each variable
• CDL (network Common Data form Language)
  description takes the following form
• netCDF name
• dimensions ...
• variables ...
• data ...

14
Storing Data in a netCDF File
15
Storing Data in a netCDF File
Time 1 Y 1 X 1 to 4
16
Storing Data in a netCDF File
Time 1 Y 1 to 4 X 1 to 4
17
Storing Data in a netCDF File
Time 1 to 3 Y 1 to 4 X 1 to 4
18
NetCDF Conventions

• CF Conventions (Recommended, if applicable)
• COARDS Conventions (1995 standard that CF
  Conventions extends and generalizes)
• GDT Conventions (1999 standard that CF
  Conventions extends and generalizes)
• CDC Conventions (for gridded data, compatible
  with but more restrictive than COARDS)
• NCAR-RAF Conventions for Aircraft Data
• AMBER Trajectory Conventions for molecular
  dynamics simulations
• NUWG Conventions (1992-1995 effort to create some
  observational data conventions)
• PMEL-EPIC Conventions
• GDV Conventions (deprecated)

19
CF Convention

• Climate and Forecast (CF) Convention
• http//www.unidata.ucar.edu/software/netcdf/docs/c
  onventions.html
• Initially developed for
• Climate and forecast data
• Atmosphere, surface and ocean model-generated
  data
• Comparable observational datasets
• The CF conventions generalize and extend the
  COARDS (Cooperative Ocean/Atmosphere Research
  Data Service) convention.
• CF is now the most widely used conventions for
  geospatial netCDF data. It has the best
  coordinate system handling.

20
Current and Future Standards

• Fill in details
• Galeon WCS
• OGC testbed
• CF v2

21
NetCDF Data Sources

• Community Climate Systems Model (CCSM) 
  http//www.ccsm.ucar.edu, https//www.earthsystemg
  rid.org/
• The CCSM is fully-coupled, global climate model
  that provides state-of-the-art computer
  simulations of the Earth's past, present, and
  future climate states.
• 100 yrs of climate change forecast data
  (2000-2099)
• Control runs (1870-1999) and scenario runs
• Temperature, precipitation flux, surface snow
  thickness, snowfall flux, cloud water content,
  etc.
• Program for Climate Model Diagnosis and
  Intercomparison (PCMDI) http//www-pcmdi.llnl.gov/

22
NetCDF Data Sources

• Vegetation and Ecosystem Modeling and Analysis
  Project (VEMAP) http//dataportal.ucar.edu/vemap/m
  ain.html
• VEMAP was a large, collaborative, multi-agency
  program to simulate and understand ecosystem
  dynamics for the continental United States.
• The VEMAP Data Portal is a central collection of
  files maintained and serviced by the NCAR Data
  Group
• Climate data interval Annual, monthly, and
  daily.
• Data type Historical and model results
• Data Temperature, irradiance, precipitation,
  humidity, incident solar radiation, vapor
  pressure, elevation, land area, vegetation, water
  holding capacity of soil, etc.

23
NetCDF Data Sources

• British Atmospheric Data Center (BADC)
  http//badc.nerc.ac.uk/data/
• The role of the BADC is to assist UK atmospheric
  researchers to locate, access and interpret
  atmospheric data.
• Many datasets are publicly available but datasets
  marked with key symbol have restricted access.
• Datasets are organized by projects or
  organizations.
• Climatology Interdisciplinary Data Collection
  (CIDC) has monthly means of over 70 Climate
  Parameters.
• Met Office - Historical Central England
  Temperature Data has the monthly series, which
  begins in 1659, is the longest available
  instrumental record of temperature in the world.
  The daily series begins in 1772.

24
NetCDF Data Sources

• National Oceanic Atmospheric Administration
  (NOAA)
• National Digital Forecast Database (NDFD)
  http//www.nws.noaa.gov/ndfd/
• Radar Integrated Display with Geospatial Element
  (RIDGE) http//www.srh.weather.gov/ridge/
• Precipitation Analysis http//www.srh.noaa.gov/rfc
  share/precip_download.php
• Climate Diagnostics Center http//www.cdc.noaa.gov
  /

25
NetCDF in ArcGIS

• NetCDF data is accessed as
• Raster
• Feature
• Table
• Direct read (no scratch file)
• Exports GIS data to netCDF

26
NetCDF Tools

• Toolbox Multidimension Tools
• Make NetCDF Raster Layer
• Make NetCDF Feature Layer
• Make NetCDF Table View
• Raster to NetCDF
• Feature to NetCDF
• Table to NetCDF
• Select by Dimension

27
Using NetCDF Data

• Behaves the same as any layer or table Display
• Same display tools for raster and feature layers
  will work on netCDF raster and netCDF feature
  layers.
• Graphing
• Driven by the table just like any other chart.
• Animation
• Multidimensional data can be animated through a
  dimension (e.g. time, pressure, elevation)
• Analysis Tools
• A netCDF layer or table will work just like any
  other raster layer, feature layer, or table.
  (e.g. create buffers around netCDF points,
  reproject rasters, query tables, etc.)

28
NetCDF Layer/Table Properties
Raster
Feature
Table
29
Making a NetCDF Raster Layer

• Drag and drop
• Use the tool

• Usage MakeNetCDFRasterLayer ltin_netCDF_filegt
  ltvariablegt ltx_dimensiongt lty_dimensiongt
  ltout_raster_layergt band_dimension Dimension
  ValueDimension Value... BY_VALUE
  BY_INDEX

30
Changing Time Slice
(Selecting a different dimension value)
31
NetCDF and Spatial Reference

• Geographic Coordinate Systems (GCS)
• X dimension units degrees_east
• Y dimension units degrees_north
• Projected Coordinate Systems (PCS)
• X dimension standard_name projection_x_coordinate
  
• Y dimension standard_name projection_y_coordinate
  
• Variable has a grid_mapping attribute.
• CF conventions currently supports only eight
  predefined coordinate systems
• Undefined
• If not GCS or PCS
• ArcGIS writes (and recognizes) PE String as a
  variable attribute.