Coordination of Common Modeling Infrastructure

1
Coordination of Common Modeling Infrastructure
Climate
Data Assimilation
Weather
Cecelia DeLuca WGCM/WMP Meeting, Exeter,
UK cdeluca_at_ucar.edu Oct 6, 2005
2
Outline

• What is ESMF?
• How Do ESMF and PRISM Differ?
• Why Do ESMF and PRISM Differ?
• Can ESMF and PRISM Be Usefully Combined?
• Model Metadata and Earth System Curator
• How Can WMP Help?

3
ESMF Background
ESMF grew out of the now defunct Common Modeling
Infrastructure Working Group, which involved many
operational and research centers in the
U.S.(Steve Zebiak and Robert Dickenson chairs).

• Three linked proposals were funded by NASA ESTO
  in 2002
• Core framework(Killeen/NCAR)
• Modeling applications (Marshall/MIT)
• Data assimilation applications (da Silva/NASA
  GSFC)

Original ESMF applicationsNOAA GFDL
atmospheresNOAA GFDL MOM4 oceanNOAA NCEP
atmosphere, analysesNASA GMAO models and
GEOS-5NASA/COLA Poseidon oceanLANL POP ocean
and CICENCAR WRFNCAR CCSMMITgcm atmosphere and
ocean
4
New ESMF-Based ProgramsFunding for Science,
Adoption, and Core Development
5
What is ESMF?

• ESMF provides tools for turning model codes into
  components with standard interfaces and standard
  drivers.
• ESMF provides data structures and common
  utilities that components use for routine
  services such as data communications, regridding,
  time management, configuration, and message
  logging.

• Outputs and outcomes
• Open-source, collaboratively developed software
  utilities and coupling interfaces, exhaustive
  test suite, documentation, support and training.
• A federation of geophysical components that can
  be assembled in multiple ways, using different
  drivers and different couplers.
• A Earth science organization that has focused
  interactions at many levels software engineer
  and support scientist, technical and scientific
  manager, scientist, director, sponsor.
• An extended community with strong connections and
  many diverse science options.

6
ESMF Components and Couplers
Application ExampleGEOS-5 AGCM

• Each box is a user-written ESMF component
• Every component has a standard interface so that
  it is (technically) swappable
• Data in and out of components are packaged as
  state types with user-defined fields
• New components can easily be added to the
  hierarchical system
• Many different structures can be assembled by
  switching the tree around

7
But!
I

• It is possible to do a wrap of an existing
  model with ESMF, without needing to change
  internal data structures, by just creating one
  Component box
• This is generally lightweight in terms of
  performance
• Users can choose to use all of ESMF or just some
  of it

• Measures overhead of ESMF superstructure in NCEP
  Spectral Statistical Analysis (SSI), 1 overall
• Run on NCAR IBM
• Runs done by JPL staff, confirmed by NCEP
  developers

8
ESMF Development Status

• Concurrent or sequential execution, single or
  multiple executable
• Support for configuring ensembles
• Logically rectangular grids with regular and
  arbitrary distributions can be represented and
  regular distributions can be regridded
• On-line parallel regridding (bilinear, 1st order
  conservative) implemented and optimized
• Other parallel methods - e.g. halo,
  redistribution, low-level comms implemented
• Utilities such as time manager, logging, and
  configuration manager usable and adding features
• Fortran interfaces and complete documentation,
  some C interfaces
• ESMF software is not yet a hardened,
  out-of-the-box solution

9
ESMF Platform Support

• IBM AIX (32 and 64 bit addressing)
• SGI IRIX64 (32 and 64 bit addressing)
• SGI Altix (64 bit addressing)
• Cray X1 (64 bit addressing)
• Compaq OSF1 (64 bit addressing)
• Linux Intel (32 and 64 bit addressing, with mpich
  and lam)
• Linux PGI (32 and 64 bit addressing, with mpich)
• Linux NAG (32 bit addressing, with mpich)
• Linux Absoft (32 bit addressing, with mpich)
• Linux Lahey (32 bit addressing, with mpich)
• Mac OS X with xlf (32 bit addressing, with lam)
• Mac OS X with absoft (32 bit addressing, with
  lam)
• Mac OS X with NAG (32 bit addressing, with lam)
• User-contributed g95 support

10
Current Challenges

• Refocus core development team
• Base infrastructure is complete now need
  support for unstructured grids, multi-block grids
  with complex boundary behavior (e.g. tripole,
  cubed sphere), more regridding options, and
  constructs for data assimilation
• Team composition must change correspondingly
• Better, smarter testing suite of 1600 unit
  tests, 15 system tests, 30 examples still needs
  supplements
• Major increase in demand for customer support and
  training
• Many new requirements
• Commercial tool for tracking requirements (DOORS)
• New representative body for prioritizing
  development tasks (Change Review Board)
• Organizationally and technically, ESMF
  infrastructure will take another 3-5 years to
  mature

11
ESMF v PRISM
PRISM
Run-time environment
Coupling Superstructure
User Code
Utility Infrastructure
ESMF
12
Other Differences
PRISM
ESMF
Seasonal Forecast
Coupler
Comp
Comp
Comp
Comp

• Components are generally in separate executables
• Components are generally not nested
• Single coupler
• Data is transferred through put/get
• Data can go from anywhere to anywhere in another
  component

ocean
assim_atm
sea ice
assim
atmland
coupler
atm
land

• Components are generally in the same executable
• Components are often nested
• Multiple couplers
• Data is passed through states at the beginning
  and end of method execution

13
Motivation for Common Modeling Infrastructure
PRISM

• Support for modeling workflows (e.g. job
  submission, version control, annotation and
  archival of experiments, integration with
  visualization and analysis tools)
• Model intercomparison and interchange of model
  components
• Better utilization of compute resources and
  performance optimization
• Cost effectiveness shared, fully featured
  common utilities (e.g. logging, timing,
  regridding, calendars, I/O, parallelization
  tools)
• Systematic internal architecture of
  multi-component models, support for many
  different drivers and configurations

ESMF
14
Why Do ESMF and PRISM Differ?

• For both ESMF and PRISM, overall design was
  decided by a large group of experienced modelers
  so how did the two efforts wind up with such
  different solutions?
• PRISM single-driver approach leads to greater
  effective interoperability for a constrained
  (climate) domain
• ESMF approach leads to limited interoperability
  for a broader set of domains climate, weather,
  space weather, data assimilation support for
  seamless prediction
• Both ESMF and PRISM face similar requirements
  but have taken different paths to fulfill them

15
Can ESMF and PRISM be Usefully Combined?

• ESMF can use PRISM run-time elements
• PRISM can use the ESMF utility layer
• ESMF can offer a put/get paradigm for greater
  flexibility
• ESMF components can be described using PRISM
  PMIOD files (XML description of model
  inputs/outputs and content), and ESMF data
  transfers expressed as PRISM put/gets, so that
  the same component can run in both systems (done
  with MOM4)

16
Model Metadata and Earth System Curator

• Earth System Curator takes the interaction of
  ESMF/PRISM a step further
• Recognize models and datasets are described by
  similar metadata
• Develop standards for model metadata, especially
  in the area of grids
• Work with umbrella groups developing metadata
  standards (e.g. GO-ESSP) to integrate model and
  data metadata
• Work with groups developing ontologies (LEAD,
  ESML) to invest metadata standards with structure
  and flexibility
• Work with GFDL, CCSM and PCMDI to link databases
  that store models, experiments, and data to serve
  MIPs and IPCC
• Anticipated result
• Coordinated growth of ESMF and PRISM
• Opportunities to develop smarter tools (e.g.
  compatibility, assembly) based on metadata
  information

17
How Can WMP Help?

• Support and promote common modeling
  infrastructure
• Maintain a science-driven methodology
• Emphasize long-term investment and continuity
• Communicate expectations the plug and play
  myth
• Support and promote efforts to generate metadata
  standards and ontologies
• For the interaction of ESMF and PRISM
• For the development of a more comprehensive and
  useful modeling environment
• Help determine how to utilize infrastructure as
  an entry point into the broader (international)
  modeling community