GRIB2 and its implementation in INT2LM and the COSMO-Model

1
GRIB2 and its implementationin INT2LM and the
COSMO-Model

• Ulrich Schättler
• Source Code Administrator
• COSMO-Model and (still) INT2LM

2
Contents

• Structure of GRIB2
• Usage of grib_api
• Local Use Areas
• New General Vertical Coordinate

3
GRIB2 ? - WMO - Code for forecast products

• GRIdded Binary Edition 0 (1985/88), Edition 1
  (1990ff)
• General Regularly-distibuted Information in
  Binary form Edition 2 (2001)
• self explaining, compressed / packed
• GRIB message contains metadata (explaining the
  product, grid,) plus data
• Edition 2 is more flexible through a
  template/table structure,
• ammendments by adding templates/tables.
• Version 2 2003
• Version 3 2005
• Version 4 2007
• Version 5 2008
• Version 6 2009
• etc.

• Changes include
• coding of new elements/parameter
• new product definition templates
• new packing/compressing formats
• new horizontal grid(s), e.g. ICON
• new vertical coordinate type

? Dörte Liermann
4
Comparing GRIB1 GRIB2
GRIB 1 vs. 2 Different Structure GRIB 1 SECTION
0 Indicator (GRIB) SECTION 1 Product
definition SECTION 2 Grid Description SECTION 3
Bitmap SECTION 4 Binary Data SECTION 5 End (7777)
GRIB 2 SECTION 0 Indicator(GRIB,) SECTION 1
Identification SECTION 2 Local Use SECTION 3 Grid
Definition SECTION 4 Product Definition SECTION 5
Data Representation SECTION 6 Bitmap SECTION 7
Binary Data SECTION 8 End (7777)

• In GRIB 2 several variables are defined with
  more precision (e.g. angles are in
  micro-degrees, more vertical coordinates
  possible)
• see www.cosmo-model.org/content/model/documenta
  tion/grib/grib2keys.htm
• In GRIB 2 the description of the data
  (parameter, time, statistics,) is more complex
  and is template/table based

? Dörte Liermann
5
Element coding of model fields in GRIB2

• GRIB1 Element coding by ee (element) and tab
  (table)
• Example temperature is ee11 in tab2 (WMO)
  
• GRIB2 Element coding by a triplet
• Discipline
• Category
• Parameter
• Example temperature is (0,0,0)
• Through setting of shortName, these three
  parameters are set (and for some products also
  some other keys see shortName.def in
  definitions.edzw)

? Dörte Liermann
6
GRIB_API (Application Programmers Interface)

• ECMWF source code for de-/encoding of GRIB1 AND
  GRIB2
• no internal knowledge of GRIB structure needed
• each element of a grib message has an
  alphanumeric name (key) that can be used to
  access the information linked to it (value)
• key value approach shortNameT
• typeOfLevel hybridLayer
• topLevel 37
• bottomLevel 38
• How to find keys grib_keys F file.grib lists
  all keys of a file
• flexible local definitions for each centre
  possible
• i.e. local definition tables shortName.def for
  edzw (GRIB1 and GRIB2),
• local definition of GME icosahedral grid for
  GRIB1
• But needs more coordination within COSMO

? Dörte Liermann
6
7
Product Identifying Keys

• significanceOfReferenceTime (GRIB2-WMO Tab. 1.2)
• productionStatusOfProcessedData (GRIB2-WMO Tab.
  1.3)
• typeOfProcessedData (GRIB2-WMO Tab. 1.4)
• generatingProcessIdentifier (local definition in
  GRIB1/2)
• backgroundGeneratingProcessIdentifier (local
  definition in GRIB2)
• typeOfGeneratingProcess (GRIB2-WMO Tab. 4.3)
• localDefinitionNumber (local section 254, 253,
  252 250 for COSMO)
• localNumberOfExperiment (GRIB2)
• localInformationNumber (GRIB2) /
  localElementNumber (GRIB1)
• localDecodeDates

8
Product Identifying Keys (II)
Key Values
significanceOfReferenceTime 0 analysis 1 start of forecast 2 verifying time of forecast 3 observation time
productionStatusOf ProcessedData (possible local use) 0 operational Routine 1 operational test Parallelsuite 2 research Experiments 3 re-analysis products
typeOfProcessedData 0 analysis 1 forecast 2 analysis and forecast 3 control forecast 4 perturbed forecast 5 control and perturbed forecast
9
Product Identifying Keys (III)
Key Values
typeOfGeneratingProcess (local use) 0 analysis 1 initialization 2 forecast 195 interpolated analysis / forecast
backgroundGeneratingProcessIdentifier (local use) 0 main run 1 pre-assimilation 2 assimilation 3 test
generatingProcessIdentifier (local use) Represents data base identifier
10
Local Use Section (localDefinitionNumber254)

• localDefinitionNumber Identifier for content
  (historical 254)
• localHostIdentifier
• localCreationDateYear/Month/Day/Hour/Minute/Second
  
• localValidityDateYear/Month/Day/Hour/Minute/Second
  
• localNumberOfExperiment Number of Experiment
• localInformationNumber
• Identifier for host system/ computer
• Also need a local use section for COSMO
  (localDefinitionNumber 250)

11
Implementation of Local Use

• SELECT CASE (ncenter)
• CASE (78) ! DWD
• CASE DEFAULT
• END SELECT
• We soon need to know about your local use of
  special keys!
• We want to document local use on the Web Page!

12
General Vertical Coordinate

• typeOfLevel 150

13
Why a new vertical coordinate?

• As a non-hydrostatic model, COSMO needs a special
  vertical grid fixed in space and time
• Also post-processing programs have to be aware of
  this grid (or the HHL)
• But the algorithm to compute it, is rather
  complex (not just ak bk ps)
• Therefore a proposition was made to WMO, to
  introduce a new typeOfLevel150
• To process atmospheric data using that
  typeOfLevel, another 3D field is necessary the
  HHL fields
• If a product has typeOfLevel150, then there are
  6 additional meta data in the Product Definition
  Section, which replace the vertical coordinate
  parameters
• numberOfVGridUsed to identify a special vertical
  coordinate
• nlev number of levels of the HHL file
• uuidOfVGrid unique universal identifier
• to ensure correct identification of HHL

14
Current Situation

• INT2LM and the COSMO-Model both compute the HHL
  fields and the reference atmosphere p0.
• The necessary vertical coordinate parameters (for
  HHL) and the reference atmosphere parameters are
  given to
• INT2LM by Namelist variables
• COSMO-Model by GRIB1 meta data but this always
  was a non-standard GRIB usage!
• The new generalized vertical coordinate does not
  know meta data for vertical coordinate parameters
  and for the reference atmosphere.
• How to compute HHL and p0 when using GRIB2?
• Or what can we do?

15
What can be done?

• HHL can be transferred from INT2LM to COSMO or
  within the assimilation cycle from COSMO to COSMO
  by an extra file.
• Do we then still need to transfer the vertical
  coordinate parameters vcoordk and vcflat?
• YES! For computations in the nudging and some
  diagnostices (meanvalues, convective indices).
  Try to discuss this with the code experts.
• Can we also transfer p0? Or the full pressure?
• For full pressure the problem is the GRIB
  packing. But we could use more bits for the
  packing (pressure deviation is a more homogeneous
  field with less precision loss in the packing)
• Other question Has anybody ever changed the
  Namelist variables in INT2LM (p0sl, t0sl, dt0lp,
  delta_t, h_scal) to compute another p0?

16
Current implemented Method for HHL

• INT2LM can compute HHL and write an extra file
  COSMO_HHL_xxx.g2 (if lnewVGrid .TRUE.) or read
  it from such a file (is done in IEEE 64-bit)
• (Note for COSMO ? COSMO such a HHL file has also
  to be read for the coarse grid)
• INT2LM computes the atmospheric variables based
  on this HHL
• The COSMO-model has to read the same
  COSMO_HHL_xxx.g2 file to get the same HHL field
  as the INT2LM (checking the UUID should ensure
  using the correct HHL field)
• INT2LM and the COSMO-Model define several sets of
  vertical coordinate parameter defaults (for
  COSMO-EU, COSMO-DE, COSMO-I7, COSMO-7, etc.).
  Every set has a special ID
• HHL stores the ID of the default set used in the
  localInformationNumber in HHL

17
Current implemented Method for p0

• INT2LM and the COSMO-Model both compute p0 (as
  before).
• To get the correct reference atmosphere
  parameters, both programs store several sets of
  default parameters. Every set has a special ID
• pp stores the ID of the used set in the local
  information number
• INT2LM and the COSMO-Model now use the new module
  vgrid_refatm_defaults.f90

18
Problems and Drawbacks of this Approach

• Have to deal with another file in our suites
• Have to hard-code all possible COSMO
  configurations in the source code

19
What can be done about HHL

• Can we transfer HHL through the laf-file? Perhaps
  in higher precision? But this imposes problems to
  archiving!
• Do we need full precision?
• Check , whether usage of vcoordk, vcflat is
  really necessary or can be avoided

20
What can be done about p0

• Can we transfer full pressure (or p0) through the
  laf-file? Perhaps in higher precision? Again this
  imposes problems to archiving!
• Which precision is really needed?
• Check , whether usage of reference atmosphere
  parameters can be fixed

21
Migration to GRIB2 Timetable

• The plans already pronounced by DWD have been
  revised last week
• A prerequisite for GRIB2 at DWD is the stable
  operation of the new database software SKY (old
  software is not able to process GRIB2 meta data).
  And this is not the case at the moment.
• New date for operational use of SKY (only the
  SKY is the limit) November
• If this date can be kept, then we could switch to
  GRIB2 in January 2014. If not, GRIB2 will not
  come before May / June 2014. Because in February
  / March we are migrating to the new CRAY computer
  system.
• Message for COSMO and other partners
• Be prepared to read GME data in GRIB2 with INT2LM

22
Thank you very much for your attention
10.09.2012
COSMO General Meeting 2012
22