Workshop on Strategy for Providing Atmospheric Information Panel 2: Interoperability and Compatibility

1
Workshop on Strategy for Providing Atmospheric
InformationPanel 2 Interoperability and
Compatibility

• Formatting Standards
• Kevin Robbins, Director
• Southern Regional Climate Center
• Louisiana State University

2
Data formatting standards are invaluable in
providing interoperability and compatibility to
technically complicated systems. The reward
offered to an IT manager in selecting a standard
is.
.. the opportunity to choose from an infinite
number of competing options ! David Wilensky
(formerly, SRCC)
3
Scope of Discussion

• Storage format standards have little or no effect
  on compatibility between different computer
  systems
• Interoperability and compatibility between
  computer systems is determined by transmission
  and transfer format standards

4
Benefits of Formatting Standards

• Facilitate data exchange
• Provide product consistency
• Facilitate reusable and maintainable software
  code by the producer and consumer
• Standards are
• Open to future improvements yet mindful of past
  technologies

5
Most Formatting Standards Now Used Are Only
Familiar to the Meteorological Community.

• METAR
• SYNOP
• SHIP
• BUFR
• GRIB
• FITS
• SHEF
• SATOB

• NIDS
• PILOT
• TEMP
• BUOY
• RAOB
• SAO
• PIREP
• others

6
Why Are There So Many Formats?

• Format development was incremental
• Transmission bandwidth was limited
• Receiving devices were primitive
• Products were made to be human-readable
• Computer processing power was limited
• Different products had different requirements
• Data formatting standards were unavailable

7
Why Consider Change ?

• The clientele for weather information has become
  more diverse
• Data formatting should be defined using open
  standards having broad acceptance
• Transmission bandwidth has increased
• The need for human-readable products has
  diminished Computers are ubiquitous
• Sophisticated formatting schemes can handle many
  different encoding requirements

8
Clientele Should Drive Decisions !

• In the past, most information was intended for
  internal use, international exchange, or
  (primarily) for the aviation industry
• Now, many industries, large and small, are
  looking for weather information for daily
  operations and decision-making
• Data and data products should be accessible to
  this broader spectrum of clientele

9
Clientele

• NWS (internal needs)
• NOAA Agencies
• Other Federal Agencies
• USDA USFS FAA EPA DOT DOE DOD etc
• International Organizations (WMO)

• Media
• PCMs
• Industry
• Transportation
• Energy
• Construction
• Researchers
• K-12 and Universities
• Citizens

10
How Should ClienteleBe Served ?

• Through Intermediaries
• Media
• PCMs
• Ad hoc WWW providers
• Directly
• Push data to clientele (data feed)
• Allow client pull (website / FTP / client apps)

11
Broader Clientele Interaction Requires Open
Formatting Standards

• Text and Digital Data
• netCDF, HDF, etc
• Spatial Data Transfer Standard (SDTS)
• GRIB (open source)
• BUFR (open source)
• XML
• with CSS or XSL

• Graphic Images
• GIF PNG JPG
• TIFF others.
• Generally volatile
• Generated on-the-fly
• Easy to adapt formats to currently accepted
  standards

12
Multiple Transfer Standards

• Push technologies can rely on a limited number of
  standards, but may benefit from a variety of
  unique data feeds tailored to the intended
  audience
• Pull technologies must support a broader range of
  standards to accommodate the specific
  requirements of individual requests

13
XML ExampleWeather Observation Markup Format
(OMF)
METAR KMRY 091345 11003KT 8SM BKN004 10/09 A3006
MM RMK A02CIG 003V008 SLP193 T01000094MM
ltSYN TitleMetar TStamp950104440 LatLon
36.583, -121.85 BId724915
SnameMonterey Peninsula Elev77gt ltSYIDgtKMRY
091345Zlt/SYIDgt ltSYCODEgtRMK A02 CIG 003V008 SLP193
T01000094lt/SYCODEgt ltSYG Wind110, 1.5
Vis12880 Ceiling400 T10 TD9
AS1018 Clouds66///gt 11003KT 8SM
BKN004 10/09 A306lt/SYGgt lt/SYNgt http//zowie.metne
t.navy.mil/spawar/JMV-TNG/XML/OMF-SYNOP.html
14
Browser Representation (XML / XSL)
KMRY KMRY Time 091354Z Time 091354Z
Monterey Peninsula Monterey Peninsula Elevation 77 Elevation 77
Latitude 36.583 Longitude -121.85
Wind Dir 110 Wind Speed 1.5
Visibility 12880 Ceiling 400
Temperature 10 Dew Point 9
Altimeter Setting 1018 Ceiling 66///
15
Impediments to Change

• Agency resistance to change
• If it aint broke, dont fix it !
• Significant retooling costs
• Compliance to international standards for data
  exchange (WMO)
• Constantly evolving standards
• Complexity of a modernization effort

16
Conclusions

• Change should be driven by clientele demands and
  anticipated needs
• Clientele should be fully engaged in the process
• Modernization should adhere to accepted
  formatting standards
• Technology is no longer a limiting factor in the
  deployment of modern data formats
• Data formats should be designed within a
  comprehensive, internally consistent system

17
And Finally

• Modernization should be
• ..open to future improvements yet mindful of
  past technologies..