Capabilities of the Space Physics Data Facility SPDF to Enable the Heliophysics Virtual Observatorie

1
Capabilities of the Space Physics Data Facility
(SPDF) to Enable the Heliophysics Virtual
Observatories R.E. McGuire and R. M.
CandeyHeliospheric Physics Laboratory (672)NASA
Goddard Space Flight Centerhttp//spdf.gsfc.nas
a.gov
2
Outline

• Goals of this Discussion
• Short Overview of SPDF Data and Services
• Metrics
• Distributed client and server access via
  Webservices
• Our Approach To Date and an Example
• Open Discussion
• VxO and others plans and needs wrt SPDF services
• Could include hosting data for the VxOs?
• What most needed and next?

3
Our Goals Today

• Review how were now making SPDFs unique span of
  data and extensive range of functions accessible
• Directly to users
• To applications via webservices (e.g. to VxO
  middleware)
• Discussion to what we should do next to best
  enable the VxOs and a more effective heliophysics
  data environment
• Users want and expect an integrated environment
• Where and how do VxO groups see tapping our
  capabilities
• How can we offer easier access to SPDF data and
  services?
• Functionality
• Structure/technology
• Description of data and interfaces
• And perhaps as an example for integrating other
  services

4
Extensive Multi-Mission Data and Services

• CDAWeb includes current data from essentially all
  (non-solar) current heliophysics missions and
  experiments since 1990s and some earlier
• Functions include plots, lists, CDF data subsets
  and file downloads,
• Access via FTP, user and webservices application
  interfaces, OPeNDAP
• Standard-based add extensive metadata through
  concept of master CDFs
• A number of datasets include computed variables
• Current CDAS webservices and CDAWeb Plus Java
  interface also support file level access and
  multi-dataset file retrievals for NSSDCs and
  PWGs currently online ftp archival data holdings
  (and other ftp sites)
• Host and maintain OMNI hourly and OMNI High-Res
  (1/5 minute) database of interplanetary
  parameters mapped to Earth at 1AU (and OMNIWeb
  services)
• Host and maintain COHO database and COHOweb
  services
• Uniformized hourly field and plasma data from
  heliospheric s/c
• Developing SPASE-XML product level for combined
  holdings and prototype variable level descriptions

5
The CDAWeb Database- Current, Public

• Plot only shows more recent data
• Full holdings include significant early data

6
Multi-Mission Data and Services (cont)

• SSCWeb offers current/predicted multi-mission
  orbit data and query logic for coordinated
  science planning and retrospective analysis
• Listings, displays, file downloads in many
  coordinate systems and by regions
• Extensive logic to identify s/c-s/c, s/c-region
  and s/c footpoint - ground station conjunctions
  (with extensive list of ground stations)
• Choice of internal/external field models for
  tracings
• TIPSOD 3-D Animated Orbit Java client
• Many s/c, ongoing updates from projects, NORAD
  TLEs and other sources
• Developed and maintain Common Data Format (CDF)
• Core s/w library
• Software to facilitate ISTP/IACG/SPDF compliant
  structure, metadata import
• IDL and MatLab access routines
• IDL CDAWlib library (including read/write) and
  derived CDFX program
• Data translation webservices, direct user i/f and
  software
• CDF lt-gt CDFML (ASCII XML), netCDF, FITS HDF4/5
  -gt CDF all into ASCII
• From a user perspective, SPDF functions as a
  Virtual Observatory
• VSPO element of SPDF is proving ability to
  connect multiple distributed services

7
Metrics

• We capture system execution (request
  file/plot/listing) statistics
• Survey JGR Space Physics and GRL for
  acknowledgements
• By article download and simple literature search
  script for keywords
• In 2005 and 2006, 20 of published JGR Space
  papers acknowledged SPDF

8
SPDFs History in Webservices

• Initial effort to build webservices interface was
  focused to SSCWeb database in order to enable 3-D
  Orbit Viewer Java client
• Added new ability to display s/c relative
  positions in time with zoom, pan
• Used SOAP technology and basic calls to see full
  inventory
• Then challenged in how SPDF could enable/support
  an evolving new NASA heliophysics data paradigm
  of Virtual (discipline) Observatories
• VSO was just defined and starting implementation
• SPDF already successful VO from a user
  perspective, with lots of current data from most
  current (non-solar) heliophysics NASA and
  NASA-related missions
• Based on fresh experience in limited SSCWeb
  webservices, SPDF then undertook an investment to
  make our extensive holdings and capabilities more
  generally applications accessible
• VxOs could pull our capabilities through their
  own interfaces and combine what we could do with
  other distributed data sources
• Create an opportunity for distributed development
  of new (and potentially highly innovative)
  interfaces and more advanced display/analysis
  capabilities
• Developers could easily leverage our holdings to
  build real working prototypes
• As new missions/datasets are subsequently added,
  inherit investment

9
SPDFs Approach to Webservices

• Originally selected SOAP to allow complex queries
  and outputs
• Typically metadata is returned as strings,
  results as URLs
• Now CGI-type interfaces also feasible
• Our Web services were initially implemented
  before Web Services Interoperability (WS-I) Basic
  Profile 1.0 and use RPC-encoded binding style
• Web service standards are evolving, subject to
  interpretation, and implementations may choose
  which options to support
• Different data bind options (RPC-encoded,
  RPC-literal, document-literal)
• Different communication model options
  (synchronous RPC, asynchronous messaging)
• Handling of null Date values (pass by reference
  Java or value .NET)
• Support for passing multi-dimension arrays (ASP
  .NET)
• Handling of empty arrays
• We are changing to document-literal to be
  compliant with WS-I and .NET.
• Methods to allow applications to determine our
  current holdings as well as methods to actually
  request data or services
• And goal to have methods to tap essentially all
  of our capabilities
• Help drive out issues and requirements in
  webservices by defining new clients that can do
  new or different things (e.g. Java clients)

10
Calling CDAWeb services

• SOAP API at lthttp//cdaweb.gsfc.nasa.gov/WebServic
  esgt
• Get lists of available missions, instruments and
  instrument types
• (getAllInstruments, getAllInstrumentTypes,
  getAllMissionGroups)
• Select desired terms from those lists, and get
  list of matching datasets
• (getDatasets, getDatasetsByInstrument,
  getDatasetsBySource)
• Select datasets of interest and get information
  about them,
• including start/end times and variables
• (getDatasetVariables, DatasetDescription)
• Optionally get more information about specific
  data files
• (FileDescription)
• For desired variables, get data as CDFs, text
  listings, CDFML or plots
• (getDataFiles, getDataAsText, getCdfmlDataUrls,
  getDataGraph)
• Can be called from
• IDL programs (IDL 6.0 IDL-Java bridge)
• Java clients and JavaScript
• Perl scripts
• Access to SSC Webservices similar
• More options for custom settings and output
  options
• Webservices are being used now by e.g. VSPO

11
Whats Needed From SPDF Next?

• Webservices exist but are (and will) continue to
  evolve
• SPDF hopes to minimize the number of distinct
  webservices over time
• May be more distinct user interfaces then
  webservice interfaces
• Ongoing challenge to maintain full functionality
• E.g. Data webservice, Ephemeris webservice,
  Format Translation webservice
• New data and missions that we host automatically
  inherit functions and interface
• Continue effort to develop SPASE product and
  variable descriptions
• Add access methods keyed to SPASE terminology
• Standardized (evolution in SPASE) description of
  capabilities and access by dataset and/or product
  and/or variable?
• New standard for structuring queries and
  responses
• VxOs will talk to each other and to provider
  services in a standard way?
• Another layer to translate existing webservices
  to TBD future SPASE standard
• Event lists? Indices and content-based
  metadata?
• Exploit the existing investment in standards and
  metadata