VO Data Model FITS, XML plus NDF

1
VO Data Model FITS, XML plus NDF

• D. Giaretta, P. Wallace, M. Bly, B. McIlwrath
• Starlink, Rutherford Appleton Laboratory, UK
• C. Page
• Dept of Physics Astronomy, University of
  Leicester, UK

2
Possible Requirements

• Flexible and extensible
• Capable of storing hierarchical data
• Strong astronomical data model
• Convenient for local access
• Usable on a wide range of platforms
• Standard enough to be used widely
• Amenable to remote access
• Able to be indexed/interrogated in sophisticated
  ways

3
XML

• Is being used very widely in many disciplines
• Is a key part of WWW development
• Is supported by a multitude of tools
• Provides a hierarchical way to structure
  information
• Cannot hold binary data within file unless
  encoded or otherwise converted to text

4
FITS

• Widely used in Astronomy
• Began as a simple image transport format
• Has become ever more complex and there are
  proposals for further increasing the complexity
• Has limited metadata capabilities
• There are still problems with capturing, for
  example, complex World Coordinate Systems (WCS)

5
Combining FITS and XML

• Simplest case
• XML points to FITS file
• some metadata duplicated from FITS header
• additional metadata also
• FITS file contains the binary data
• Advantage
• Allows additional metadata for remote or
  read-only files
• XML file easily integrated into Semantic Web
• Data (e.g. binary image) can be accessed
  efficiently

6
More complex example

• XML file points to, e.g. image, spectrum and
  table
• Each a separate FITS file
• XML metadata/structure shows the relationship
  between the files
• Same FITS image may be related to may tables by
  means of different XML files
• E.g. multiple spectral extractions and line
  identifications
• Could also include non-FITS data

7
Example
8
VO data handling

• VO applications might be expected to deal with
• Remote data
• Need to add metadata e.g. additional
  calibrations, refined astrometric data
• Large numbers of interrelated files organised
  into various collections
• Any single file may be part of several
  collections
• New links may need to be created by users between
  distributed files
• Distributed/ parallel processing
• Automated processing may be needed e.g. dealing
  with large numbers of files

9
Complex Relationships possible problems

• Applications could create increasingly complex
  aggregations of data, to express increasingly
  complex interrelationships
• How are the relationship between the various
  components defined?
• How can different applications exchange
  information effectively?
• How can automated processing be facilitated?

10
XML/W3C and ISO techniques

• RDF
• XTM
• Tools not yet readily available
• Special astronomical standardised relationships/
  structures needed

11
NDF

• A number of standard relationships should be
  expected between the image/spectrum itself and
  other data (also images/spectra)
• Estimates of data errors
• Information flags for each pixel
• World Coordinate System
• AXIS information
• Centre of each pixel on each axis PLUS
• width of pixels along the axis
• whether or not normalisation is needed
• errors in axis centre positions
• Mechanism needed to handle extensions, together
  with rules for processing/ propagating them

12
(No Transcript)
13
API (e.g. Java binding)

• Based on Extension of DOM (or JDOM for JAVA)
• Normal DOM to deal with structural information
• Data hierarchy
• Interrelationships

14
API image access

• Extensions deal with image data access
• Pixel values
• Error and Quality values
• WCS access
• Includes facilities for multi-threaded and
  distributed access

15
API table access

• Could be a generalised version of image access
  e.g. XDF approach (http//xml.gsfc.nasa.gov/XDF/XD
  F_home.html)
• Or
• Could treat tables as fundamentally different
  beast with own specialised API
• Needs further planning