Packaging simulation results with CostGlue

1
Packaging simulation results with CostGlue

• Dragan Savic
• Francesco Potortì
• Matev Pustiek

2
Agenda

• Objective
• Formats for exchanging data
• Database
• CostGlue software architecture
• Module API
• Demo
• Conclusions

3
Objective

• To design and implement software that can
• import simulation or measurement results in
  form of trace files into a common data
  structure
• prototype will read ns2 and tcpdump
• launch postprocessing (i.e. filtering and
  calculations) applications on the imported data,
  then
• complement the raw data with the results of
  post-processing,
• store the results of post-processing separately,
• feed them to other applications (e.g. for
  presentation).
• export data into various output formats (ASCII,
  XML, )

4
Database 1/3

• The need for a common file format
• Thorough analysis of different file formats
• HDF4 and HDF5 (Hierarchical Data Format),
• netCDF (Network Common Data Format),
• PDB (Portable Binary Database),
• FITS (Flexible Image Transport System) and
• OpenDX (Data Explorer Visualization Software).
• Results of analysis HDF5 (NCSA) meets all the
  requirements of data organization and storage
• HDF5 format short overview
• two primary objects
• dataset simple dataset (homogonous data) or
  compound dataset (different number of data types
  within one dataset)
• group a structure contain zero or more HDF5
  objects
• hierarchical data organization tree structure
  of HDF5 objects
• support for metadata adding attributes to HDF5
  objects

5
Database 2/3

• One archive composed of several simulation runs
• Flexible representation of the stored simulation
  data matrix with P 2 dimensions
• indexing table represents the logical part
• first P indices data group identification
  (arbitrary simulation run)
• last two indices field and row identification
  in data groups table
• groups and datasets represent actual raw
  simulation data

6
Database 3/3

• HDF5 database structure
• two-level tree with one root group
• raw simulation data datasets under each data
  group
• indexing table, metadata and postprocessing data
  relative to the whole archive and to each
  simulation run

7
CostGlue software architecture

• Functional parts
• CoreGlue
• reads module descriptions,
• executes modules, passes
• parameters, reads module
• execution results,
• handles HDF5,
• provides a HTML interface and,
• generates XML for additional User Interface.
• different modules for
• CLI
• preprocessing (conversion
• to a common format) and
• postprocessing (statistical analysis)
• HTML GUI

8
CostGlue API 1/2
9
CostGlue API 2/2
10
CLI debug demo (1/2)

• Create an archive
• Import data from a file to hdf5 archive

11
CLI debug demo (2/2)

• Export data from hdf5 archive to a file

12
Archive data (1/2)

• Simulation data

13
Archive data (2/2)

• Index table

14
Conclusions

• Facilitate the exchange and management of
  simulation data among researches
• Ease the task of using different simulation,
  measurement, data processing and visualization
  tools
• Software release choosing among MIT X, GNU LGPL
  and GNU GPL licenses