HDF5 Tools

1
HDF5 Tools

• June 17, 2004

2
HDF5 tools power tools

• h5dump prints contents of an HDF5 file
• h5ls prints contents of an HDF5 file
• h5diff show differences in hdf5 objects
• h5repack copy hdf5 objects, optionally applying
  filters

3
HDF5 tools other tools

• gif2h5, h52gif conversion between GIF-HDF5
• h5debug - print low level information
• h5import convert ASCII, binary data to HDF5
• h5repart - repartitions a file family

4
h5dump - output

• HDF5 "../testfiles/file1.h5"
• GROUP "/"
• DATASET "dset1"
• DATATYPE H5T_IEEE_F64LE
• DATASPACE SIMPLE ( 3, 2 ) / ( 3, 2 )
  
• DATA
• (0,0) 1, 1,
• (1,0) 1, 1,
• (2,0) 1, 1

5
h5ls - output

• /g1 Group
• /g1/dset1 Dataset 3, 2
• Data
• (0,0) 1, 1, 1, 1, 1, 1

6
h5dump - options

• -H, --header Print the header only no
  data is displayed
• HDF5 "../testfiles/file1.h5"
• GROUP "/"
• GROUP "g1"
• DATASET "dset1"
• DATATYPE H5T_IEEE_F64LE
• DATASPACE SIMPLE ( 3, 2 ) / ( 3, 2 )
  

7
h5dump - options

• -d P, --datasetP Print the specified dataset
• HDF5 "../testfiles/file1.h5"
• DATASET "g1/dset1"
• DATATYPE H5T_IEEE_F64LE
• DATASPACE SIMPLE ( 3, 2 ) / ( 3, 2 )
• DATA
• (0,0) 1, 1,
• (1,0) 1, 1,
• (2,0) 1, 1

8
h5dump new options

• -n, --contents Print a list of the file
  contents and exit
• HDF5 "../testfiles/tfilters.h5"
• FILE_CONTENTS
• dataset /all
• datatype /mytype
• group /mygroup
• dataset /mygroup/shuffle
• dataset /string
• dataset /szip
• dataset /vlen

9

• -B, --bootblock Print the content of the
  boot block
• HDF5 "../testfiles/tfilters.h5"
• SUPER_BLOCK
• SUPERBLOCK_VERSION 0
• FREELIST_VERSION 0
• SYMBOLTABLE_VERSION 0
• OBJECTHEADER_VERSION 0
• USERBLOCK_VERSION 0
• OFFSET_SIZE 8
• LENGTH_SIZE 8
• BTREE_RANK 16
• BTREE_LEAF 4
• FILE_DRIVER H5FD_SEC2
• ISTORE_K 32
• GROUP "/"
• DATASET "all"

10

• -p, --properties Print dataset filters,
  storage layout and fill value (deflate filter
  example)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "deflate"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 385 ( 10, 5 )
• FILTERS
• COMPRESSION DEFLATE LEVEL 9
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME INCR
• VALUE -99

11

• -p, --properties Print dataset filters,
  storage layout and fill value (SZIP filter
  example)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "szip"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 372 ( 10, 5 )
• FILTERS
• COMPRESSION SZIP
• PIXELS_PER_BLOCK 4
• MODE K13
• CODING NEAREST NEIGHBOUR
• BYTE_ORDER LSB
• HEADER RAW
• FILLVALUE

12

• -p, --properties Print dataset filters,
  storage layout and fill value (checksum filter
  example)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "fletcher32"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 816 ( 10, 5 )
• FILTERS
• CHECKSUM FLETCHER32
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME INCR
• VALUE -99

13

• -p, --properties Print dataset filters,
  storage layout and fill value (shuffle filter
  example)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "shuffle"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 800 ( 10, 5 )
• FILTERS
• PREPROCESSING SHUFFLE
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME INCR
• VALUE -99

14

• -p, --properties Print dataset filters,
  storage layout and fill value (user filter
  example)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "myfilter"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 800 ( 10, 5 )
• FILTERS
• UNKNOWN_FILTER 405 PARAMS 5 6
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME INCR
• VALUE -99

15

• -p, --properties Print dataset filters,
  storage layout and fill value (compact layout)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "compact"
• COMMENT "This is a dataset with compact storage"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT COMPACT
• SIZE 800
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME EARLY
• VALUE -99

16

• -p, --properties Print dataset filters,
  storage layout and fill value (contiguous layout)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "contiguous"
• COMMENT "This is a dataset with contiguous
  storage"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CONTIGUOUS
• SIZE 800 OFFSET 4096
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME LATE
• VALUE -99

17

• -p, --properties Print dataset filters,
  storage layout and fill value (chunked layout)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "shuffle"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 800 ( 10, 5 )
• FILTERS
• PREPROCESSING SHUFFLE
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME INCR
• VALUE -99

18

• -p, --properties Print dataset filters,
  storage layout and fill value (external file)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "external"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 100 ) / ( 100 )
• STORAGE_LAYOUT CONTIGUOUS EXTERNAL
• FILENAME ext1.bin SIZE 200 OFFSET 0
• FILENAME ext2.bin SIZE 200 OFFSET 0
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME LATE
• VALUE 0

19

• -p, --properties Print dataset filters,
  storage layout and fill value (fill value
  example)
• HDF5 "../testfiles/tfilters.h5"
• DATASET "shuffle"
• DATATYPE H5T_STD_I32LE
• DATASPACE SIMPLE ( 20, 10 ) / ( 20, 10 )
• STORAGE_LAYOUT CHUNKED
• SIZE 800 ( 10, 5 )
• FILTERS
• PREPROCESSING SHUFFLE
• FILLVALUE
• FILL_TIME IFSET
• ALLOC_TIME INCR
• VALUE -99

20

• -e, Print string data with tabs and newlines
• HDF5 "../testfiles/tfilters.h5"
• DATASET "string"
• DATATYPE H5T_STRING
• STRSIZE 59
• STRPAD H5T_STR_NULLTERM
• CSET H5T_CSET_ASCII
• CTYPE H5T_C_S1
• DATASPACE SIMPLE ( 1 ) / ( 1 )
• DATA
• (0) "this is
• a string with three
• newline
• characters"

21

• DATA
• (0) "GROUPSwathStructure
• GROUPSWATH_1
• SwathName\"H2SO4_H2O_Tisd
  ale\"
• GROUPDimension
• 
  OBJECTDimension_1
• 
  DimensionName\"nChans\"
• Size21
• 
  END_OBJECTDimension_1
• 
  OBJECTDimension_2
• 
  DimensionName\"nRmax\"
• Size20
• 
  END_OBJECTDimension_2
• 
  OBJECTDimension_3
• 
  DimensionName\"nSmax\"
• Size20
• 
  END_OBJECTDimension_3
• 
  OBJECTDimension_4
• 
  DimensionName\"nCompmax\"

22

• Proposed output without the escaped quotes
• DATA
• (0) "GROUPSwathStructure
• GROUPSWATH_1
• SwathName"H2SO4_H2O_Tisda
  le"
• GROUPDimension
• 
  OBJECTDimension_1
• 
  DimensionName"nChans"
• Size21
• 
  END_OBJECTDimension_1
• 
  OBJECTDimension_2
• 
  DimensionName"nRmax"
• Size20
• 
  END_OBJECTDimension_2
• 
  OBJECTDimension_3
• 
  DimensionName"nSmax"
• Size20
• 
  END_OBJECTDimension_3
• 
  OBJECTDimension_4

23

• DATA
• (0) "GROUPSwathStructure\n\tGROUPSWATH_1
  \n\t\tSwathName\"H2SO4_H2O_Ti
• sdale\"\n\t\tGROUPDimension\n\t\t\tOBJECTDimensi
  on_1\n\t\t\t\tDimensionName\"
• nChans\"\n\t\t\t\tSize21\n\t\t\tEND_OBJECTDimens
  ion_1\n\t\t\tOBJECTDimension_
• 2\n\t\t\t\tDimensionName\"nRmax\"\n\t\t\t\tSize2
  0\n\t\t\tEND_OBJECTDimension_
• 2\n\t\t\tOBJECTDimension_3\n\t\t\t\tDimensionName
  \"nSmax\"\n\t\t\t\tSize20\n\
• t\t\tEND_OBJECTDimension_3\n\t\t\tOBJECTDimensio
  n_4\n\t\t\t\tDimensionName\"n
• Compmax\"\n\t\t\t\tSize8\n\t\t\tEND_OBJECTDimens
  ion_4\n\t\tEND_GROUPDimension
• \n\t\tGROUPDimensionMap\n\t\tEND_GROUPDimensionM
  ap\n\t\tGROUPIndexDimensionMa
• p\n\t\tEND_GROUPIndexDimensionMap\n\t\tGROUPGeoF
  ield\n\t\tEND_GROUPGeoField\n
• \t\tGROUPDataField\n\t\t\tOBJECTDataField_1\n\t\
  t\t\tDataFieldName\"Wavenumbe
• r\"\n\t\t\t\tDataType\n\t\t\t\tDimList(\"nChans\
  ")\n\t\t\t\tMaxdimList(\"nCha
• ns\")\n\t\t\tEND_OBJECTDataField_1\n\t\t\tOBJECT
  DataField_2\n\t\t\t\tDataField

24

• Escaped characters (other than \n and \t)
• case '"'
• h5tools_str_append(str, "\\\"")
• case '\\'
• h5tools_str_append(str, "\\\\")
• case '\b' (backspace)
• h5tools_str_append(str, "\\b")
• case '\f' (form feed)
• h5tools_str_append(str, "\\f")
• case '\r' (carriage return)
• h5tools_str_append(str, "\\r")

25

• Path of referenced objects
• HDF5 "../testfiles/tfilters.h5"
• DATASET "reference"
• DATATYPE H5T_REFERENCE
• DATASPACE SIMPLE ( 5 ) / ( 5 )
• DATA
• (0) DATASET 976 /compact

26
Summary

• -p option controls filters, layout, fill value
• -B option controls superblock
• -n option controls file contents
• -e option controls print new line and tabs
• Array indices , no control option, (on by
  default, off for XML)
• Path of referenced objects. Always print path, no
  control option