nub - npoess user block tool

1
nub - npoess user block tool

• Richard Ullman
• NASA GSFC

2
nub

• nub (nub) n.
• 1. A protuberance or knob.
• 2. A small lump.
• 3. The essence the core the nub of a story.
• Variant of knub, probably from Low German
  knubbe, from Middle Low German, variant of
  knobbe.
• (selected from http//www.answers.com/topic/nub)

3
NPOESS user block

• For NPOESS files, the user block is ASCII text
  written in XML that provides a quick-look into
  the metadata attributes contained in the HDF5
  file.
• The content of the user block includes metadata
  that is attached to the NPOESS
• / (root) group
• ltcollection_shortnamegt group, and
• ltcollection_shortnamegt_Aggr group.

4
nub_Motivation

• When evaluating NPOESS sample data last year, I
  wished to examine the user block, as a
  stand-alone item, but I found that the h5unjam
  tool to be expensive
• h5unjam reads an HDF5 file and writes two output
  files
• 1. a file containing the content of the user
  block
• 2. a file containing the H5 data without the user
  block.
• The second part was unnecessary for my purposes,
  but because the NPOESS files are large, it is
  expensive.
• So .. I wrote nub

5
nub_Design

• Design criteria
• c program (thats all I know how to write)
• Efficient - do no EXTRA work.
• Command line operation, with Unix-like options.
• Pipe-able. (stdin/stdout)
• Assume user block is text.
• Work only on the front user block.
• Complete set of functions to manipulate the user
  block
• List, Create, Overwrite, Append, Delete, Get size
• Requirements creep
• Complete replacement for h5jam/h5unjam/tellub
• No reliance on h5 library (no need).
• Availability
• Applied for NASA Open Source release. All
  paperwork completed. Waiting for final approval.
• Evaluation copy available to anyone interested in
  NPOESS.

6
nub operates on the front user block ONLY
signature \211HDF\r\n\32\n
HDF5
user block
sizeof( user_block) 2n, where ngt8, (or n0)
7
nub file.h5
signature \211HDF\r\n\32\n
HDF5
content,content/0/0/0/0/0/0/0/0
--tell -b
--tell -a
--tell -s

• Tell name comes from HDF5 internal tool
  tellub
• This is the nub default operation.
• Count the size of the user block content -b,
  the size of the user block -s, and the size of
  the complete H5 file -a in bytes.
• No options given is equivalent to nub --tell
  -bsa file.h5
• Output is similar to wc

8
nub -l file.h5
signature \211HDF\r\n\32\n
file.txt
HDF5
content,content/0/0/0/0/0/0/0/0

• List list the content of the user block
• write to stdout, or if -f write to the given file

9
nub -w file.h5
signature \211HDF\r\n\32\n
user_file.txt
HDF5
content,content/0/0/0/0/0/0/0/0

• Write write text content to the user block
• append by default, or clobber (replace)
• fit into existing space (by overwriting only null
  chars) by default, or jam (extend the block)
• kram option shrinks user block into smallest
  that will fit the content
• read from stdin, or use the file specified on the
  -u switch

10
nub_ compatibility

• nub will respond to the same command line options
  that are used with h5jam and h5unjam.
• nub -J is an explicit substitute for h5jam
• nub -U is an explicit substitute for h5unjam
• nub -T is an explicit substitute for tellub
• Or, use just nub as a substitute for either and
  the code will decide which of the two is most
  appropriate based on the flags provided.

11
nub_test

• Results of nub as a substitute for h5jam, h5unjam
  in the standard hdf5 tools regression tests
• 68 PASSED
• 6 FAILED
• fails are due to difference in concept of
  concatenation.
• nub appends to the end of the content of an
  existing user block.
• h5jam appends to the end of the existing user
  block.
• User blocks are of size2n, where size gt 256 and
  is big enough to fit the content. So, if user
  block content is 513 bytes, the size of the user
  block is 1024 bytes, including 511 bytes of NULL
  padding.
• nub will overwrite the NULL padding, h5jam leaves
  the padding in place.