Hierarchical Data Format

From Wikipedia, the free encyclopedia

Hierarchical Data Format, commonly abbreviated HDF, HDF4, or HDF5 is a library and multi-object file format for the transfer of graphical and numerical data between computers. It is created and maintained by the NCSA. The freely available HDF distribution consists of the library, command-line utilities, test suite source, Java interface, and the Java-based HDF Viewer (HDFView).

HDF supports several different data models, including multidimensional arrays, raster images, and tables. Each defines a specific aggregate data type and provides an API for reading, writing, and organizing the data and metadata. New data models can be added by the HDF developers or users.

HDF is self-describing, allowing an application to interpret the structure and contents of a file without any outside information. One HDF file can hold a mixture of related objects which can be accessed as a group or as individual objects. Users can create their own grouping structures called "vgroups."

Contents 1 HDF5 2 Interfaces 2.1 Low-level APIs 2.2 High-level APIs 3 See also 4 External links

[edit] HDF5

The HDF5 format is designed to address some of the limitations of the HDF4 library, and to address current and anticipated requirements of modern systems and applications.

The next version of NetCDF, version 4, is based on HDF5.

Because it uses B-Trees to index table objects, it works well for Time series data like stock market ticks or network monitoring data. The bulk of the data goes into straightforward arrays (the table objects) that can be accessed much quicker than the rows of an SQL database. But you still have B-Tree access for non-array data. If you find yourself designing a Star schema to fit your data into SQL, then you might want to investigate HDF5 as a simpler, faster alternative storage mechanism.

[edit] Interfaces

[edit] Low-level APIs

• C
• C++
• Fortran
• F90
• Java
• Perl
• PyTables – an interface for Python

[edit] High-level APIs

• HDF5 Lite (H5LT) – a light-weight interface for C
• HDF5 Image (H5IM) – a C interface for images or rasters
• HDF5 Table (H5TB) – a C interface for tables
• HDF5 Packet Table (H5PT) – interfaces for C and [C++] to handle "packet" data, accessed at high-speeds
• HDF5 Dimension Scale (H5DS) – allows dimension scales to be added to HDF5; to be introduced in the HDF5-1.8 release

[edit] See also

• Common Data Format (CDF)
• NetCDF
• FITS, a data format used in astronomy
• GRIB (GRIdded Binary), a data format used in meteorology
• Q5cost a FORTRAN API to use hdf5 in quantum chemistry

[edit] External links

• HDF home page
• NCSA HDF5 home page
• What is HDF5?
• "An Introduction to Distributed Visualization"; section 4.2 contains a comparison of CDF, HDF, and netCDF.
• A presentation on how to handle large datasets in Quantum Chemistry using hdf5

This article was originally based on material from the Free On-line Dictionary of Computing, which is licensed under the GFDL.