What versions of HDF and HDF-EOS are available and how are they different?

Versions

There are two main versions of the HDF library developed by the National Center for Supercomputing Applications (NCSA): HDF4 and HDF5. These two versions are not compatible and support for both will continue. However, the support for HDF4 will be limited to bug-fixes and buidling and testing on new platforms; no new features will be added. There are also two main versions of HDF-EOS that are based on the HDF libraries. HDF-EOS2.x versions are based on HDF4 and HDF-EOS5.x versions are based on HDF5. There are no HDF-EOS3 or HDF-EOS4 versions.

HDF5 was developed to handle very large datasets (> 2GB), parallel processing environments, data that span multiple computing environments, new data models and more complex visualization and analysis. HDF4 supports only datasets smaller than 2 gigabytes, with fewer than 20,000 datasets in any one file, and is not capable of efficiently performing I/O in parallel computing environments.

Comparison of HDF4 and HDF5 Features

(Source: National Center for Supercomputing Applications (NCSA's) HDF vs. HDF5 web page (September 7, 2001). Retrieved May 16, 2003, from http://hdf.ncsa.uiuc.edu/h4vsh5.html)


HDF HDF (4.x and previous releases)
 
Can handle files over 2GB -
No limit on the number of objects Max limit is 20000 objects and sometimes less
One flexible data model Several rigid data models
Supports parallel I/O Minimal parallel I/O support
Threadsafe (designed to be) -
Supports multi-dimensional array of compound datatypes Supports only 1-dimensional array of compound datatypes
Supports variable length datatypes -
Supports references to objects (?) -
Supports "mounting" of files -
Supports hard/soft links
  -makes delete/move objects easy
 -
Difficult to do
Can flush data immediately -
Extensive datatype/datatype conversion support More restricted
Virtual File layer -
Performance tuning - lots of things Minimal
Ability to register your own compression methods -
Supports multiple unlimited dimensions Supports 1 unlimited dimension
Supports complex subsetting features Supports basic subsetting features
True hierarchical file structure "pseudo-flat" file structure using vgroups
More metadata support Metadata support
Can convert to/from XML -


For a comprehensive list of differences and a list of available documents on this topic, please visit NCSA's FAQ web page..

About Us | Contact Info Sponsored by NASA Cooperative Agreement Grant Number NNX06AC83A -- Maintained by The HDF Group