Versions
There are two main versions of the HDF library developed by the National Center for Supercomputing Applications (NCSA): HDF4 and HDF5. These two versions are not compatible and support for both will continue. However, the support for HDF4 will be limited to bug-fixes and buidling and testing on new platforms; no new features will be added. There are also two main versions of HDF-EOS that are based on the HDF libraries. HDF-EOS2.x versions are based on HDF4 and HDF-EOS5.x versions are based on HDF5. There are no HDF-EOS3 or HDF-EOS4 versions.
HDF5 was developed to handle very large datasets (> 2GB), parallel processing environments, data that span multiple computing environments, new data models and more complex visualization and analysis. HDF4 supports only datasets smaller than 2 gigabytes, with fewer than 20,000 datasets in any one file, and is not capable of efficiently performing I/O in parallel computing environments.
Comparison of HDF4 and HDF5 Features
(Source: National Center for Supercomputing Applications (NCSA's) HDF vs. HDF5 web page (September 7, 2001). Retrieved May 16, 2003, from http://hdf.ncsa.uiuc.edu/h4vsh5.html)
| HDF | HDF (4.x and previous releases) | |
| Can handle files over 2GB | - | |
| No limit on the number of objects | Max limit is 20000 objects and sometimes less | |
| One flexible data model | Several rigid data models | |
| Supports parallel I/O | Minimal parallel I/O support | |
| Threadsafe (designed to be) | - | |
| Supports multi-dimensional array of compound datatypes | Supports only 1-dimensional array of compound datatypes | |
| Supports variable length datatypes | - | |
| Supports references to objects (?) | - | |
| Supports "mounting" of files | - | |
| Supports hard/soft links -makes delete/move objects easy | - Difficult to do | |
| Can flush data immediately | - | |
| Extensive datatype/datatype conversion support | More restricted | |
| Virtual File layer | - | |
| Performance tuning - lots of things | Minimal | |
| Ability to register your own compression methods | - | |
| Supports multiple unlimited dimensions | Supports 1 unlimited dimension | |
| Supports complex subsetting features | Supports basic subsetting features | |
| True hierarchical file structure | "pseudo-flat" file structure using vgroups | |
| More metadata support | Metadata support | |
| Can convert to/from XML | - | |