2.1.2.2 Hierarchical Data Format 5 (HDF5 )

XML is not very efficient at handling large volumes of numerical or array data, so for this purpose RESQML uses the Hierarchical Data Format, version 5 (HDF5). HDF5 is a data model, a set of open file formats, and libraries designed to store and organize large amounts of data for improved speed and efficiency of data processing. Specifically, HDF5 provides:

• Machine/architecture-independent "binary" format (supported on Windows, Linux, etc. APIs are available in C++, Java, and .NET).
• Built-in data compression.

• Hyper-slabbing of array data so that sub-arrays may be extracted without reading the entire data file.

Applications for HDF5 in earth modeling workflows include storage and retrieval of geometry and property data and multi-million cell models.

• For more information on how RESQML uses HDF5, see 3.2.6.1 HDF5 Files and EPC External Part Reference , Section 6.2.2 Multi-Dimensional Arrays and HDF5 Data Storage , and Chapter 18 Appendix: HDF5 Implementation Overview .
• For more information on HDF, including available tools and tutorials, or to download the libraries, see the HDF Group website at: http://www.hdfgroup.org/HDF5/. The HDFView tool is especially useful for visualizing and understanding the data stored in an HDF5 file.

NOTE: HDF5 implementation on Windows can crash if the file system is full or fails. Before writing HDF5 files, check available space in the file system. The HDF5 group has been engaged, but no workarounds or solutions are available now.