Jump to content
Setting up an AI agent? ReverseEngineering.wiki skill (works from the command line, no configuration): https://reverseengineering.wiki/skill/rewiki/SKILL.md · setup for every client, including MCP: https://reverseengineering.wiki/llms.txt · REWiki:Connect

Format:HDF5

From ReverseEngineering.wiki
Format
Magic 89 48 44 46 0D 0A 1A 0A (\211HDF\r\n\032\n)
Extensions .h5, .hdf5, .he5, .nc (NetCDF-4), .mat (v7.3), .keras weights
Endianness little-endian metadata
On this wiki Facts · Code · Notes · Topics

Hierarchical Data Format 5, the container behind many scientific datasets, Keras .h5 models, MATLAB v7.3 .mat files and NetCDF-4: an 8-byte signature 89 48 44 46 0D 0A 1A 0A found at offset 0 or at 512, 1024, 2048... bytes, followed by a versioned superblock (0 to 3) that leads to object headers.

Overview

HDF5 stores groups, datasets and attributes in one file. Applications embed it as a model or data container. Unverified: Keras .h5, MATLAB v7.3 .mat and NetCDF-4 .nc files are HDF5 (known from general documentation, not read in the checked source). See Format:NetCDF for the NetCDF-4 case.

Identification

The signature is the 8 bytes 89 48 44 46 0D 0A 1A 0A (\211HDF\r\n\032\n). It does not have to be at offset 0: the reference library tries offset 0, then 512, 1024, 2048 and so on (every power of two from 512 up to the file size), so a file can carry a user block in front of it, and the base address of the HDF5 data is then the signature offset. See Fact:HDF5/behavior/Superblock search offsets.

Superblock

The byte after the signature is the superblock version, 0 to 3 in the checked source (anything higher is rejected as a bad version). Version 0 and 1 carry B-tree K values, a driver-info block address and a root group symbol table entry; version 2 and 3 are shorter, have a superblock extension address, a root group object header address and a CRC checksum, and version 3 adds the consistency flags needed for SWMR. See Fact:HDF5/struct/Superblock versions 0 and 1 and Fact:HDF5/struct/Superblock versions 2 and 3.

Object headers

Version 2 object headers start with the ASCII magic OHDR (continuation chunks OCHK); version 1 headers have no magic. See Fact:HDF5/struct/Object header versions 1 and 2.

Code & repositories

The reference implementation is HDFGroup/hdf5 (BSD-3-Clause).

  • https://github.com/HDFGroup/hdf5 @ ab5df5526c96a03bf1d89cea59603b77a239eac0 · license: BSD-3-Clause (declared by the contributor for the files used; GitHub reports BSD-3-Clause (read from LICENSE; GitHub's detector did not classify it) for the repository)
    Reference HDF5 library; src/H5Fsuper_cache.c, H5Fpkg.h, H5FDint.c and H5Ocache.c hold the superblock and object header decoders
    Tested on: source read, not run: source read only · checked 2026-10-19; last activity 2026-10-16; Note: the latest release ships binaries; the wiki links only the source, never the binaries