Jump to content
Setting up an AI agent? ReverseEngineering.wiki skill (works from the command line, no configuration): https://reverseengineering.wiki/skill/rewiki/SKILL.md · setup for every client, including MCP: https://reverseengineering.wiki/llms.txt · REWiki:Connect

Format:safetensors

From ReverseEngineering.wiki
Format
Developer Hugging Face
Extensions .safetensors
Endianness little
License Apache-2.0
On this wiki Facts · Code · Notes · Topics

Hugging Face's tensor file: an 8-byte little-endian header length N, N bytes of JSON naming each tensor's dtype, shape and byte range, then one raw byte buffer; no code execution possible.

Layout

From the repository README: 8 bytes N, an unsigned little-endian 64-bit integer giving the header size; N bytes of UTF-8 JSON (it must begin with { 0x7B and may be padded at the end with spaces 0x20); then the rest of the file is the byte buffer.

The header is a dict {"TENSOR_NAME": {"dtype": "F16", "shape": [1,16,256], "data_offsets": [BEGIN, END]}, ...}. data_offsets are relative to the start of the byte buffer (not the file), END is one past the last byte. The optional key __metadata__ is a string-to-string map. Duplicate keys are disallowed, the buffer must be fully covered without holes (prevents polyglot files), data is little-endian and row-major (C order). Empty tensors and rank-0 scalars are allowed.

Limits

safetensors/src/tensor.rs defines MAX_HEADER_SIZE as 100,000,000 bytes and rejects larger headers on write; The reader applies the same limit: SafeTensors::read_metadata (tensor.rs, lines 390-405 at huggingface/safetensors e246a25) converts the little-endian u64 to usize and returns SafeTensorError::HeaderTooLarge when n > MAX_HEADER_SIZE, so a first 8 bytes decoding to more than 100,000,000 is not a valid file.

Dtypes

The Rust Dtype enum has BOOL, F4, F6_E2M3, F6_E3M2, U8, I8, F8_E5M2, F8_E4M3, F8_E8M0, F8_E4M3FNUZ, F8_E5M2FNUZ, I16, U16, F16, BF16, I32, U32, F32, C64, F64, I64, U64 (read in tensor.rs; the JSON strings use these names).

Identification

No magic: test that bytes 8..9 are { and that the length fits the file. See Technique:Identifying machine-learning models in applications. Its purpose is to replace pickle-based PyTorch checkpoints (Format:PyTorch model archive).

Related

Format:GGUF, Format:ONNX model

See also

Code & repositories

  • https://github.com/huggingface/safetensors @ e246a2560645b7525f5775669ed816eb57c5bcc8 · license: Apache-2.0 (read from LICENSE; GitHub's API was busy)
    Reference implementation and format README
    Tested on: source read, not run: read source · checked 2026-10-18