Format:GGUF
| Developer | ggml-org |
|---|---|
| Versions | 3 (GGUF_VERSION) |
| Magic | GGUF (47 47 55 46) |
| Extensions | .gguf |
| Endianness | little by default; big-endian files exist (docs/gguf.md) but nothing in the header says which |
| License | MIT |
| On this wiki | Facts · Code · Notes · Topics |
ggml/llama.cpp's single-file model container: magic 'GGUF', a uint32 version (3), tensor and key-value counts, typed metadata, tensor infos and an aligned tensor data blob.
Layout
Per the comment in ggml's gguf.h, a file is: the magic "GGUF" (4 bytes); the version (uint32); the tensor count (int64); the key-value count (int64); then each KV pair (key string, value type, value; arrays carry an element type and a uint64 count); then one info record per tensor (name string, uint32 n_dims, int64 size per dimension, ggml_type as int32, uint64 offset into the data blob); then the tensor data blob, padded to the alignment. Strings are a uint64 length followed by the bytes without a terminator; enums are int32; bool is one byte.
The alignment is the uint32 KV general.alignment if present, else 32 (GGUF_DEFAULT_ALIGNMENT). Tensor offsets are relative to the start of the data blob, which is padded to that alignment after the tensor infos.
Value types
UINT8 0, INT8 1, UINT16 2, INT16 3, UINT32 4, INT32 5, FLOAT32 6, BOOL 7, STRING 8, ARRAY 9, UINT64 10, INT64 11, FLOAT64 12 (see the fact on the enum).
Metadata keys
The ggml docs/gguf.md defines the naming convention for files ([Sidecar-]BaseName-SizeLabel[-FineTune]-Version[-Encoding][-Type][-Shard].gguf, sidecars mmproj and mtp, Type LoRA or vocab, shards like 00003-of-00009) and the general.* keys such as general.basename, general.size_label, general.finetune and general.version. docs/gguf.md (ggml a05c992) defines general.architecture (string, lowercase [a-z0-9]+, line 373) as the model architecture name used as key prefix, general.alignment (uint32, must be a multiple of 8, default 32 when absent, line 384), and general.quantization_version (uint32, required whenever any tensor is quantized, line 383). Tokenizer keys use the tokenizer.ggml.* prefix (for example tokenizer.ggml.model, line 680).
Identification
Check the first four bytes for GGUF, then the version. Quantisation type is per tensor (the ggml_type field), so a file can mix types. See Technique:Identifying machine-learning models in applications. The format succeeds the older GGML, GGMF and GGJT files (stated in docs/gguf.md).
Related
Format:safetensors, Format:ONNX model
Tensor types
The ggml_type field of each tensor info is an int32 from ggml.h enum ggml_type (ggml a05c992, lines 389-434): F32 0, F16 1, Q4_0 2, Q4_1 3, (4 and 5 removed: Q4_2, Q4_3), Q5_0 6, Q5_1 7, Q8_0 8, Q8_1 9, Q2_K 10, Q3_K 11, Q4_K 12, Q5_K 13, Q6_K 14, Q8_K 15, IQ2_XXS 16, IQ2_XS 17, IQ3_XXS 18, IQ1_S 19, IQ4_NL 20, IQ3_S 21, IQ2_S 22, IQ4_XS 23, I8 24, I16 25, I32 26, I64 27, F64 28, IQ1_M 29, BF16 30, (31-33 removed: Q4_0_4_4, Q4_0_4_8, Q4_0_8_8), TQ1_0 34, TQ2_0 35, (36-38 removed), MXFP4 39, NVFP4 40, Q1_0 41, Q2_0 42, COUNT 43. Dimensions are stored in ggml order: the first dimension is the fastest-varying one (ne[0]; strides nb[0] = type size, in ggml.h around line 690). A quantized type packs blocks of elements, so the first dimension must be a multiple of the type's block size. Unverified: block sizes and byte layouts of each quantization type are defined in ggml-common.h (block_q4_0 and similar structs); check there before decoding raw data.
Older GGML, GGMF and GGJT files
docs/gguf.md (section Historical State of Affairs, lines 786-806) names three predecessors: GGML (unversioned, no alignment), GGMF (the same with one version number) and GGJT (tensors aligned for mmap; v1, v2 and v3 have the same layout but v3 uses an incompatible quantization scheme). They share a structure: a magic number with an optional version, untyped hyperparameters (including an ftype field; for GGML files the quantization version is ftype divided by 1000), an embedded vocabulary of length-prefixed strings (GGMF/GGJT add a float32 score per token), then tensors with length-prefixed name, type and data. Nothing in them names the architecture, which is why GGUF added key-value metadata. Unverified: the magic values are the ASCII tags 'ggml' (0x67676d6c), 'ggmf' (0x67676d66), 'ggjt' (0x67676a74) and 'ggla' (LoRA adapters) stored as little-endian uint32, so on disk the bytes read reversed (for example 'tjgg'); the docs read here do not print them.
Code & repositories
- https://github.com/ggml-org/ggml @
a05c9922613c6a2230dbb1d0de679519ecafb4f0· license: MIT (read from LICENSE; GitHub's API was busy)
ggml tensor library; include/gguf.h and docs/gguf.md define the GGUF container
Tested on: source read, not run: read source · checked 2026-10-18