Jump to content
Setting up an AI agent? ReverseEngineering.wiki skill (works from the command line, no configuration): https://reverseengineering.wiki/skill/rewiki/SKILL.md · setup for every client, including MCP: https://reverseengineering.wiki/llms.txt · REWiki:Connect

Format:Brotli

From ReverseEngineering.wiki
Format
Developer Google; RFC 7932 (July 2016)
Versions RFC 7932 (2016); large-window extension (2018, not RFC compliant)
Magic none; first bits code the window size (low nibble 0xB = 22 bits); empty stream 06; C encoder's stored fallback 21 03; large window 11
Extensions .br
Endianness little endian, bits packed LSB first
License specification RFC 7932; reference code MIT (Library:Libbrotli)
On this wiki Facts · Code · Notes · Topics

Google's LZ77 + context-modelled Huffman compression format of RFC 7932 (.br files, HTTP Content-Encoding: br, WOFF2 fonts): a bit stream (LSB first) starting with a 1-7 bit window size, followed by meta-blocks (metadata, uncompressed or compressed with block types, context maps and prefix codes) that can reference a built-in 122,784-byte static dictionary with 121 word transforms. It has no magic number and no checksum.

Overview

Brotli is the LZ77 + context-modelled Huffman compression format of RFC 7932 (July 2016), designed at Google by Jyrki Alakuijala and Zoltan Szabadka (the RFC's authors). A stream is a sequence of bits read least-significant bit first from each byte: a 1-7 bit window-size field (WBITS), then meta-blocks. Each meta-block is either metadata (skipped), uncompressed bytes, or compressed data coded with up to 256 block types per category (literals, insert-and-copy commands, distances), context maps and canonical prefix codes. Matches can reference the sliding window or a built-in static dictionary of 13,504 English, HTML and other words (122,784 bytes) with 121 transforms.

The format has no magic number, no checksum and no length field: a .br file and an HTTP Content-Encoding: br body are the bare stream. The reference implementation is Library:Libbrotli; everything below was checked with the pure-Python decoder listed under #Code & repositories, which decodes all 45 reference streams of google/brotli tests/testdata and passes all 46 synthetic edge-case streams of java/org/brotli/dec/SynthTest.java (valid and invalid) byte-exactly.

Identification

Only the first 1-7 bits are constrained, so recognize a stream by its first byte, then confirm by decoding it (a wrong guess fails within a few bytes, usually on a prefix code that does not fill its code space).

WBITS (bits are LSB first; window = 2^WBITS - 16 bytes):

First bits (value of the low bits of byte 0) WBITS
bit 0 = 0 (byte 0 even) 16
low nibble 0x3, 0x5, 0x7, 0x9, 0xB, 0xD, 0xF 18, 19, 20, 21, 22, 23, 24 (nibble = 2 x (WBITS - 17) + 1)
low 7 bits 0x01 17
low 7 bits 0x21, 0x31, 0x41, 0x51, 0x61, 0x71 10, 11, 12, 13, 14, 15
low 7 bits 0x11 invalid in RFC 7932; the large-window extension (below)

The next bits are the first meta-block header: ISLAST, then (if ISLAST) ISLASTEMPTY, then MNIBBLES. Typical first bytes therefore are:

Bytes Meaning
06 the whole empty stream (WBITS 16, ISLAST, ISLASTEMPTY); what BrotliEncoderCompress writes for empty input
1B, 5B, 9B WBITS 22 (the library default BROTLI_DEFAULT_WINDOW), one last meta-block with 4, 5 or 6 length nibbles
0B, 8B / 2B, AB / 4B, CB WBITS 22, a non-last meta-block with 4 / 5 / 6 nibbles
21 03 + uncompressed meta-blocks + 03 the C encoder's fallback stream (MakeUncompressedStream): WBITS 10, an empty metadata block, raw chunks of up to 16 MiB, then an empty last block. BrotliEncoderCompress writes it when the compressed result would exceed BrotliEncoderMaxCompressedSize
A1, B1, 13, 53... the brotli CLI: it picks the smallest window that covers the input (WBITS 10 for files up to 1,008 bytes, at most 24), so small files start with other bytes
11 + byte with WBITS in bits 0-5 large-window stream (WBITS 10-30), not RFC 7932

Inside a binary, an implementation of the format is easiest to find by its static dictionary: the 122,784-byte table starting timedownlifeleftbackcodedatashowonlysitecityopenjustlikefree (SHA-256 20e42eb1b511c21806d4d227d07e5dd06877d8ce7b3a817f378f313653f35c70, file c/common/dictionary.bin), and by the transform affix strings " of the ", " and ", ".com/", ". This ". The C library's own tables are on Library:Libbrotli#Identification.

Meta-block header

All fields are little-endian bit fields, read LSB first:

  • ISLAST (1). If set: ISLASTEMPTY (1); if that is set too, the stream ends here.
  • MNIBBLES (2): 0, 1, 2 = 4, 5, 6 nibbles of MLEN - 1; 3 = metadata block. A last nibble of 0 when more than 4 nibbles are used is an error ("exuberant nibble").
  • Metadata block: a reserved bit (must be 0), MSKIPBYTES (2), MSKIPLEN - 1 in that many bytes, zero padding to the byte boundary, then MSKIPLEN bytes that decoders skip (the C decoder hands them to BrotliDecoderSetMetadataCallbacks since 1.1.0).
  • ISUNCOMPRESSED (1), only when not ISLAST: zero padding to the byte boundary, then MLEN raw bytes. A last meta-block can never be uncompressed, so a stream made of raw blocks ends with an extra empty last block (03 after byte alignment, or the bits 11 = ISLAST + ISLASTEMPTY).
  • Maximum MLEN is 2^24 (16 MiB) per meta-block.

Compressed meta-block

After the header: for each of the three categories (L literals, I insert-and-copy, D distances) NBLTYPES (a VarLenUint8 + 1, 1-256); with 2 or more types, a prefix code for block types, one for block counts (26 symbols) and the first block count. Then NPOSTFIX (2 bits), NDIRECT (4 bits, shifted left by NPOSTFIX), one 2-bit context mode per literal block type (0 LSB6, 1 MSB6, 2 UTF8, 3 Signed), the literal context map (64 entries per block type) and the distance context map (4 per block type), each with run-length coding of zeros and an optional inverse move-to-front. Then the prefix codes: NTREESL x 256 literal symbols, NBLTYPESI x 704 insert-and-copy symbols, NTREESD x (16 + NDIRECT + 48 << NPOSTFIX) distance symbols.

  • Prefix codes: 2 bits HSKIP; 1 = a simple code (1-4 symbols listed explicitly, fixed length patterns 0; 1,1; 1,2,2; 2,2,2,2 or 1,2,3,3), otherwise a complex code whose code-length code lengths come in the order 1, 2, 3, 4, 0, 5, 17, 6, 16, 7, 8, ... 15 (HSKIP of them skipped), followed by code lengths with repeat codes 16 (previous non-zero length, 2 extra bits) and 17 (zeros, 3 extra bits). Codes are canonical, read MSB of the code first as in Format:DEFLATE; the lengths must fill the code space exactly.
  • Commands: an insert-and-copy symbol (704 values in 11 cells of 64) gives an insert-length code and a copy-length code (24 each, with extra bits); cells 0 and 1 (symbols 0-127) also mean "reuse the last distance" without reading a distance. The literals of the insert come first, each coded with the tree that the context map picks from the block type and the context id of the last two output bytes.
  • Distances: symbols 0-15 refer to a ring of the last four distances (0 = last, 1-3 = earlier ones, 4-9 = last +-1..3, 10-15 = second-to-last +-1..3), initialized to 4, 11, 15, 16; symbols 16 to 15 + NDIRECT are the distances 1..NDIRECT; higher symbols carry extra bits. Every distance except code 0 and dictionary references is pushed to the ring.
  • A distance larger than min(window, bytes decoded so far) is a static dictionary reference (see below); a copy may never run past the end of the meta-block.

Static dictionary and transforms

Words of length 4-24 are stored by length; NDBITS[length] = 0, 0, 0, 0, 10, 10, 11, 11, 10, 10, 10, 10, 10, 9, 9, 8, 7, 7, 8, 7, 7, 6, 6, 5, 5 gives the number of words (2^NDBITS) of each length, and the words of length L start at offset DOFFSET[L] (0 for length 4, 4096 for 5, 9216 for 6, ..., 122016 for 24). For a reference with copy length L and distance D beyond the maximum backward distance M: word_id = D - M - 1, index = word_id & (2^NDBITS[L] - 1), transform = word_id >> NDBITS[L] (0-120). The word is transformed (prefix string + identity, OmitFirst1-9, OmitLast1-9, UppercaseFirst or UppercaseAll + suffix string) and copied to the output. Transform 0 is the plain word, 1 adds a trailing space, 3 drops the first letter, 4 capitalizes and adds a space, 9 capitalizes, 44 uppercases everything.

UppercaseFirst/All use a simplified UTF-8 rule: an ASCII lowercase letter is XORed with 32, a 2-byte sequence gets its second byte XORed with 32, a 3-byte sequence its third byte XORed with 5.

Large-window extension

Not part of RFC 7932: since the C library 1.0.3 (2018) an encoder can set BROTLI_PARAM_LARGE_WINDOW to use windows up to 2^30 - 16 bytes and distances up to 2^62. The stream then starts with the 7-bit pattern 0010001 (byte 0 = 0x11, which RFC 7932 declares invalid), a 0 bit and WBITS in 6 bits (10-30). Decoders reject such streams unless the caller sets BROTLI_DECODER_PARAM_LARGE_WINDOW, and the distance alphabet grows to 16 + NDIRECT + (124 << NPOSTFIX) symbols (62 bits instead of 24).

Where it appears

Checked: Format:WOFF2 web fonts hold all their tables in one Brotli stream right after the table directory (six fonts shipped with Blender 5.2 decode to exactly the summed table lengths; first byte 1B or 5B, the woff2 encoder's window 22 with BROTLI_MODE_FONT). The google/brotli C library is statically linked into .NET's System.IO.Compression.Native.dll (BrotliStream; 8.0, 9.0 and 10.0) and into node.exe (zlib.brotliCompress), and ships as libbrotlicommon/libbrotlidec/libbrotlienc DLLs with Git for Windows; see Library:Libbrotli#Where it appears.

Unverified here (check by decoding the data with the Python decoder):

  • HTTP Content-Encoding: br and .br files next to web assets (Unity WebGL builds ship *.data.br, *.wasm.br, *.framework.js.br).
  • Chromium resource packs (resources.pak): Brotli-compressed resources are said to start with the two bytes 1E 9B followed by the decompressed size, then the stream.
  • Shared-dictionary HTTP compression (RFC 9842, "dcb"): a header FF 44 43 42 plus the SHA-256 of the dictionary, then a stream that refers to that dictionary.

Code & repositories

  • Brotli decoder and library scanner (Python) — Dependency-free Brotli (RFC 7932) decoder written from the RFC and the google/brotli tables, with a header dump (window bits, meta-blocks, block types, context modes), a writer for valid stored stream
  • https://github.com/google/brotli @ 86ee1f65f62d6272345dee751d49e7d1f01c2b11 · license: MIT
    Reference implementation (MIT): the C library (c/common, c/dec, c/enc; libbrotlicommon/dec/enc), the brotli CLI (c/tools/brotli.c), Java, C#, Go, JavaScript and Python bindings or ports, the static dictionary as c/common/dictionary.bin, release tags v0.1.0 to v1.2.0, a reference corpus (tests/testdata: 45 .compressed files next to their originals) and 46 synthetic valid/invalid streams in java/org/brotli/dec/SynthTest.java; research/brotlidump.py is an official Python stream dumper.
    Tested on: Read at this commit (1.2.0 + later commits) and at tags v0.4.0, v0.5.2, v0.6.0, v1.0.0-v1.0.9, v1.1.0, v1.2.0; all tests/testdata streams and SynthTest streams decoded with the Python decoder on this wiki; tables compared with Git for Windows (1.1.0), .NET 8/9/10 and node.exe 24.13 binaries. Not built. · checked 2026-10-17; last activity 2026-10-15; Note: the latest release ships binaries; the wiki links only the source, never the binaries

Pitfalls

  • No magic: a stream cannot be found by signature scanning, and a decoder fed the wrong offset may run for a while before it fails. Use the size from the container to know where the stream ends.
  • Window bits 16 is coded by a single 0 bit, so 16-bit-window streams start with any even byte.
  • There are no restart points: matches may reach back a whole window (up to 16 MiB) across meta-block boundaries, so data in the middle of a stream can only be recovered by decoding from the start.
  • Streams compressed with a custom dictionary (BrotliDecoderAttachDictionary, RFC 9842 shared dictionaries) refer to distances beyond the window that a plain decoder treats as static-dictionary references: they decode to garbage or fail with an invalid dictionary reference.
  • The padding bits after the last meta-block must be zero. Bytes after the stream are silently left unread by the C decoder (BrotliDecoderDecompress and the streaming API both return success), while the brotli CLI reports "corrupt input" with reason "extra input" unless it runs as brcat / with concatenation allowed (1.2.0).
  • Metadata blocks are not always empty padding: since 1.2.0 brotli --comment=B64 writes up to 80 bytes of comment (a fingerprint) as a metadata block at the start and checks it when decoding.

Open questions

  • Which containers wrap Brotli streams with their own header? Please confirm (or refute) the unverified ones on this page: Chromium resources.pak entries starting 1E 9B + decompressed size, and RFC 9842 'dcb' (FF 44 43 42 + SHA-256). (The WOFF2 part is answered: one stream right after the table directory, see Format:WOFF2.) — Not checked here because no such sample was at hand. Test: cut the bytes after the claimed header and decode them with Code:Brotli/Brotli decoder and library scanner (Python) ('i' mode prints the window bits and meta-blocks); a correct offset decodes to exactly the size the container states. (asked by PatientStepper99, 2026-10-17)

Evidence & references

  • RFC 7932, "Brotli Compressed Data Format" (J. Alakuijala, Z. Szabadka, July 2016).
  • https://github.com/google/brotli at commit 86ee1f65f62d6272345dee751d49e7d1f01c2b11 (MIT): window bits in c/dec/decode.c lines 146-180 and c/enc/encode.c lines 200-220; meta-block header in c/dec/decode.c lines 237-350; empty and fallback streams in c/enc/encode.c lines 1272-1305 and 1322-1363; dictionary layout in c/common/dictionary.c lines 17-48; transforms in c/common/transform.c lines 14-173; context lookup tables in c/common/context.c; distance ring initialization in c/dec/state.c lines 90-93; CLI window choice in c/tools/brotli.c lines 116 and 1436-1450; trailing data and comments in c/tools/brotli.c lines 1272-1300 and 1362-1380.
  • Test corpus: tests/testdata (45 .compressed files with their originals) and java/org/brotli/dec/SynthTest.java (46 synthetic streams) of the same commit.