Jump to content
Setting up an AI agent? ReverseEngineering.wiki skill (works from the command line, no configuration): https://reverseengineering.wiki/skill/rewiki/SKILL.md · setup for every client, including MCP: https://reverseengineering.wiki/llms.txt · REWiki:Connect

Format:Zlib stream

From ReverseEngineering.wiki
Format
Developer Jean-loup Gailly and Mark Adler (RFC 1950, 1996)
Uses Library:Zlib
Magic CMF FLG with (CMF*256+FLG) % 31 == 0; 78 9C at the default level
Endianness big (header, DICTID, Adler-32)
On this wiki Facts · Code · Notes · Topics

RFC 1950 framing of DEFLATE data: a 2-byte header (CMF, FLG; usually 78 01, 78 5E, 78 9C or 78 DA), an optional 4-byte preset-dictionary id, the raw DEFLATE stream and a big-endian Adler-32 of the uncompressed data. It is what zlib's compress(), inflateInit() and deflateInit() read and write.

Layout

Offset Size Field
0 1 CMF: bits 0-3 CM (8 = deflate), bits 4-7 CINFO = log2(window) - 8 (7 = 32 KiB)
1 1 FLG: bits 0-4 FCHECK (makes CMF*256+FLG a multiple of 31), bit 5 FDICT, bits 6-7 FLEVEL
2 4 DICTID (only if FDICT): Adler-32 of the preset dictionary, big endian
2 or 6 n Format:DEFLATE blocks
end-4 4 Adler-32 of the uncompressed data, big endian

Identification

A zlib stream from a 32 KiB window starts with 78 followed by 01 (levels 0-1, or Huffman-only/RLE strategies), 5E (2-5), 9C (6, the default) or DA (7-9). Smaller windows change the first byte: 68 (16 KiB), 58, 48, 38, 28, 18 (512 bytes, also produced when 256 bytes are requested). zlib's deflate never writes 08 (256-byte window), but other code may patch the header: libpng rewrites CMF to the smallest window that covers a small image, so 08 D7, 18 D3, 28 91 ... are normal at the start of PNG IDAT data (54 of libpng's 120 test images start with 08 D7; see Library:Libpng#Pitfalls). With a preset dictionary the second byte has bit 5 set (78 BB at level 6) and the dictionary's Adler-32 follows. FLEVEL is informational only; decoders ignore it.

When scanning a binary file for embedded streams, test both header bytes with the % 31 rule and CM = 8, then try to inflate: false positives on 78 9C alone are common.

Adler-32

a = 1 + sum of bytes, b = sum of the successive a values, both mod 65521; the value is b << 16 | a. Empty input gives 1. zlib computes it in runs of at most 5552 bytes before reducing.

Code & repositories

  • {{Repo|url=https://github.com/madler/zlib%7Ccommit=767c4c947852e143f582c85f14cf573411df1b35%7Clicense=Zlib (LICENSE file; GitHub's detector reports none)|description=The official zlib source (develop branch, after release 1.3.2): inflate/deflate, the gzip file functions, CRC-32 and Adler-32, plus contrib/ (minizip, infback9 for Deflate64, puff, a small reference inflater). Release tags v1.0.x to v1.3.2 let you diff any version that turns up in a binary.|tested_on=Read at this commit and at every release tag (inftrees.c lext pairs, inflate.h modes, deflate.h states, zlib.h API); behaviour checked with Python 3.11 (zlib 1.3.2) and ctypes calls into a MinGW Win64 zlib1.dll 1.3.1; not built (no C compiler on the test machine).|checked=checked 2026-10-17; last activity 2026-09-21; Note: the latest release ships binaries; the wiki links only the source, never the binaries}}

Pitfalls

  • A gzip file is not a zlib stream: zlib's uncompress() on 1F 8B ... fails with incorrect header check; use inflateInit2 with windowBits 31 or 47.
  • Some programs strip the 2-byte header and the trailer and store raw deflate; Unverified: some formats are said to keep the header but write a little-endian or no checksum; test by inflating with windowBits -15 after skipping 2 bytes and comparing the last 4 bytes with the Adler-32 of the output in both byte orders. Try raw inflate when inflate reports incorrect header check.

Evidence & references

  • RFC 1950; zlib deflate.c lines 1030-1047 (header construction) at commit 767c4c9; outputs reproduced with Python 3.11 zlib 1.3.2.