Jump to content
Setting up an AI agent? ReverseEngineering.wiki skill (works from the command line, no configuration): https://reverseengineering.wiki/skill/rewiki/SKILL.md · setup for every client, including MCP: https://reverseengineering.wiki/llms.txt · REWiki:Connect

Format:LZMA

From ReverseEngineering.wiki
Format
Developer Igor Pavlov (LZMA SDK, 1998 onwards); .lzma format documented by Lasse Collin
Uses Library:Liblzma
Magic none; .lzma usually starts 5D 00 00 xx xx (properties 0x5D, little-endian dictionary size)
Extensions .lzma, .tlz (tar.lzma)
Endianness little (.lzma header); LZMA2 chunk sizes big endian
On this wiki Facts · Code · Notes · Topics

Igor Pavlov's LZ77 + range-coder compression (LZMA SDK, 7-Zip) as stored in files: the legacy 13-byte .lzma (LZMA_Alone) header (properties byte (pb*5+lp)*9+lc, usually 5D; 32-bit dictionary size; 64-bit uncompressed size or eight FF bytes), raw LZMA streams whose first byte is always 00, and LZMA2, the chunked framing used inside .xz and 7z.

Overview

LZMA (Lempel-Ziv-Markov chain algorithm) is an LZ77 match finder with an adaptive binary range coder; LZMA2 wraps LZMA in chunks that can also be stored uncompressed and that reset state or dictionary. The bit stream is defined by the LZMA SDK's code, not by a document; the containers around it are documented: Format:XZ (LZMA2 only), the .lzma file (below), Format:Lzip (LZMA with fixed parameters) and 7z archives. Many game, firmware and installer formats store a raw LZMA stream behind a home-made header. Decoders: Library:Liblzma, the LZMA SDK and 7-Zip (C, public domain SDK), XZ Embedded (Linux kernel), XZ for Java.

Properties (lc, lp, pb)

One byte encodes the three model parameters: props = (pb * 5 + lp) * 9 + lc, with lc 0-8 (literal context bits), lp 0-4 (literal position bits) and pb 0-4 (position bits), so valid values are 0x00-0xE0. The universal default 3/0/2 gives 0x5D; 0/0/0 gives 0x00, 8/4/4 gives 0xE0. Decode with pb = p / 45; lp = (p % 45) / 9; lc = p % 9. liblzma (so xz) also requires lc + lp <= 4 and rejects other files; the LZMA SDK decodes any combination.

.lzma file (LZMA_Alone)

Offset Size Field
0 1 properties byte (nearly always 0x5D)
1 4 dictionary size, u32 LE (LZMA Utils wrote only 2^n with n 16-25; liblzma writes 2^n or 2^n + 2^(n-1), rounding other values up)
5 8 uncompressed size, u64 LE; FF FF FF FF FF FF FF FF = unknown, the data then ends with an end-of-payload marker
13 n LZMA data, first byte 00

lzma.compress(b"hello\n", format=lzma.FORMAT_ALONE) (Python, preset 6) gives the 29 bytes 5D 00 00 80 00 FF FF FF FF FF FF FF FF 00 34 19 49 EE 8D DD 3D 3A DF FF FF DD 12 00 00. There is no magic number; recognise the header by a properties byte of at most 0xE0 (nearly always 0x5D), a dictionary size that is a power of two or 1.5 times one, an uncompressed size that is either all FF or plausibly small (XZ Utils refuses known sizes of 256 GiB or more for exactly this reason) and a 00 at offset 13. .lzma files cannot be concatenated: LZMA Utils and the SDK ignore anything after the first stream, XZ Utils reports it as corruption.

Raw LZMA streams

The range decoder starts by reading 5 bytes, and the first of them is always 00 (liblzma returns LZMA_DATA_ERROR otherwise). Every raw LZMA stream, and every LZMA chunk of LZMA2, therefore starts with a 00 byte: a cheap test when guessing where an embedded stream begins. A stream ends either after a size known from elsewhere or at an end-of-payload marker (EOPM, the SDK's "end marker"). liblzma's raw LZMA1 encoder always writes the marker; its LZMA1EXT filter (5.4.0+) can omit it.

Home-made containers usually store the 5 SDK property bytes (properties byte + u32 dictionary size, LZMA_PROPS_SIZE in the SDK) before the data, sometimes followed by a 4- or 8-byte uncompressed size, or nothing when the parameters are fixed in code. Look for 5D followed by a little-endian power of two and then a 00 byte. Example: an LZMA sector of Format:MPQ (MoPaQ)#File data is the method byte 12, a 00 filter byte, then exactly the 13-byte .lzma header and the stream (StormLib).

LZMA2

An LZMA2 stream is a sequence of chunks ended by a 00 byte. The first byte of each chunk is a control byte:

Control Chunk Bytes after the control byte
0x00 end of stream none
0x01 uncompressed, dictionary reset 2: size - 1 (u16 big endian), then the data
0x02 uncompressed, no reset same
0x03-0x7F invalid
0x80-0x9F LZMA, no reset 4: uncompressed size - 1 (bits 0-4 of the control byte are bits 16-20, then u16 BE), compressed size - 1 (u16 BE)
0xA0-0xBF LZMA, state reset same
0xC0-0xDF LZMA, state reset + new properties same, then 1 properties byte (lc/lp/pb)
0xE0-0xFF LZMA, state reset + new properties + dictionary reset same, then 1 properties byte

A chunk holds at most 2 MiB of uncompressed and 64 KiB of compressed data, and the first chunk must reset the dictionary (0x01 or 0xE0-0xFF). The dictionary size is not in the stream: it is the 1-byte LZMA2 filter property of the container (Format:XZ#Filters). Each LZMA chunk restarts the range coder, so its data begins with 00. Seen from liblzma: hello\n became 01 00 05 68 65 6C 6C 6F 0A 00 (one uncompressed chunk, end marker); 4.4 MB of text became LZMA chunks with control bytes 0xFF (dictionary reset, properties 0x5D), 0x9F and 0x83 (continuations of 2 MiB each); 70000 random bytes became uncompressed chunks 0x01 (60556 bytes) and 0x02 (9444 bytes).

Related containers

  • Format:XZ: LZMA2 behind optional filters, with checks and an Index.
  • Format:Lzip: "LZIP", version, coded dictionary size, LZMA with lc=3 lp=0 pb=2 and end marker, then CRC32 and sizes.
  • MicroLZMA (EROFS; liblzma 5.4.0+): a raw LZMA stream without end marker whose first byte (always 00) is replaced by the bitwise NOT of the properties byte, so a 3/0/2 stream starts with A2.
  • 7z archives: Unverified here (from the 7-Zip method list, not tested): LZMA is method id 03 01 01 with the 5 property bytes, LZMA2 is 21 with the 1-byte dictionary property.

Code & repositories

  • https://github.com/tukaani-project/xz @ 299eb5c3f597f107fd6019bf156885ac8bfaa5ea · license: 0BSD AND LGPL-2.1-or-later AND GPL-2.0-or-later (declared by the contributor; GitHub's detector did not recognize COPYING)
    Official XZ Utils repository: liblzma (src/liblzma, 0BSD), the xz tools, the .xz and .lzma format specifications (doc/xz-file-format.txt, doc/lzma-file-format.txt, public domain) and the test files in tests/files. Release tags v5.0.0 onwards let you diff the tables of any version found in a binary. Per COPYING: liblzma and the tools are 0BSD, the getopt fallback LGPL-2.1+, the xzgrep/xzdiff scripts GPL-2.0+.
    Tested on: Read at this commit and at tags v5.0.0, v5.2.5, v5.4.0, v5.6.0 (filter_common.c, price_table.c, fastpos_table.c, crc64_table_*.h, lzma2_decoder.c, lzip_decoder.c, liblzma_linux.map); behaviour checked with Python 3.11 lzma and ctypes calls into a MinGW-w64 liblzma-5.dll 5.6.3; not built. · checked 2026-10-17; last activity 2026-10-16; Note: the latest release ships binaries; the wiki links only the source, never the binaries

Pitfalls

  • The .lzma header's uncompressed size is a hint for allocation only when known; with FF x 8 a decoder must find the end marker. liblzma 5.2.5 and older reject .lzma files that have both a known size and an end marker.
  • lc + lp > 4 is legal LZMA but not decodable by xz/liblzma; use the LZMA SDK for such streams.

Evidence & references

  • doc/lzma-file-format.txt (lines 71-150) and src/liblzma/lzma/lzma2_decoder.c, lzma_decoder.c, rangecoder/range_decoder.h in https://github.com/tukaani-project/xz at commit 299eb5c3f597f107fd6019bf156885ac8bfaa5ea; NEWS (5.4.0) for MicroLZMA.
  • Samples made with Python 3.11's lzma (bundling liblzma 5.4.x) and parsed with Code:XZ/xzinfo (Python).