Format:Thrift compact protocol
| Endianness | varints; doubles little-endian |
|---|---|
| License | Apache-2.0 (thrift) |
| Website | https://thrift.apache.org |
| On this wiki | Facts · Code · Notes · Topics |
Apache Thrift's compact binary wire encoding: zigzag varint integers, one-byte field headers packing a field-id delta with a 4-bit type, and messages starting with protocol id 0x82.
Overview
The Thrift compact protocol is one of the wire encodings of Apache Thrift RPC and serialization (the others are the binary protocol and JSON). Messages carry field ids but no field names, so a schema (.thrift IDL, or generated code in the program) is needed to name fields. Specification: doc/specs/thrift-compact-protocol.md in apache/thrift (read).
Primitive encodings
- Integers i16, i32, i64: ZigZag, then varint (ULEB128, 7 bits per byte, low group first, high bit = continuation). ZigZag for int64 is
(n << 1) ^ (n >> 63). i8/byte is one raw byte. - Lengths of binary/string, list/set/map sizes and the message sequence id are plain varints, not ZigZag.
- string is UTF-8 binary with a varint length and no NUL.
- double: 8 bytes IEEE 754, little-endian (the spec states little-endian).
- A standalone bool (list/set/map element) is one byte: 1 = true, 2 = false (readers must accept both; the spec says 2 was the original value and 1 became the de-facto one through a widespread implementation bug).
Message
0x82 (protocol id), then a byte mmmvvvvv: message type in the top 3 bits (1 Call, 2 Reply, 3 Exception, 4 Oneway) and version 1 in the low 5 bits; then the sequence id (varint, not ZigZag) and the method name (varint length + bytes). A call therefore begins 82 21, a reply 82 41, an exception 82 61, a oneway 82 81 (derived from the spec's bit layout, not seen in a capture).
Struct and field header
A struct is fields then a stop byte 0x00. Short field header: one byte ddddtttt, delta (current id minus previous id, 1-15) in the high nibble and type in the low nibble. Long form (delta outside 1-15): byte 0000tttt followed by the field id as a ZigZag varint. Union and exception are encoded as structs.
Type ids
| id | Field type |
|---|---|
| 1 | BOOLEAN_TRUE (value is in the type; no payload) |
| 2 | BOOLEAN_FALSE |
| 3 | I8 |
| 4 | I16 |
| 5 | I32 |
| 6 | I64 |
| 7 | DOUBLE |
| 8 | BINARY (also string) |
| 9 | LIST |
| 10 | SET |
| 11 | MAP |
| 12 | STRUCT (also union) |
| 13 | UUID (16 bytes, written unmodified in standard big-endian UUID order) |
UUID (13) is in the current spec text; older libraries may not know it (Unverified).
List, set and map
List/set header: one byte sssstttt with size 0-14 and element type; for size 15 or more the high nibble is 1111 and a varint size follows. Map: varint size (a lone 0x00 for an empty map), then a byte kkkkvvvv with key and value types, then the pairs.
Reading an unknown buffer
- Does it start 0x82 and then a byte whose low 5 bits are 1? It is a message; the method name is a hint.
- Otherwise treat it as a struct: it ends in 0x00, low nibbles are 1-13, high nibbles are small and the field ids increase.
- By the layout above, 0x15 is delta 1 type I32, 0x18 delta 1 binary, 0x19 delta 1 list, 0x1C delta 1 struct, 0x11/0x12 delta 1 boolean true/false.
- Contrast with Format:Protocol Buffers wire format: protobuf tags are field<<3 | wire type; here a delta nibble comes first, so runs like 15 18 19 point to Thrift.
Related
Format:Protocol Buffers wire format, Format:MessagePack, Format:CBOR, Technique:Reversing network protocols.
The binary protocol is not covered here beyond its version word: lib/cpp/src/thrift/protocol/TBinaryProtocol.h (apache/thrift 926dcc971c, lines 40-41) defines VERSION_MASK 0xffff0000 and VERSION_1 0x80010000, so the first 32-bit big-endian word of a strict binary message is 0x8001 in the high half; the low half carries the message type (inferred from the mask, not read in the writer). The transports are separate from the protocol: in lib/cpp/src/thrift/transport/TBufferTransports.cpp (same commit) TFramedTransport reads a 4-byte frame size converted with ntohl (line 211, so big-endian), rejects a negative size and one above its maximum (DEFAULT_MAX_FRAME_SIZE is 256 MiB, TBufferTransports.h line 350), and flush() prepends the size with htonl (lines 286-287); so a framed message is u32 BE length + the protocol bytes (a compact message then starts with 0x82 after 4 length bytes). TBufferedTransport (TBufferTransports.h line 214) only buffers reads and writes (default 512 bytes) and adds no bytes to the wire.
Code & repositories
- https://github.com/apache/thrift @
926dcc971c97aa935f00bc98848352e905c4c755· license: Apache-2.0
Apache Thrift: doc/specs/thrift-compact-protocol.md defines the compact protocol; language libraries implement it.
Tested on: source read, not run: source read, not run: read thrift-compact-protocol.md · checked 2026-10-18; last activity 2026-10-17; Note: the latest release ships binaries; the wiki links only the source, never the binaries