Format:Protocol Buffers wire format
| On this wiki | Facts · Code · Notes · Topics |
|---|
Google's binary serialization for .proto messages: a stream of (tag varint = field_number<<3 | wire_type, value) pairs with six wire types (varint, 64-bit, length-delimited, start/end group, 32-bit). Messages carry no field names or message types, so recovering a schema means finding the descriptor embedded in the program.
Overview
A serialized protobuf message is a flat sequence of fields. Each field starts with a tag, a base-128 varint (7 bits per byte, low group first, high bit = continuation) equal to (field_number << 3) | wire_type. Unknown fields are skipped by wire type, so old readers accept new messages. There is no message length, type name or field name on the wire; a top-level message ends where the buffer ends. Schema recovery therefore means finding the compiled-in descriptor: see Library:Protocol Buffers.
Wire types
| Value | Name | Payload | Used for |
|---|---|---|---|
| 0 | VARINT | base-128 varint | int32, int64, uint32, uint64, sint32, sint64 (ZigZag), bool, enum |
| 1 | I64 (fixed64) | 8 bytes little-endian | fixed64, sfixed64, double |
| 2 | LEN (length-delimited) | varint length, then bytes | string, bytes, embedded messages, packed repeated fields, map entries |
| 3 | SGROUP | none (starts a group) | deprecated proto2 groups |
| 4 | EGROUP | none (ends a group) | deprecated proto2 groups |
| 5 | I32 (fixed32) | 4 bytes little-endian | fixed32, sfixed32, float |
Wire types 6 and 7 are invalid. Negative int32/int64 values (not sint) are sign-extended and always take 10 bytes. sint32/sint64 use ZigZag: (n << 1) ^ (n >> 31 or 63).
Reading an unknown message by hand
- A tag byte below 0x80 is field number = byte >> 3, so 0x0A is field 1 LEN, 0x12 field 2 LEN, 0x08 field 1 VARINT, 0x10 field 2 VARINT, 0x1A field 3 LEN, 0x22 field 4 LEN, 0x2A field 5 LEN.
- A LEN field is ambiguous: try UTF-8 string, then a nested message, then packed scalars. A packed repeated field is LEN whose payload is a run of varints or fixed-size values.
- The same descriptor structure (field numbers, labels, types) is itself a protobuf message,
FileDescriptorProto, which is how binaries embed their schema. - Maps are
repeatedentry messages with key = field 1 and value = field 2. - With a known schema,
protoc --decode_rawprints any message without a schema as field-number trees;protoc --decode=pkg.Msg file.protouses a schema.
Related
- Embedded schemas, generated-code fingerprints and extracting .proto files: Library:Protocol Buffers
- Field numbers of the descriptor messages: Library:Protocol Buffers/Embedded descriptors
Code & repositories
- https://github.com/protocolbuffers/protobuf @
522b8d41d5d5cdc7ef581c913277aa309f9b7c6a· license: BSD-3-Clause (declared by the contributor; GitHub's detector did not recognize LICENSE)
protoc and runtimes: descriptor.proto, C++/Python/Java generators and the DescriptorTable struct.
Tested on: source read, not run: source reading · checked 2026-10-18
Evidence & references
- Wire type constants: protocolbuffers/protobuf
src/google/protobuf/wire_format_lite.hlines 88-93 (WIRETYPE_VARINT=0 ... WIRETYPE_FIXED32=5). - Encoding specification: https://protobuf.dev/programming-guides/encoding/ (public documentation).