Library:Protocol Buffers
| On this wiki | Facts · Code · Notes · Topics |
|---|
Google's schema-based serialization library and the protoc code generator (BSD-3-Clause). Programs that use it contain generated message classes plus the full serialized .proto file as an embedded string, so the message definitions can usually be recovered from the binary without any decompilation.
Overview
protoc compiles .proto files into per-language code (C++, Java, Python, C#, Go, Rust, ...). Messages are encoded with Format:Protocol Buffers wire format. In the C++, Python and Java generators the whole FileDescriptorProto of each .proto file is serialized and embedded in the output as a byte string, so that reflection, text format, JSON and Any work at runtime. The embedded descriptor contains message, field, enum, service and method names, field numbers, types and labels, and the file name and package, which is nearly the original .proto. Only comments are missing (SourceCodeInfo is not embedded) and, in newer versions, options with source retention are stripped. Details and the recovery recipe: Library:Protocol Buffers/Embedded descriptors.
Identifying protobuf in a binary
- C++: symbols/RTTI containing
google::protobuf::Message,MessageLite,internal::DescriptorTable,.?AVMessage@protobuf@google@@(MSVC) or_ZTVN6google8protobuf7MessageE(Itanium). Generated classes derive from::google::protobuf::Message(full runtime) orMessageLite(optimize_for = LITE_RUNTIME). - C++ generated data symbols:
descriptor_table_<file>_2eproto(the DescriptorTable),descriptor_table_protodef_<file>_2eproto(the serialized descriptor), where non-alphanumeric characters of the .proto path become_plus two hex digits (foo/bar.proto->foo_2fbar_2eproto). Present only if not stripped; the strings below survive stripping. - The embedded blob begins with tag 0x0A (field 1 = file name) then the length-prefixed path ending in
.proto, so search for the bytes0A xx ... 2E 70 72 6F 74 6F(.proto);google/protobuf/descriptor.proto,google/protobuf/any.proto,google/protobuf/timestamp.protoare in nearly every full-runtime binary. - Strings:
type.googleapis.com/(Any type URL prefix),libprotobuf ERROR,CHECK failed:(absl logging),proto3(syntax field, field 12 of the descriptor),google.protobuf.prefixes. - Generated C++ headers contain the comment
// Generated by the protocol buffer compiler. DO NOT EDIT!and// Protobuf C++ Version: <ver>(not compiled into binaries); the version is checked at compile time againstPROTOBUF_VERSION(7038000-style: major1000000 + minor1000 + patch, with the language-runtime version, e.g. C++ 7.x belongs to protoc 3x). - Python:
_descriptor_pool.Default().AddSerializedFile(b'...')in*_pb2.py; a# Protobuf Python Version:header and_runtime_version.ValidateProtobufRuntimeVersion(call in recent generators; error textsDetected incompatible Protobuf Gencode/Runtime versions when loading. - Java:
descriptorData = { ... }string array passed tointernalBuildGeneratedFileFrom. - gRPC services show up as ServiceDescriptorProto entries; method full names
/pkg.Service/Methodare used as HTTP/2 paths.
Generated C++ class shape
A message class has FullMessageName() returning the full proto name (a plain string in the binary), GetClassData(), _InternalSerialize / table-driven parse (TcParseTable) in new versions, _internal_metadata_, a _has_bits_ word array for optional fields and _cached_size_. Field numbers appear as k<Name>FieldNumber constants in headers (not in the binary); in the binary the field numbers are in the parse tables and, as data, in the embedded descriptor.
Topics
- Embedded descriptors — How to find and decode the FileDescriptorProto that protoc embeds in generated C++, Python and Java code, how the DescriptorTable that points to it is laid…
Pitfalls
optimize_for = LITE_RUNTIMEandstrip_nonfunctional_codegenbuilds have no (or an empty) descriptor: field names are gone; use the wire format and the parse tables.- Descriptors above 65535 bytes are emitted as a char array on MSVC instead of a string literal (no difference in the binary).
- Obfuscators and some mobile SDKs rename messages but the field numbers/types are unchanged.
- Third-party runtimes (nanopb, protobuf-c, Go protobuf-go, prost, Swift) have different layouts; Go embeds the descriptor as
file_<path>_proto_rawDesc(unverified here, from the protobuf-go project, not checked).
Code & repositories
- https://github.com/protocolbuffers/protobuf (BSD-3-Clause)
- https://github.com/protocolbuffers/protobuf @
522b8d41d5d5cdc7ef581c913277aa309f9b7c6a· license: BSD-3-Clause (declared by the contributor; GitHub's detector did not recognize LICENSE)
protoc and runtimes: descriptor.proto, C++/Python/Java generators and the DescriptorTable struct.
Tested on: source read, not run: source reading · checked 2026-10-18
Evidence & references
- Embedding of the serialized descriptor and naming:
src/google/protobuf/compiler/cpp/file.cclines 1152-1285,compiler/cpp/helpers.cclines 989-1008,compiler/python/generator.cclines 340-370 and 571-617,compiler/java/shared_code_generator.cclines 142-188 (commit 522b8d41d5d5cdc7ef581c913277aa309f9b7c6a).