Technique:Reversing network protocols
Appearance
| On this wiki | Facts · Code · Notes · Topics |
|---|
How to work out an application's own network protocol from captures of your own traffic: capture, strip the transport, find message boundaries, then length fields, type ids, checksums, compression and serialization, and write a dissector to test the model.
Scope
For analysing your own application, a server you run or a documented protocol. Capture only traffic you own. Encrypted transports (TLS) need the client's own key log (Format:SSLKEYLOGFILE, which can be embedded in a Format:Pcapng Decryption Secrets Block) or a debug build; do not intercept other people's traffic.
Workflow
- Capture with tcpdump or Wireshark into Format:pcap (libpcap savefile) or Format:pcapng. Use the loopback or a dedicated interface, fixed ports, and short scripted sessions (one action per capture) so differences are attributable. Note snaplen: truncated packets break length analysis.
- Separate the layers. Check the link type (see the pcap page), then IP/TCP or UDP. Find the stream with "Follow TCP stream" and look at each direction as raw bytes. Check first whether it is a known protocol on a nonstandard port (HTTP, TLS, DNS, MQTT, WebSocket): if the first bytes are
16 03 0xit is a TLS handshake; see Technique:TLS client fingerprinting for what the Client Hello reveals about the client library. - Find message boundaries. TCP is a byte stream, so one packet is not one message. Look for (a) a fixed-size header, (b) a length prefix, (c) a delimiter such as CRLF or a NUL, (d) one message per UDP datagram. Reassemble per stream before judging.
- Length fields. In a hexdump of several messages of different sizes, look for a 1, 2, 3 or 4 byte integer at a fixed offset that equals the message size, size minus header, or size minus a constant; try both byte orders. Varints (7 bits per byte, high bit as continuation) are typical of Format:Protocol Buffers wire format and many custom protocols.
- Type / opcode ids. A small field near the start that takes few distinct values and correlates with the action you triggered. Make a table: value, direction, size range, what you did. Request and response often share the id or differ by one bit.
- Sequence and session fields. A counter that increments per message, a constant session id after login, or an echoed request id. Change one action and diff.
- Checksums. A trailing 1, 2 or 4 byte field that changes with the content: use the method of Technique:Identifying checksums in file formats (CRC-16/32 parameter search, sums, XOR). Remember transport checksums (TCP/UDP) are offloaded and show as wrong in captures taken on the sending host; this is normal.
- Compression. High-entropy payload with no readable strings after the header may be compressed or encrypted. Test for magics: zlib
78 9C/78 01/78 DA(Format:Zlib stream, Format:DEFLATE), gzip1F 8B, zstd28 B5 2F FD, LZ4 frame04 22 4D 18(Unverified: magic values from general knowledge). See Technique:Identifying compression libraries in binaries. If the client has a compression library imported, trace the call that sits next to the send. - Serialization. Quick tells by first bytes: protobuf tags like 08 0A 12 (Format:Protocol Buffers wire format); a Thrift compact message starts 82 21 or 82 41, a Thrift struct uses field-delta nibbles and ends in 00 (Format:Thrift compact protocol); a MessagePack map starts 8N or DE followed by fixstr keys A0-BF (Format:MessagePack); CBOR maps start A0-BF with keys 60-7B, and may begin D9 D9 F7 (Format:CBOR); BSON starts with an int32 equal to the total length and ends in 00 (Format:BSON); a FlatBuffers buffer starts with a small forward u32 offset and often a 4-letter identifier at offset 4 (Format:FlatBuffers). Readable field names mean JSON/BSON/MessagePack-like text-ish formats; tag bytes
0x08 0x0A 0x12and length-delimited strings point to Format:Protocol Buffers wire format; fixed-offset structs point to C structs sent raw (check padding and endianness). - Cross-check with the binary. Search the client for the type ids as constants, the strings that appear in the packets, and the functions that call
send/WSASend/SSL_write; names found go on the software page. - Write a dissector (Tool:Wireshark Lua dissectors) so every new capture is parsed automatically; unparsed trailing bytes are your to-do list.
Pitfalls
- Fields change with the version of the client: record it.
- Padding and alignment make lengths look odd; look at several sizes.
- Nagle and delayed ACK merge or split messages; never rely on packet boundaries for TCP.
- Data may be obfuscated with a fixed XOR key: XOR of two ciphertexts of the same plaintext is zero.
- UDP game protocols often have reliability layers (sequence, ack bitmaps) before the real message.