CS Fundamentals Trainer · Guides

Protocols: HTTP, SOAP, REST, gRPC, GraphQL, WebSockets and messaging

The protocols a working engineer meets, compared with the same six questions: HTTP and REST, SOAP and the WS-* family, JSON-RPC and gRPC with protobuf on the wire, GraphQL, WebSockets, server-sent events and long polling, webhooks, AMQP, MQTT and STOMP, and SMTP, FTP, SFTP and SSH, with real bytes, an error-vocabulary table across protocols, ports, versioning, tracing and a decision chart.

66 min read

"Which protocol should this use?" is usually answered by habit: HTTP and JSON, because that is what the last service used. Habit is not a bad default, but it hides the question. A protocol decides how two programs find each other, how a message is framed, who may speak first, what counts as an error, and what happens when one side is upgraded and the other is not. Those decisions are what make a protocol cheap or expensive to live with, long after the first version ships.

This guide covers the protocols a working engineer actually meets: HTTP as the substrate, REST and SOAP on top of it, JSON-RPC and gRPC, GraphQL, WebSockets and server-sent events, webhooks, the messaging protocols (AMQP, MQTT, STOMP), and the older ones that refuse to die (SMTP, FTP, SFTP, SSH). It compares them with the same six questions throughout, and it shows the actual bytes wherever bytes make the point.

How to read it.

You wantRead
A decision, todaySection 1, then section 13 (choosing), then the family you picked
To understand one protocolIts own section; each ends with where it fits and where it does not
To read a protocol you have never seenSection 1's six questions, then section 12 (the cross-cutting concerns)
The whole pictureIn order. Sections 4 (SOAP), 5 (gRPC) and 10 (messaging) are the deep dives

What kind of evidence is in this guide.

KindWhereHow far to trust it
Specification textthe RFCs, W3C and OASIS documents named in each section and in Sourcesas far as the document; always check which version of the protocol you have
Bytes produced herethe hex dumps and sizes in sections 2, 5, 6, 8 and 10they were produced by the script in the appendix, on Python 3.13 and curl 8.5, and you can run it
Arithmeticthe message-size and round-trip comparisonsit follows from the byte counts shown; nothing is extrapolated to your traffic
Measured in productionnoneno latency or throughput figure here describes a real system. Section 12 says what to measure instead

Transport and infrastructure are covered by their own guides, and this one does not repeat them: TCP in practice, DNS, HTTP/2, HTTP/3 and CDNs, TLS and HTTPS, load balancing and what happens when you type a URL.

1. Six questions every protocol answers

In short. Every protocol on this page answers the same six questions. Learn the questions and a protocol you have never seen becomes readable in an afternoon.

#QuestionWhy it decides how you live with it
1Framing: where does one message end?A length prefix, a terminator, or "until the connection closes". This decides whether streaming is possible and whether a parser can be attacked with a mismatched length
2Addressing: how is a thing named?A URL path, a method name, a topic, a queue, a mailbox. It decides what a cache, a proxy or an access rule can reason about
3Who speaks first, and how often?Request/response, one-way push, or either side at any time. It decides how firewalls, load balancers and timeouts behave
4Types: is there a schema, and who holds it?None (JSON), a document (WSDL, OpenAPI), or a compiled contract (protobuf). It decides whether a breaking change is caught at build time or at 3 a.m.
5Errors: how is failure expressed?A status code, a fault document, an error array, a reason code. It decides whether a retry is safe, and whether a client can tell "no" from "not now"
6Evolution: how do you add a field?Additive by rule, by a new version in the URL, by a new namespace, or not at all. It decides how often you break somebody

Use the questions as a checklist. The table below is the whole guide in one screen; the rest of it fills in the detail.

ProtocolFramingAddressingWho speaksSchemaErrorsTypical use
HTTP/1.1Content-Length or chunkedURLclient asks, server answersnone built instatus codeeverything on the web
HTTP/2 and /3binary frames per streamURLclient asks; many streams at oncenone built instatus code plus stream errorsthe same, faster
REST over HTTPHTTP'sURL per resourceclientoptional (OpenAPI)status code plus a bodypublic and internal web APIs
SOAPHTTP's, body is one XML envelopeone endpoint plus an action nameclientWSDL and XSD, mandatoryFault elemententerprise, government, legacy integration
JSON-RPC 2.0HTTP's, or a stream of JSONmethod nameclient, and notificationsnone built innumeric error objectinternal tools, blockchain nodes, editors
gRPC5-byte prefix per message inside HTTP/2 frames/package.Service/Methodboth, streaming either wayprotobuf, compilednumeric status in trailersservice-to-service inside one system
GraphQLHTTP's; one POSTone endpoint, the query names the dataclientschema, introspectableerrors array, usually with HTTP 200one API feeding many different screens
WebSocketits own frame headerthe URL of the upgradeboth, at any timenone built inclose codechat, games, live dashboards
SSElines, blank line ends an eventa URLserver pushes onlynoneHTTP status at connecttickers, progress, notifications
WebhookHTTP's, invertedyour URL at their systemthey call youtheirsyour status codeintegrations between companies
AMQP 0-9-1binary framesexchange plus routing keybroker in the middlenonechannel and connection errorswork queues, enterprise integration
MQTT2 or more byte fixed headertopicbroker in the middlenonereason codes (MQTT 5)IoT, mobile, anything on a bad link
SMTPline-based, . ends the datamailboxclient pushes to a servernonethree-digit reply codesmail, and nothing else any more
SFTPbinary packets over an SSH channelfile pathclientnonestatus codesfile transfer between machines

2. HTTP: the substrate almost everything sits on

In short. HTTP is a request/response protocol with a text-like surface, a huge amount of agreed semantics, and an infrastructure of caches and proxies that only works if you use its vocabulary correctly.

HTTP is specified in pieces: the semantics (methods, status codes, headers) in RFC 9110, the HTTP/1.1 syntax in RFC 9112, caching in RFC 9111, HTTP/2 in RFC 9113 and HTTP/3 in RFC 9114. The pieces matter because "HTTP" means two different things in a sentence: the meaning of a request, which is the same in every version, and the way bytes are written, which is not.

What a client really sends

Here is what curl put on the wire for one GET against a local server, as the server read it:

GET /orders/ord_7 HTTP/1.1\r\n
Host: 127.0.0.1:8731\r\n
User-Agent: curl/8.5.0\r\n
Accept: application/json\r\n
\r\n

Four lines and a blank line. Three things are worth noting. Host is mandatory in HTTP/1.1, which is what makes virtual hosting possible: one address, many sites. The empty line is the framing: headers end there, and whatever follows is the body. And Accept is a wish, not a command; the server chooses.

The answer carried its own framing and its cache rules:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 27
ETag: "v3"
Cache-Control: max-age=60

{"id":"ord_7","total":4299}

The second request repeated the GET with If-None-Match: "v3", and the server answered without the body:

HTTP/1.1 304 Not Modified\r\n
ETag: "v3"\r\n
Cache-Control: max-age=60\r\n
\r\n

The 200 answer was 131 bytes of headers and body; the 304 was 68 bytes and no body. That is the whole idea of a conditional request: the server proves the client's copy is still good instead of sending it again. On a 2 MB response the saving is not 63 bytes, it is 2 MB, and it costs one header.

Framing without a length

When the server does not know the size in advance it can frame with chunks instead:

a\r\n
{"page":1}\r\n
a\r\n
{"page":2}\r\n
0\r\n
\r\n

Each chunk is a hex length, the bytes, then a zero-length chunk to end. Twenty bytes of payload cost 35 on the wire here, which is the price of being able to start sending before you know the total. This is how HTTP/1.1 streams, and it is why a proxy that buffers whole responses ruins streaming APIs. HTTP/2 and HTTP/3 do not use chunked encoding: they have their own frames, and Transfer-Encoding: chunked is not allowed there.

The method table, with the only two words that matter

MethodSafeIdempotentBodyTypical meaning
GETyesyesno defined semanticsread
HEADyesyesnoread the headers only
OPTIONSyesyesnowhat may I do here (and CORS preflight)
PUTnoyesyesreplace at a known address
DELETEnoyesoptionalremove
POSTnonoyesanything else: create, act, submit
PATCHnonot by definitionyespartial change (RFC 5789)

Safe means the request is not meant to change anything, so a crawler or a prefetch may issue it. Idempotent means doing it twice has the same effect as doing it once, which is exactly the property a retry needs. A proxy, a mobile network or your own retry policy will repeat requests, and POST is the method where that can double a charge, which is why idempotency keys exist (idempotency and exactly-once effects).

PATCH needs a sentence of its own, because the common shorthand "POST and PATCH are not idempotent" mixes up two different things. The method is not defined as idempotent: RFC 5789 says PATCH is neither safe nor idempotent, so a proxy, a gateway or a generic retry layer must not assume that repeating it is harmless. Whether a particular patch is idempotent depends on what the patch document says. {"op": "replace", "path": "/qty", "value": 5} lands on the same state however often it is applied; {"op": "increment", "path": "/qty", "by": 1} does not. So: the method gives you no guarantee, your patches may well have one, and the only way a client learns which is from your documentation or from an idempotency key you accept.

Content in a GET is a different kind of trap. Message framing does not forbid it, but RFC 9110 states that content received in a GET has no generally defined semantics: it cannot alter the meaning or the target of the request, caches and proxies key on the method and the URL alone, and it has enough history as a request-smuggling vector that some servers and intermediaries reject such a request outright. "Servers ignore it" is the usual outcome rather than a rule you can rely on, which is reason enough to keep a query in the URL, or to use POST and accept the loss of caching.

Status codes, read as a protocol

The class is the part a client should branch on, and the specific code is the part a human reads.

ClassMeaning for a clientCodes worth knowing exactly
1xxkeep waiting, something is coming100 Continue, 101 Switching Protocols (the WebSocket upgrade)
2xxit worked200, 201 with Location, 202 accepted and not finished, 204 no body
3xxlook elsewhere, or you already have it301/308 permanent, 302/307 temporary, 304 not modified
4xxthe request is wrong; do not retry it unchanged400 malformed, 401 not authenticated, 403 authenticated and not allowed, 404, 409 conflict, 412 precondition failed, 422 understood but semantically wrong, 429 too many requests
5xxthe server failed; a retry may work500, 502 bad gateway, 503 unavailable (with Retry-After), 504 gateway timeout

Two distinctions carry most of the value. 401 against 403: the first says "I do not know who you are", the second says "I know, and no". 429 and 503 against 500: the first two are invitations to come back, ideally with Retry-After; a bare 500 is not, and retrying it hard is how a wobble becomes an outage (resilience patterns).

For the body of an error, RFC 9457 defines application/problem+json, which replaced RFC 7807:

{
  "type": "https://api.example.com/problems/insufficient-funds",
  "title": "Insufficient funds",
  "status": 409,
  "detail": "Account ends 4291 is short by EUR 12.40",
  "instance": "/transfers/t_8812",
  "balance_cents": 128760
}

type is a stable identifier a client can switch on, title and detail are for people, and extra members are allowed. Using it costs nothing and saves every client from parsing your prose.

Content negotiation, and the header that makes caching correct

A client states preferences (Accept, Accept-Encoding, Accept-Language), the server picks one and must then say what the answer depended on:

Vary: Accept-Encoding, Accept-Language

Without Vary, a shared cache can hand a German gzip answer to an English client asking for plain text. With too much Vary (User-Agent, for example) the cache hit rate collapses. Caching itself is covered in DNS, HTTP/2, HTTP/3 and CDNs.

The versions, in one table

HTTP/1.1HTTP/2HTTP/3
Wire formattext, one request at a time per connectionbinary frames, many streams per connectionbinary frames over QUIC
TransportTCPTCPUDP, with QUIC providing streams and TLS 1.3
Header compressionnone (repeat everything)HPACKQPACK
Blockingone slow response blocks the connection; browsers open about sixstreams are independent in HTTP, but a lost TCP segment stalls them alla lost packet stalls only its own stream
Server pushnospecified, then abandoned by browsersno
Trailerschunked trailers, rarely supportedyes, and gRPC depends on themyes

What this means in practice: HTTP/1.1 rewards few, large requests and cache-friendly URLs; HTTP/2 and HTTP/3 make many small requests reasonable, which changes API design (the request-waterfall problem). Nothing above changes the meaning of a GET.

Where HTTP fits, and where it does not

It fits anything a client initiates, anything that benefits from caches, proxies and existing tooling, and anything that must cross a strange network: port 443 is the one port that always works. It does not fit server-initiated messages (sections 8 and 9), very chatty internal calls where the header overhead dominates (section 5), or a device on a metered radio link sending four bytes (section 10).

3. REST: HTTP used on purpose

In short. REST is not "JSON over HTTP". It is a set of constraints whose practical payoff is that generic infrastructure understands your API: caches, proxies, retries and conditional requests all work because the meaning lives in the method and the URL.

Roy Fielding described REST in 2000 as an architectural style with constraints: a uniform interface, statelessness, a client/server split, cacheability, layering, and code on demand. The part that earns its keep daily is the uniform interface. When "read order 7" is GET /orders/7, a CDN can cache it, a proxy can retry it, a browser can bookmark it and a log line tells you what happened. When it is POST /api with {"op":"getOrder","id":7}, none of that works, and you have an RPC API wearing REST's clothes.

A compact example of the vocabulary in use:

POST /orders                       -> 201 Created, Location: /orders/7
GET /orders/7                      -> 200, ETag: "v3", Cache-Control: max-age=60
GET /orders/7 (If-None-Match "v3") -> 304 Not Modified
PUT /orders/7 (If-Match: "v3")     -> 200, or 412 if someone else changed it first
PATCH /orders/7                    -> 200, with a media type that says what a patch is
DELETE /orders/7                   -> 204, and 404 or 204 again if repeated
GET /orders?status=open&limit=50   -> 200, with a cursor for the next page

Two patterns in there are worth copying. If-Match on a write is optimistic concurrency: the write fails with 412 instead of silently overwriting a change made in the meantime. And the collection URL carries filters as query parameters, so the same resource can be cached per filter.

The design questions that follow (resource modelling, pagination, filtering, versioning, partial responses, bulk operations) are the subject of their own guide: designing a REST API. Two points belong here because they are protocol points, not design taste:

Where REST fits, and where it does not

It fits public APIs, anything a browser touches, anything with a long tail of unknown clients, and anything that benefits from caching. It struggles when one screen needs twelve resources (section 7 answers that), when the calls are internal and extremely frequent (section 5), and when the interaction is genuinely an action rather than a thing: POST /transfers is a resource, but "cancel this job if it has not started" is a verb, and dressing verbs as resources is where REST designs get silly.

4. SOAP and the WS-* family

In short. SOAP is an XML envelope with a mandatory machine-readable contract (WSDL) and a stack of optional standards for security, addressing and reliability. It is verbose and old, it is still everywhere in banking, insurance, telecoms, healthcare and government, and the reason is the contract, not nostalgia.

A SOAP message is one XML document with a fixed shape: an Envelope containing an optional Header and a mandatory Body.

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Header>
    <wsa:MessageID xmlns:wsa="http://www.w3.org/2005/08/addressing">urn:uuid:9e1f...</wsa:MessageID>
    <t:Transaction xmlns:t="urn:shop:tx" soap:mustUnderstand="true">8812</t:Transaction>
  </soap:Header>
  <soap:Body>
    <GetOrder xmlns="urn:shop:orders"><Id>ord_7</Id></GetOrder>
  </soap:Body>
</soap:Envelope>

Three details in there explain most SOAP behaviour.

The two versions differ in ways that break clients

SOAP 1.1 (2000)SOAP 1.2 (W3C Recommendation, 2007)
Envelope namespacehttp://schemas.xmlsoap.org/soap/envelope/http://www.w3.org/2003/05/soap-envelope
Content typetext/xmlapplication/soap+xml
Which operationthe separate SOAPAction HTTP header, mandatoryan action parameter on the content type (SOAPAction optional)
Fault shapefaultcode, faultstring, faultactor, detailCode with Value and Subcode, Reason with language, Detail
Fault codesClient, Server, VersionMismatch, MustUnderstandSender, Receiver, VersionMismatch, MustUnderstand, DataEncodingUnknown

Plenty of production endpoints are still 1.1. If a client sends a 1.2 envelope to a 1.1 service, the usual answer is a VersionMismatch fault, and a missing SOAPAction against 1.1 often produces a bare HTTP error instead of a fault, which is the classic "it works in SoapUI but not from my code" situation.

A fault, in 1.2, and the status code that carries it:

<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Body>
    <soap:Fault>
      <soap:Code><soap:Value>soap:Sender</soap:Value>
        <soap:Subcode><soap:Value xmlns:s="urn:shop:orders">s:UnknownOrder</soap:Value></soap:Subcode>
      </soap:Code>
      <soap:Reason><soap:Text xml:lang="en">No order with that id</soap:Text></soap:Reason>
      <soap:Detail><OrderId xmlns="urn:shop:orders">ord_7</OrderId></soap:Detail>
    </soap:Fault>
  </soap:Body>
</soap:Envelope>

Over the HTTP binding a fault arrives with HTTP 500 (a sender fault in SOAP 1.2 may use 400), which is the single most confusing thing about SOAP for people coming from REST: a perfectly well-handled business error ("no such order") is an HTTP 500. Monitoring that counts 5xx as "server broken" will scream at a SOAP service forever unless it parses the envelope.

WSDL: the contract that is the reason SOAP survives

A WSDL 1.1 document describes the service in five parts: types (XML Schema for every message), message, portType (the operations), binding (how they travel: SOAP over HTTP, document/literal) and service (the endpoint URL). It is machine-readable in a way OpenAPI only approximates, because the payload schema is XSD: types, cardinality, enumerations, patterns, nested structures, all validated by the stack before your code runs.

The practical consequence is tooling. Point a generator at a URL and get typed client code:

# .NET
dotnet-svcutil https://partner.example.com/orders.svc?wsdl
# Java
wsimport -keep -p com.example.orders https://partner.example.com/orders?wsdl
# Python
python -m zeep https://partner.example.com/orders?wsdl        # inspect the operations
from zeep import Client                    # illustrative; zeep is a third-party library
client = Client("https://partner.example.com/orders?wsdl")
order = client.service.GetOrder(Id="ord_7")   # types, faults and the envelope are handled
print(order.TotalCents)

That is the trade SOAP makes: a lot of ceremony, and in exchange a contract both sides can compile against, with a version mismatch caught by a schema validator rather than by a null in production.

The WS-* standards you may actually meet

StandardWhat it addsWhere you see it
WS-Securitysigned and encrypted parts of the message itself, with X.509 or username tokens, independent of the transportbanking, government, anything where the message must stay signed after it leaves the TLS tunnel
WS-Addressingmessage ids, reply-to and fault-to endpoints in headersasynchronous SOAP, callbacks, message brokers in the middle
WS-ReliableMessagingsequence numbers and acknowledgements, with a delivery assurance chosen per sequence: at-most-once, at-least-once, exactly-once or in-ordertelecom and ERP integrations over unreliable links
WS-Policymachine-readable statements of what a service requires (which security, which addressing)negotiating with a partner's stack
MTOM / XOPbinary attachments sent as real bytes instead of base64 inside the XMLdocuments, images, X-rays

Message-level security is the one that has no REST equivalent in common use. TLS protects the hop; a WS-Security signature stays with the document through a queue, a broker and an archive, and an auditor can verify it a year later. When a bank insists on SOAP, this is usually why.

Where SOAP fits, and where it does not

It fits a partner integration where a contract must be negotiated, validated and audited; where the message, not the connection, must be signed; and where the other side already has it. It does not fit browsers (XML parsing and envelope overhead for nothing), mobile clients on metered links, or any internal service you control both ends of. And it is not a reason to avoid an integration: generated clients make calling a SOAP service a day of work, not a quarter.

5. RPC: calling a function that lives somewhere else

In short. RPC protocols give up URLs and HTTP semantics and get back a typed function call. gRPC is the modern one: protobuf for types, HTTP/2 for transport, streaming in both directions, and status in trailers.

The family tree is short. XML-RPC (1998) put a method name and typed parameters in a small XML document. JSON-RPC 2.0 (2010) did the same with JSON and is still a good choice when you want RPC without a code generator:

--> {"jsonrpc":"2.0","method":"orders.get","params":{"id":"ord_7"},"id":1}
<-- {"jsonrpc":"2.0","result":{"id":"ord_7","total_cents":4299},"id":1}

--> {"jsonrpc":"2.0","method":"metrics.report","params":{"ok":true}}          // no id: a notification, no answer
<-- (nothing)

--> [{"jsonrpc":"2.0","method":"a","id":1},{"jsonrpc":"2.0","method":"b","id":2}]   // a batch

Its error object uses reserved numeric codes, which is a cheap way to tell a protocol problem from a business one:

CodeMeaning
−32700invalid JSON
−32600not a valid request object
−32601method not found
−32602invalid parameters
−32603internal error
−32000 to −32099reserved for your own server errors

JSON-RPC is transport-agnostic: the same frames travel over HTTP POST, a WebSocket, a pipe or a TCP socket. That is why editors (the Language Server Protocol), Ethereum nodes and plenty of internal daemons use it.

gRPC

gRPC is an RPC system with four parts that are easier to understand separately: protobuf for the contract and the bytes (section 6), HTTP/2 for transport, a call model with four shapes, and a status model of its own.

The service is declared, then both sides generate code:

syntax = "proto3";
package shop;

service Orders {
  rpc GetOrder (GetOrderRequest) returns (Order);                    // unary
  rpc ListOrders (ListRequest) returns (stream Order);               // server streaming
  rpc ImportOrders (stream Order) returns (ImportSummary);           // client streaming
  rpc Watch (stream WatchRequest) returns (stream OrderEvent);       // bidirectional
}

On the wire a unary call is one HTTP/2 request whose path is the method and whose body is a sequence of length-prefixed messages:

:method = POST
:scheme = https
:path = /shop.Orders/GetOrder
content-type = application/grpc+proto
te = trailers
grpc-timeout = 2S                       <- the client's deadline travels with the call
authorization = Bearer ...              <- metadata is just HTTP/2 headers

Each message gets a five-byte prefix: one byte saying whether it is compressed, then a four-byte big-endian length. Here is a real one, wrapping the 39-byte protobuf record from section 6:

0000  00 00 00 00 27 0a 05 6f 72 64 5f 37 12 04 61 63  ....'..ord_7..ac
0010  6d 65 18 cb 21                                   me..!

00 means uncompressed, 00 00 00 27 is 39, and the rest is the message. Because the length comes first, a receiver always knows where a message ends, which is what makes streaming many messages over one call straightforward.

The status does not travel in the body. It arrives as HTTP/2 trailers after the last message:

grpc-status: 0
grpc-message:

That design is what allows a server to start streaming results before it knows whether the whole operation will succeed, and it is also why gRPC needs HTTP/2 and why a proxy that drops trailers breaks it in a way that is painful to diagnose: the data arrives, the call never completes.

The status codes are a fixed, small set, which is a real advantage over "pick a 4xx":

CodeNameRetry?
0OK—
1CANCELLEDno, the caller gave up
3INVALID_ARGUMENTno, fix the request
4DEADLINE_EXCEEDEDmaybe, with a longer deadline
5NOT_FOUNDno
6ALREADY_EXISTSno, and it often means your retry worked the first time
7PERMISSION_DENIEDno
8RESOURCE_EXHAUSTEDyes, with backoff
9FAILED_PRECONDITIONno, until the state changes
10ABORTEDyes, usually a concurrency conflict
12UNIMPLEMENTEDno
13INTERNALno, a bug
14UNAVAILABLEyes, with backoff
16UNAUTHENTICATEDno, get a token

Two practical consequences of the design. Deadlines are part of the protocol: grpc-timeout is sent, propagated to downstream calls and enforced on both sides, which is a discipline most HTTP stacks have to be talked into (timeouts and retries). And a gRPC channel is one long-lived HTTP/2 connection, so a connection-level load balancer pins every call to one backend; balancing needs an L7 proxy or client-side balancing (load balancing, service meshes).

Browsers cannot speak gRPC directly, because they cannot control HTTP/2 frames or read trailers. gRPC-Web fixes that with a slightly different framing and a proxy that translates; bidirectional streaming is not available.

Where RPC fits, and where it does not

gRPC fits internal service-to-service traffic, high call rates, polyglot teams that want one generated contract, and streaming both ways. JSON-RPC fits internal tools and anything that must be debuggable by hand. RPC does not fit public APIs with unknown clients (a URL and curl are a better front door), anything that needs HTTP caching, or a team with no appetite for a build step and a schema registry.

6. Schemas and the bytes they produce

In short. The payload format decides size, speed, and whether a wrong field is caught at build time. The same small record is 39 bytes as protobuf, 122 as compact JSON and 406 inside a SOAP envelope.

One order with two items, serialized three ways and then gzipped (the script in the appendix produced these numbers; they are byte counts of this one record, not a benchmark of your traffic):

FormatBytesGzipped
protobuf3957
JSON, compact122118
SOAP 1.2 envelope, XML406258

Note the honest surprise: gzip makes the protobuf message bigger, because at 39 bytes the gzip header and dictionary cost more than they save. Compression pays on repetitive text, which is exactly what XML and JSON are. The 10x gap between protobuf and XML closes to about 4.5x once both are compressed, and a single HTTP response header set can be larger than any of them, which is why HTTP/2's header compression mattered so much.

How protobuf gets to 39 bytes

The bytes of that record, from the real library:

0000  0a 05 6f 72 64 5f 37 12 04 61 63 6d 65 18 cb 21  ..ord_7..acme..!
0010  22 03 45 55 52 2a 07 0a 03 41 2d 31 10 02 2a 07  ".EUR*...A-1..*.
0020  0a 03 42 2d 39 10 01                             ..B-9..

There are no field names. Each field is a tag byte that packs the field number and a wire type, then the value. 0a is field 1, wire type 2 (length-delimited), so what follows is a length (05) and five bytes: ord_7. 18 cb 21 is field 3, wire type 0 (varint), carrying 4299.

A varint stores seven bits per byte and uses the top bit to mean "more to come":

varint(1)    = 01
varint(300)  = ac 02
varint(4299) = cb 21

Small numbers cost one byte, which is why field numbers 1 to 15 fit in a single tag byte and why the protobuf style guide tells you to spend them on the fields that appear most.

The wire types are the whole encoding:

Wire typeUsed forHow it is framed
0int32, int64, uint*, bool, enumvarint
1fixed64, double8 bytes
2string, bytes, embedded messages, packed repeated fieldsvarint length, then that many bytes
5fixed32, float4 bytes

(Wire types 3 and 4 were the old start-group and end-group markers and are deprecated.)

The evolution rules, which are the real contract

Because the wire carries numbers and not names, protobuf has clear compatibility rules, and they are worth memorising:

Compare how each format answers question 6 from section 1:

FormatWhere the schema livesAdding a fieldRemoving oneCaught by
JSONnowhere, or OpenAPI beside itsafe if clients ignore extrasbreaks whoever read ittests, or production
XML with XSDthe schema, shippedneeds a schema change, often a new namespacesamea validator, before your code
protobufthe .proto, compiled into both sidessafesafe if the number is reservedthe compiler, and a schema-registry check in CI
GraphQLthe server's schema, introspectablesafe@deprecated, then watch usagethe schema check in CI

This is the clearest practical difference between the families. JSON buys speed of change and pays for it in uncertainty; compiled schemas buy certainty and pay for it with a build step.

7. GraphQL: the client names the data

In short. One endpoint, one POST, a query language that says exactly which fields are wanted, and a schema the client can introspect. It solves over-fetching and the twelve-request screen, and it moves caching, authorization and cost control from the infrastructure into your code.

A query and its answer:

query OrderScreen($id: ID!) {
  order(id: $id) {
    id
    totalCents
    customer { name email }
    items(first: 10) { sku qty }
  }
}
{ "data": { "order": { "id": "ord_7", "totalCents": 4299,
  "customer": { "name": "Acme", "email": "ops@acme.test" },
  "items": [ { "sku": "A-1", "qty": 2 }, { "sku": "B-9", "qty": 1 } ] } } }

The request is a POST of {"query": "...", "variables": {...}, "operationName": "OrderScreen"} to a single URL, with content-type: application/json. Three protocol consequences follow immediately, and they are what people actually trip over.

A failed field comes back with HTTP 200. A GraphQL response can carry data and errors together, because one field may fail while the rest succeeded:

{ "data": { "order": { "id": "ord_7", "customer": null } },
  "errors": [ { "message": "Forbidden", "path": ["order","customer"],
                "extensions": { "code": "FORBIDDEN" } } ] }

The status code is not always 200, though, and "GraphQL always answers 200" is the simplification that gets teams into trouble. The specification separates a request error, where the document could not be parsed or validated so nothing executed, from a field error raised while executing. Which status you see then depends on the response media type:

application/json, the traditional typeapplication/graphql-response+json, the type the spec recommends
Malformed JSON or an unreadable requestoften 400400
A document that fails validationcommonly 200, with errors and no data400
A field error, with partial data200200
Wrong method, or a media type the server will not take405, 415405, 415

Read "often" and "commonly" as written: GraphQL over HTTP is still a draft, later revisions change some of these codes, and servers differ, so check what yours actually does instead of trusting a table. What holds everywhere is the consequence: a client that branches only on the HTTP status misses field errors, and a dashboard counting non-2xx responses can show a flawless service while every request comes back half-failing. Put your machine-readable code in extensions, alert on the share of responses that contain errors, and ask for the newer media type if you want the status line to carry information.

HTTP caching does not apply. A POST to one URL with a body that differs per screen is uncacheable by any CDN or browser cache. The answers are persisted queries (register the query once, send a hash), GET for queries that are safe to cache, and caching inside the server at the resolver or entity level.

The query is an attack surface. A client can ask for a deeply nested, expensive graph. The protections are depth limits, complexity or cost analysis with a budget per caller, pagination arguments that are mandatory rather than optional, persisted queries in production (so only known queries run), and timeouts per resolver.

The other famous problem is N+1: a list of 50 orders whose customer field triggers 50 database queries. The fix is batching inside the resolver layer (DataLoader-style: collect the ids requested during one tick, fetch them in one query, hand each resolver its slice). This is a server design issue, not a protocol one, but it is the first thing to look for when a GraphQL API is slow.

Subscriptions are not really part of the protocol. The specification defines the operation type, not the transport. In practice subscriptions run over a WebSocket with a sub-protocol (graphql-transport-ws, which replaced the older subscriptions-transport-ws) or over SSE. That means section 8's warnings about long-lived connections apply to GraphQL subscriptions too.

Where GraphQL fits, and where it does not

It fits several different clients with different data needs over one backend, a mobile app that must minimise round trips, and an organisation willing to own the schema as a product. It does not fit a simple CRUD service with one client (it buys nothing and costs a server), file uploads and downloads (HTTP already does that well), or a team that needs CDN caching to carry the load.

8. When the server needs to speak first: WebSocket, SSE and long polling

In short. HTTP's shape is "client asks". Three techniques work around it: hold a request open (long polling), stream a one-way text channel (SSE), or trade the HTTP connection for a two-way frame protocol (WebSocket). Pick the weakest one that does the job, because every long-lived connection is a thing to operate.

WebSocket

A WebSocket starts as an ordinary HTTP request and then stops being HTTP:

GET /live HTTP/1.1
Host: api.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: AAECAwQFBgcICQoLDA0ODw==
Sec-WebSocket-Version: 13
Sec-WebSocket-Protocol: graphql-transport-ws
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: Bz3qJYTGdOe8gUSpLosEdiLKDrk=
Sec-WebSocket-Protocol: graphql-transport-ws

That Sec-WebSocket-Accept value is not a secret and not a token. It is base64(sha1(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11")), with that fixed GUID from RFC 6455. The pair above was computed by the script in the appendix, so you can check it. Its only job is to prove the server understood the upgrade rather than echoing bytes back like a dumb proxy, which also explains why there is no point "validating" it for security.

After the 101 there are no more headers, only frames. A client text frame carrying {"op":"subscribe"}:

0000  81 92 a3 1f 00 7c d8 3d 6f 0c 81 25 22 0f d6 7d  .....|.=o..%"..}
0010  73 1f d1 76 62 19 81 62                          s..vb..b

81 is FIN plus opcode 1 (text). 92 is the mask bit plus length 18. The next four bytes are the masking key, and the rest is the payload XORed with it; unmasking gives back the original JSON exactly. So the overhead is six bytes for a client frame (2 header + 4 mask) and two for a server frame, against roughly 100-800 bytes of headers for an HTTP request. That is the real argument for WebSocket: not that it is faster, but that a message costs almost nothing once the connection exists.

The frame details worth knowing:

FieldValues
Opcodes0x0 continuation, 0x1 text, 0x2 binary, 0x8 close, 0x9 ping, 0xA pong
Maskingmandatory on client-to-server frames, forbidden on server-to-client ones. It exists to stop a crafted payload from poisoning an old proxy
Length0-125 inline, 126 means a 2-byte length follows, 127 means an 8-byte one
Close codes1000 normal, 1001 going away, 1002 protocol error, 1008 policy violation, 1009 message too big, 1011 internal error. 1006 means "the connection died" and is never sent on the wire
Extensionspermessage-deflate (RFC 7692) compresses payloads; it costs memory per connection

What the protocol does not give you, and you therefore have to build:

RFC 8441 allows WebSocket over HTTP/2 via extended CONNECT, which removes the one-connection-per-socket cost; support is uneven. Over HTTP/3 the forward-looking option is WebTransport, which adds unreliable datagrams for cases where a late message is worse than a lost one, such as games.

Server-sent events

If the traffic only flows one way, SSE is a tenth of the work. It is a normal HTTP response that never ends:

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-store

id: 41
event: price
data: {"sym":"ACME","p":12.5}

retry: 5000

: keep-alive comment

The rules fit on a postcard: fields are data, event, id and retry; a blank line dispatches the event; a line starting with : is a comment used as a heartbeat. The browser's EventSource reconnects on its own and sends Last-Event-ID with the id it last saw, so a server that honours that header gives you resumable streams for free. It is text only (UTF-8), one-way, and on HTTP/1.1 it occupies one of the six connections a browser allows per origin, which is the usual reason an SSE page mysteriously stops loading other things. Over HTTP/2 the stream is one of many on a single connection, so that bottleneck goes away, but not every limit does: the ceiling becomes the concurrent-stream limit the server advertises in SETTINGS_MAX_CONCURRENT_STREAMS, commonly in the low hundreds.

Long polling, and when it is right

The client sends a request; the server holds it until there is news or a timeout; the client immediately asks again. It is ugly, it works through every proxy ever built, and it needs no new infrastructure. For a notification that arrives a few times an hour it is a perfectly reasonable answer, and it is the fallback every realtime library still ships.

Long pollingSSEWebSocket
Directionserver to client, one message per requestserver to client, continuousboth ways
Transportplain HTTPplain HTTPHTTP upgrade, then frames
PayloadanythingUTF-8 texttext or binary
Reconnect and resumeyou write itbuilt in (Last-Event-ID)you write it
Proxy and firewall troublealmost nonelittle (watch response buffering)some (upgrade must be allowed, idle timeouts)
Cost per messagea full request and responsea few bytes2 to 6 bytes
Good forrare notificationstickers, progress, logs, feedschat, collaboration, games, trading

The honest default: SSE until you need to send upstream, then WebSocket. Most "we need WebSockets" projects are a one-way feed plus occasional POSTs, and that combination is simpler to operate, debug and cache.

9. Webhooks: HTTP with the arrow reversed

In short. A webhook is somebody else's system making an HTTP request to your URL when something happens. The protocol is trivial; the engineering is all in signatures, retries, ordering and idempotency.

POST /hooks/payments HTTP/1.1
Host: shop.example.com
Content-Type: application/json
Webhook-Id: evt_9f21
Webhook-Timestamp: 1762790400
Webhook-Signature: v1,k3Jp8Qn...=

{"type":"payment.succeeded","id":"evt_9f21","data":{"amount_cents":4299}}

The receiver's checklist, in order, because the order matters:

  1. Verify the signature on the raw bytes, before parsing. The usual scheme is HMAC-SHA256 over the id, the timestamp and the exact body, with a shared secret. Re-serializing the JSON first changes the bytes and breaks the check.
  2. Reject old timestamps (a few minutes' window), or a captured request can be replayed forever.
  3. Answer 2xx quickly, then work. Senders time out in seconds. Store the event and process it asynchronously; do not do the invoice run inside the handler.
  4. Deduplicate on the event id. Retries, at-least-once delivery and your own 500s mean the same event will arrive twice (idempotency).
  5. Do not assume order. payment.succeeded can arrive before order.created. Either tolerate it, or fetch the current state from the sender's API and treat the webhook purely as a nudge: the "thin payload" pattern, which also avoids trusting data you did not fetch.
  6. Expect your endpoint to be probed. Validate the content type and size, cap the body, and keep the handler free of anything that runs on untrusted input (web application security).

On the sending side the obligations mirror these: retry with exponential backoff and a cap, show the customer a delivery log with the response codes, sign every attempt, allow endpoint rotation with two live secrets during the change, and disable an endpoint that has failed for days instead of hammering it. If you cannot be called back at all (no public URL), the remaining options are polling with a cursor or a message broker both sides can reach (section 10).

10. Messaging protocols: AMQP, MQTT, STOMP

In short. These protocols put a broker between the sender and the receiver, so the two never have to be online at the same time. AMQP is the enterprise work-queue protocol, MQTT is the small-device protocol, STOMP is the simple text one. What they buy you is decoupling; what they cost you is a broker to run and a new set of failure modes.

The patterns, the delivery guarantees and the operational side are covered in message queues and delivery guarantees. This section is about the protocols themselves.

AMQP 0-9-1, and the trap in the version number

AMQP 0-9-1 is what RabbitMQ speaks (port 5672, 5671 with TLS). Its model is three nouns: a publisher sends to an exchange with a routing key; bindings connect exchanges to queues; consumers read from queues. Exchange types decide the routing: direct (exact key), topic (orders.*.eu patterns), fanout (everywhere) and headers. One TCP connection carries many channels, so a process does not need a connection per thread.

The protocol features that matter in production:

AMQP 1.0 is a different protocol with the same name. It was standardised by OASIS and as ISO/IEC 19464, and it describes symmetric peers exchanging messages over links with credit-based flow control, not the exchange/queue model. Azure Service Bus and ActiveMQ speak it; a 0-9-1 client cannot talk to a 1.0-only broker. When someone says "it is AMQP, so it will interoperate", ask which version.

MQTT (port 1883, 8883 with TLS) was designed for devices on slow, expensive, unreliable links, and the design shows in the byte counts. A whole CONNECT packet, built by hand in the appendix script:

0000  10 15 00 04 4d 51 54 54 04 02 00 3c 00 09 73 65  ....MQTT...<..se
0010  6e 73 6f 72 2d 31 37                             nsor-17

10 is the packet type (CONNECT), 15 is the remaining length (21 bytes), then the protocol name MQTT, level 4 (which means MQTT 3.1.1), the flags, a 60-second keep-alive and the client id. A publish of a temperature reading with QoS 1:

0000  32 14 00 0c 70 6c 61 6e 74 2f 33 2f 74 65 6d 70  2...plant/3/temp
0010  00 01 32 31 2e 35                                ..21.5

Twenty-two bytes carry the topic, a packet id and the reading 21.5; the acknowledgement (PUBACK) is four bytes. The leanest possible HTTP/1.1 POST of the same four bytes is 103 bytes before any response, any TLS and any retry. On a battery-powered sensor reporting every minute over a cellular link, that ratio is the product decision.

The parts of MQTT you must understand before using it:

FeatureWhat it doesThe catch
QoS 0fire and forgetlost on any hiccup, and that is often fine for a sensor
QoS 1at least once, acknowledgedduplicates, so the receiver must be idempotent
QoS 2exactly once, four-step handshakethe slowest and the most state per message; rarely worth it
Retained messagethe broker keeps the last message per topic and gives it to new subscribersit is not history, only the newest value, and it persists until replaced
Last Will and Testamentthe broker publishes a message if a client disappearsthe only way others learn a device went offline
Keep-alivea heartbeat interval agreed at CONNECTa half the interval rule of thumb for the client's own timer
Clean session / session expirywhether subscriptions and queued messages survive a disconnecta persistent session on a chatty topic can make the broker store a lot
Wildcards+ matches one level, # matches the rest and must be last# at the root subscribes to everything, including other tenants' topics

MQTT 5 is worth asking for: it adds reason codes on every acknowledgement (so a rejected publish says why), user properties (arbitrary key/value headers, which is where a correlation id belongs), shared subscriptions ($share/group/topic, so a topic can be load balanced across workers), topic aliases, message expiry and real flow control. MQTT 3.1.1 answers most failures with a closed connection and no reason.

Note what MQTT does not have: no schema, no request/response (MQTT 5 adds a response-topic convention), no ordering guarantee across topics, and authorisation is the broker's job, usually per topic pattern. Topic design is the API design, and it is also the security boundary: tenant/{id}/# is the pattern that keeps tenants apart.

STOMP, and Kafka's protocol

STOMP is a text protocol (CONNECT, SEND, SUBSCRIBE, MESSAGE, ACK) that looks like HTTP and is trivial to implement. Its common home is a browser talking to a broker over a WebSocket, which is what Spring's messaging stack does.

Kafka does not speak AMQP or MQTT. It has its own binary, length-prefixed TCP protocol, and the model is different enough to matter: a partitioned, append-only log with consumer-managed offsets, rather than queues that drain. Choosing between a queue and a log is an architecture decision, not a protocol one; the queues guide covers it.

Where messaging fits, and where it does not

It fits work that may take longer than a request, traffic spikes that must be absorbed, several consumers of one event, systems that are not up at the same time, and devices on bad links. It does not fit a client waiting for an answer right now (a queue plus a reply queue is a slow, fragile RPC), simple internal calls between two services you deploy together, or any team that is not ready to monitor queue depth and consumer lag.

11. The survivors: SMTP, IMAP, FTP, SFTP and SSH

In short. Mail and file transfer are the protocols you inherit rather than choose. They are worth a page because the failure modes are specific, old and still cost people days.

SMTP

SMTP (RFC 5321) is a line-based conversation where every answer is a three-digit code. The shape of a submission:

S: 220 mail.example.com ESMTP ready
C: EHLO shop.example.com
S: 250-mail.example.com
   250-SIZE 36700160
   250-STARTTLS
   250 PIPELINING
C: STARTTLS            -> then the whole conversation restarts inside TLS
C: MAIL FROM:<orders@shop.example.com>
S: 250 OK
C: RCPT TO:<ops@acme.test>
S: 250 OK
C: DATA
S: 354 Start mail input; end with <CRLF>.<CRLF>
C: (headers, blank line, body)
C: .
S: 250 OK id=1A2B3C

The details that cause real incidents:

Reading mail is a different protocol: IMAP (IMAP4rev2, RFC 9051; ports 143 and 993) keeps the messages on the server with folders, flags and server-side search, while POP3 (110 and 995) is a download-and-delete model from a time when one person used one device.

FTP, FTPS, SFTP, SCP

FTP (RFC 959) uses two connections: a control channel on port 21 for commands, and a separate data connection per transfer. That is its whole problem. In active mode the server connects back to a port the client names with PORT, which almost no firewall or NAT allows any more; in passive mode (PASV, or EPSV for IPv6) the server names a port and the client connects, which works but means the server must expose a port range. Half of all FTP support tickets are "the listing works, the transfer hangs", and that is active mode meeting a NAT.

FTPSSFTPSCP
What it isFTP with TLS (AUTH TLS, RFC 4217)a file-transfer subsystem of SSH, on port 22a copy command, historically its own protocol
Relation to FTPthe same protocol, encryptedunrelated, despite the nameunrelated
Ports21 plus a data rangeone, 2222
Firewall friendlinesspoor (the data range)goodgood
Featuresdirectory listing, resumelisting, resume, attributes, partial reads and writescopy only

The practical advice is short: use SFTP. It is one port, authenticated and encrypted by SSH, scriptable, and it is what partners mean nowadays when they say "FTP". OpenSSH 9.0 even made scp use the SFTP protocol underneath, because the old SCP protocol had a long history of surprises. Use FTPS only when the other side insists, and if you must run an FTP server, run passive mode with a fixed, open port range and log which mode each client used.

SSH, underneath all of that

SSH (RFC 4251 to 4254) is three layers: a transport layer that negotiates algorithms, exchanges keys and authenticates the server with its host key; a user authentication layer (public key, password, keyboard-interactive); and a connection layer that multiplexes channels over the one encrypted stream. Channels are why one SSH connection can carry a shell, a file transfer and three forwarded ports at once.

Two things follow from that structure. The host key check is trust on first use: the fingerprint saved in known_hosts on the first connection is what later warns you about a man in the middle, so blindly accepting a changed key throws the protocol's main guarantee away. And port forwarding (-L, -R, -D) is a channel like any other, which is why SSH is the universal tool for reaching a database that is not exposed, and also why it is the first thing a security review looks at.

12. The concerns that cut across all of them

In short. Ports, authentication, error vocabulary, versioning, tracing and what to measure look different in each protocol but are the same six problems. Knowing the mapping is most of what "experienced with protocols" means.

Ports, in one table

ProtocolPlainWith TLSNote
HTTP80443443 is the only port you can rely on crossing a corporate network
WebSocket80 (ws://)443 (wss://)it is HTTP until the 101
gRPC—443, or any port internallyHTTP/2, so the same ports as HTTP
SMTP25 relay, 587 submission465 submission, or STARTTLS on 587outbound 25 is blocked nearly everywhere
IMAP / POP3143 / 110993 / 995
FTP21 plus data21 with AUTH TLS (FTPS)the data channel is the problem, not the port
SSH, SFTP2222 (SSH is always encrypted)one port for shell, transfer and forwarding
AMQP 0-9-156725671RabbitMQ's management UI is 15672, a different thing
MQTT18838883MQTT over WebSocket uses the web ports, or one the broker picks
DNS53853 (DoT), 443 (DoH)see the DNS guide

Authentication, by protocol

ProtocolUsual mechanismsThe sharp edge
HTTP, REST, GraphQLAuthorization: Bearer (RFC 6750), Basic (RFC 7617), cookies, API keys, mTLSa token in a URL query parameter ends up in logs, caches and referrers
gRPCmetadata headers (authorization), or mTLS with per-channel credentialsmetadata is per call, but a channel is long-lived: refresh tokens need a call-time credential, not a connection-time one
SOAPWS-Security headers (username token, X.509 signature), or HTTP Basic plus TLSmessage-level signatures survive brokers and archives; transport auth does not
WebSocketcookie on the handshake, token in the first message, or a short-lived ticket in the URLbrowsers cannot set headers on the handshake, so the design is forced
AMQP, MQTTusername and password at connect, client certificates, broker ACLs per queue or topicauthorisation is the broker's, not the application's: get the topic or queue patterns right
SMTPSASL on the submission portauthentication says who may send, not what address they may claim
SSH, SFTPpublic key, certificates, (passwords)the host key is the client's only protection; an accepted change is an accepted attacker

Sessions, tokens and their lifetimes are their own subject: authentication and sessions.

The same failure, said six ways

This is the table I wish I had when I started. One row is one real situation, and the columns are how each protocol expresses it.

SituationHTTP / RESTgRPCGraphQLJSON-RPCSOAPMQTT 5
Malformed request4003 INVALID_ARGUMENT400, or 200 with errors and no data (section 7)−32600 / −32602Sender faultreason code 0x81
Not authenticated40116 UNAUTHENTICATED200 with an auth errorcustomsecurity fault0x87 at connect
Authenticated, not allowed4037 PERMISSION_DENIEDerrors with a path and null fieldcustomSender fault0x87 on publish
Thing does not exist4045 NOT_FOUNDnull plus an errorcustomapplication fault—
Conflict with current state409 / 41210 ABORTED, 9 FAILED_PRECONDITIONerrorscustomapplication fault—
Too many requests429 with Retry-After8 RESOURCE_EXHAUSTEDerrorscustom0x97 quota exceeded—
Server broke50013 INTERNALerrors−32603Receiver fault—
Temporarily down503 with Retry-After14 UNAVAILABLEerrors−32603Receiver faultconnection closed
Took too long5044 DEADLINE_EXCEEDEDerrorstimeouttimeoutkeep-alive timeout

Read the columns and two protocol-level lessons appear. First, the protocols that separate "do not retry" from "retry later" make clients much easier to write: HTTP (4xx against 429, 503) and gRPC (INVALID_ARGUMENT against UNAVAILABLE) do; GraphQL and MQTT 3.1.1 mostly do not, so you must encode it yourself in extensions or in a payload field. GraphQL's status line says something only once you negotiate the newer media type, and even then only about request errors. Second, a protocol that answers 200 or closes the connection for everything forces every client to parse prose, which is how retry storms begin.

Versioning, by protocol

ProtocolHow a version is expressedAdditive changeBreaking change
RESTpath (/v2/orders), media type (application/vnd.shop.v2+json), or a headeradd a field, ignore unknown onesa new version, both served for a while
GraphQLnormally none: one evolving schemaadd fields and types@deprecated, measure usage, then remove
gRPC, protobufthe package name (shop.v2)add a field with a new numbera new service or package; never reuse a field number
SOAPa new target namespace and WSDL URLa new optional element, if the schema allows ita new namespace; the old one keeps running
JSON-RPCthe method name (orders.get.v2)add an optional parametera new method name
MQTT, AMQPthe topic or routing key, and a version field in the payloada new field, if the consumer tolerates ita new topic, with both published during the change

The pattern across all of them: adding is cheap, removing is a project. Every protocol above gives you a way to add and a way to run two versions side by side; none gives you a way to delete something clients still use.

Correlation and tracing

One field, carried everywhere, is what makes a request traceable across protocols. The W3C Trace Context header is the common form:

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
ProtocolWhere it goes
HTTP, REST, GraphQLthe traceparent header
gRPCmetadata, which is HTTP/2 headers, so the same name
AMQPa message header or application property
MQTT 5a user property; in 3.1.1 there is nowhere but the payload
SOAPa header element, usually WS-Addressing's MessageID
Webhooksa header the sender defines; log their event id as well as yours

The reason to care is that a request crossing four protocols with no shared id is four separate investigations (observability).

What to measure, per protocol

No guide can tell you your numbers. It can tell you which four signals are worth a dashboard.

ProtocolWatch
HTTP, RESTrequest rate, latency percentiles per route, status-class rates, and the share of 304s and cache hits
GraphQLthe share of responses carrying errors, resolver time per field, query depth and cost, and the top queries by cost
gRPCstatus-code counts (not just error rate), deadline-exceeded share, message sizes, and per-connection call distribution
WebSocket, SSEopen connections, connections per node, message rate, ping failures, and send-buffer depth
Webhooks (sending)delivery success, attempts per event, age of the oldest pending event, endpoints disabled
AMQP, MQTTqueue depth and consumer lag, redelivery counts, dead-lettered messages, and broker memory
SMTPaccepted against deferred against bounced, and DMARC reports

13. Choosing

In short. Pick by who starts the conversation, who the clients are, and how strict the contract must be. In that order.

Diagram: Choosing a protocol

The same thing as a table, with the cost you are accepting:

SituationUseWhat you accept
A public or partner-facing APIREST over HTTP with JSONno schema enforcement unless you add OpenAPI and check it in CI
Internal calls, many per second, several languagesgRPCa build step, a schema registry, and proxies that must handle HTTP/2 trailers
One backend, several very different front endsGraphQLcaching, cost control and authorization become your code
A bank, an insurer, a government body on the other sideSOAPverbosity, and 5xx on business errors
Live updates to a browserSSE, then WebSocket if you need upstreamlong-lived connections to operate and scale
Another company must tell you about eventswebhookssignature checks, replays, duplicates and no ordering
Work that must not be lost and may be slowAMQP or a loga broker to run, and queue depth to watch
Battery-powered or metered devicesMQTTno schema, and topic design as your security boundary
Files between companiesSFTPkey management, and a transfer that is not a transaction
MailSMTP, through a providerdeliverability is DNS and reputation, not code

Three defaults worth stating plainly, because they save the most time:

Migrating without a rewrite

FromToThe usual route
SOAPREST or gRPCkeep the SOAP endpoint, add the new one in front of the same service layer, move clients one at a time, and keep a generated SOAP client for the partners who will never move
RESTgRPC internallyintroduce it for one chatty pair of services, keep REST at the edge; a gateway can translate (grpc-gateway, Envoy)
REST pollingSSE or WebSocketship SSE for the feed first and keep polling as the fallback; the client code for both is similar
Polling a partnerwebhooksaccept webhooks as a nudge, keep the polling job at a low frequency as a safety net
A queue used as RPCgRPC or HTTPif the caller waits for the answer, it was never messaging

14. Common mistakes

15. Things to mention in an interview

Quick recap

Appendix: the demonstration script

Everything quoted as bytes in this guide came from one script: protocols.py, Python 3.13 and the standard library, plus curl for the HTTP part and the protobuf package for the one cross-check that a real library parses the bytes shown. It starts a local HTTP server on 127.0.0.1, so it touches no network, and it prints each exchange, hex dump and byte count in the order the guide quotes them.

# 1. what a client sends: a local socket server prints the exact request bytes,
#    answers one GET with an ETag, and answers the conditional GET with 304.
# 2. chunked framing, built by hand: hex length, bytes, CRLF, then a zero chunk.
# 3. the SSE wire format, written out in full.
# 4. the WebSocket handshake answer and a masked text frame:
GUID = b"258EAFA5-E914-47DA-95CA-C5AB0DC85B11"          # RFC 6455
accept = base64.b64encode(hashlib.sha1(key + GUID).digest())
masked = bytes(b ^ mask[i % 4] for i, b in enumerate(payload))
frame = bytes([0x81, 0x80 | len(payload)]) + mask + masked   # FIN, text, masked
# 5. the same record as compact JSON, as a SOAP 1.2 envelope and as protobuf,
#    with gzip sizes; the protobuf message is built with a descriptor made at
#    run time, so the bytes come from the real implementation, and
#    Order.FromString(as_pb) == pb checks the round trip.
# 6. gRPC framing: struct.pack(">BI", 0, len(as_pb)) + as_pb
# 7. MQTT 3.1.1 CONNECT and a QoS 1 PUBLISH, with the remaining-length varint:
def remaining_length(n):
    out = bytearray()
    while True:
        d, n = n % 128, n // 128
        out.append(d | (0x80 if n else 0))
        if not n:
            return bytes(out)

Two things the script deliberately does not do: it does not measure throughput or latency (a loopback socket would tell you nothing about a real network), and it does not implement any protocol fully. It produces the bytes the specifications describe, so that the numbers in this guide can be checked rather than believed.

Sources

Practise this topic: Networking & APIs

More guides