CS Fundamentals Trainer · Guides
DNS, HTTP/2, HTTP/3 and CDNs
How a name is resolved and cached, what a new connection costs, how HTTP/2 multiplexing and HTTP/3 over QUIC remove head-of-line blocking, and how a CDN's hit ratio, cache keys and headers decide its value.
32 min read
Before a browser can show a page it has to find the server (DNS), open a connection and agree on encryption (TCP and TLS, or QUIC), ask for the page and its dozens of subresources (HTTP), and the bytes have to travel across the world (the network, helped by a CDN). Each step adds round trips, and a round trip across an ocean costs more than any amount of clever code on either side. This guide follows the request in that order and explains what DNS caching, HTTP/2, HTTP/3 and CDNs each change, with experiments I ran for the numbers.
The shorter walk-through of the whole journey is in what happens when you type a URL, and the encryption part is in TLS and HTTPS. This guide goes deeper into the three places where performance and operations are decided.
1. DNS
The Domain Name System maps names to records, most often an IP address. It is a distributed, hierarchical, cached database.
Diagram: Resolving www.example.com through a recursive resolver
Step through it, with and without a cache:
Interactive animation: dns lookup
With an empty cache the lookup took four round trips (browser to resolver, then root, .com and the authoritative server); with the answer cached at the resolver it took one. Real resolvers also cache the delegations (the NS records for .com), so a lookup for a new name under a known domain usually skips the root and TLD steps.
What is on the wire
A DNS message is small. I built a query for www.example.com by hand following RFC 1035: a 12-byte header (id, flags, counts) followed by the name as length-prefixed labels (03 www 07 example 03 com 00), then the record type (1 for A) and class (1 for IN). The whole query is 33 bytes, and a response with one A record is 49 bytes, because the answer's name is a 2-byte pointer back to the question (c0 0c) instead of repeating it. DNS normally travels over UDP port 53, with a fallback to TCP when the response is too large or has been truncated. Encrypted variants (DNS over TLS on port 853, DNS over HTTPS) hide the queries from the network between the client and the resolver; the resolver still sees them.
Record types worth knowing
| Type | Meaning |
|---|---|
| A / AAAA | IPv4 / IPv6 address |
| CNAME | an alias: "this name is really that other name". Not allowed at the zone apex next to other records |
| NS | which servers are authoritative for a zone |
| MX | mail servers for a domain |
| TXT | free text, used for domain verification, SPF, DKIM |
| SOA | zone metadata, including the TTL for negative answers |
| SRV / HTTPS (SVCB) | service location and parameters, such as the ports and protocols (for example HTTP/3) a service supports |
A negative answer (the name does not exist) is cached as well, for a time given by the zone's SOA record (RFC 2308), so a typo in a hostname that your service retries aggressively does not reach the authoritative server each time.
TTL: the dial between speed and agility
Every record has a time to live that tells resolvers how long they may reuse it. A short TTL lets you move traffic quickly; a long one saves queries and speeds up clients. In a simulation of one name queried about twice a second, a recursive resolver sent this many queries upstream in a day:
| TTL | Upstream queries per day |
|---|---|
| 0 s (no caching) | 172,304 (every lookup) |
| 5 s | 15,702 |
| 60 s | 1,429 |
| 300 s | 288 |
| 3600 s | 24 |
And when you change a record, the old answer lives on in caches until its TTL expires. If the resolvers' copies were fetched at random moments, they update evenly over the old TTL: with a 300-second TTL half of them see the new value within about 150 s and the last one after 300 s; with a 24-hour TTL the same spread takes a day. Browsers, operating systems and some applications (the JVM famously) add their own caches, and some resolvers ignore small TTLs or keep serving stale answers when the authority is unreachable (RFC 8767). So:
- Lower the TTL before a planned change, at least one old-TTL ahead of time, and raise it afterwards.
- DNS-based failover is slow and coarse. A health check can change the answer, but clients with the old answer stay with it until it expires, and some never leave their open connection. It is a useful layer, not a fast one.
- Do not put dynamic load balancing into long TTLs. See load balancing for the layers that react faster.
DNS is also a classic hidden dependency: if the resolver is slow, every request is slow; if the authoritative provider is down, your site is down even though your servers are fine. Use more than one nameserver, ideally at independent providers or at a provider that is itself spread across networks, and apply the timeouts and retries of the resilience guide to resolver calls too.
2. Opening the connection: round trips
Before the first byte of the page can be requested, the client must set up a transport and encryption. Counting round trips (RTT) from the start of the connection:
| Stack | Round trips before the request can be sent |
|---|---|
| HTTP/1.1 or HTTP/2 over TCP and TLS 1.2 | 3 (TCP: 1, TLS 1.2: 2) |
| TCP and TLS 1.3 | 2 (TCP: 1, TLS: 1) |
| QUIC (HTTP/3), new server | 1 (transport and TLS 1.3 handshake combined) |
| QUIC with 0-RTT resumption | 0, with the request in the first flight (replayable, so only for safe requests) |
If the round trip to the server is 100 ms, the difference between 3 and 1 is 200 ms before anything useful happens, on every new connection, plus the DNS lookup.
Read the table as the idealised minimum for a first connection, because it assumes things that are often not true:
- No packet loss. A lost handshake packet costs a retransmission timeout, which is much longer than one round trip (an initial timeout of a second or so is common for TCP), so the tail of the connection-time distribution is far worse than the median.
- A full handshake with nothing remembered. TLS 1.2 with session resumption needs one round trip, not two. TLS 1.3 resumption saves the certificate exchange, and TLS 1.3 with early data allows a request in the first flight (with the same replay risk as QUIC 0-RTT). TCP Fast Open can carry data in the SYN, though it is rarely deployed across the open internet.
- A certificate chain that fits. A long chain or a large certificate can take more than the initial congestion window and cost an extra round trip, on TCP and on QUIC (QUIC servers may send at most about three times what they have received from an unvalidated client address, which also limits the first flight). Revocation checks (OCSP) and other validation steps can add time, unless the server staples the response.
- The client already knows to use HTTP/3. A browser normally learns that a server speaks HTTP/3 from an
Alt-Svcheader or anHTTPSDNS record, so the first visit commonly goes over TCP and TLS, and later visits use QUIC. Some clients race the two. - An existing connection is reused. Connection reuse, multiplexing and connection coalescing (using one HTTP/2 or HTTP/3 connection for several hostnames that share a certificate and address) mean most requests on a real page need no handshake at all.
- The RTT is the whole story. Cryptography, server processing, middleboxes, DNS (a separate lookup, cached or not) and the first bytes of slow start all add to it, and the round trip itself varies from one request to the next.
The counts are therefore a way to reason about where time goes, not a prediction of how long a connection will take. To know your own numbers, measure them (browser developer tools, curl -w with its timing variables, real user monitoring), at the percentiles that matter.
Two practical consequences: reuse connections (keep-alive, HTTP/2 multiplexing, connection pools), and put the server close to the user, which is what a CDN does.
3. HTTP/1.1, HTTP/2 and HTTP/3
A page is not one request; it is the HTML plus scripts, styles, images and API calls, often 50 or more.
HTTP/1.1 carries one request at a time per connection. Pipelining existed on paper but was never reliably usable. Browsers worked around the limit by opening up to six connections per host and by "sharding" assets across several hostnames, both of which cost handshakes and congestion control per connection.
HTTP/2 (RFC 9113) keeps the same semantics (methods, status codes, headers) but changes the framing:
- Binary framing and multiplexing: many request/response streams share one connection, their frames interleaved.
- Header compression (HPACK): repeated headers such as cookies and user agents are sent as references to a table.
- Stream prioritisation (a hint to the server about which resources matter most; implementations differ widely).
- Server push existed in the specification, but proved hard to use well and browsers have removed it. Do not design around it.
To see the effect I served 60 small responses, each taking 50 ms to produce on the server, from a local Node server and fetched them all:
| Client | Time for 60 requests |
|---|---|
| HTTP/1.1, one connection | 3,058 ms (60 times 50 ms, one after the other) |
| HTTP/1.1, six connections | 515 ms (ten rounds of six) |
| HTTP/2, one connection | 59 ms (all in parallel) |
This test uses a fixed delay on the server in place of network latency and runs over loopback without encryption (cleartext HTTP/2 is only for testing; browsers use it over TLS), so it shows the queueing effect, not real page-load times. It explains why the practices of the HTTP/1.1 era (domain sharding, bundling everything into one file, spriting) matter less or hurt under HTTP/2.
The remaining problem: head-of-line blocking
HTTP/2 multiplexes streams, but it is the transport underneath it that causes the trouble, not HTTP/2's own framing. All the streams ride one TCP connection, and TCP delivers a single ordered byte stream to the application. If one TCP segment is lost, TCP holds back every byte that arrived after it, belonging to any stream, until the retransmission fills the gap. HTTP/2 cannot see the segments, so streams that lost nothing stall too. (This is transport-level head-of-line blocking. HTTP/1.1 had its own, at the HTTP level: a slow response blocked the next one on the same connection. HTTP/2 removed that one and left the TCP one.) Whether a single loss hurts a lot depends on how much data of how many streams was in flight behind it, which is why pages with many parallel streams on lossy links suffer most.
Diagram: One lost packet and what it blocks
Step through the same situation, first with one ordered connection and then with independent streams:
Interactive animation: head of line
In the animation, three streams send a packet per tick and one packet of stream A is lost. Independent streams are better, not free of cost: the lost data still has to be retransmitted, the stream that lost it still waits, and (next section) all the streams still share one connection's congestion control. With one ordered connection, packets spent 26 ticks in total waiting behind someone else's loss; with independent streams only 6, the ones in stream A itself. I also ran a larger simulation, with random loss and a retransmission delay of 5 ticks, and measured the average time a packet waits between arriving and being handed to the application:
| Packet loss, 3 streams | Ordered connection (TCP) | Independent streams (QUIC) |
|---|---|---|
| 0.5% | 0.32 ticks | 0.08 ticks |
| 1% | 0.54 | 0.15 |
| 2% | 0.72 | 0.22 |
| 5% | 1.63 | 0.55 |
And at 1% loss, with a growing number of streams, the ordered connection gets worse (0.10, 0.54, 1.33 and 2.67 ticks for 1, 3, 10 and 30 streams) while independent streams stay around 0.10. With one stream there is no difference: the damage is proportional to the number of streams that have to wait for each other. This is a model, not a network measurement. It treats loss as independent per packet, gives each packet one stream, ignores congestion control (so it cannot show that every stream slows down after a loss in either protocol), and counts waiting in ticks. Real gains depend on the loss pattern, the congestion control, the size and number of the streams and the page, and field measurements of HTTP/3 against HTTP/2 show benefits that are real but modest at the median and larger in the tail on poor networks.
HTTP/3 and QUIC
HTTP/3 (RFC 9114) runs HTTP over QUIC (RFC 9000), a transport built on UDP that provides:
- Independent streams, so a loss delays only the streams whose data was in the lost packet. QUIC orders data per stream, not for the whole connection, so another stream's data that arrived can be handed to the application at once. Three caveats keep this from being a complete cure: a packet can carry frames of several streams, and then each of them waits for its retransmission; all streams on a connection still share one congestion controller and connection-level flow control, so a loss makes the sender slow down for every stream; and a loss in the handshake or in connection-control data can stall the whole connection. The gain is largest with several concurrent streams, on lossy networks, with bursty loss avoided by good congestion control.
- TLS 1.3 built in: the transport and cryptographic handshakes are combined, giving the 1-RTT (and 0-RTT) connection setup in the table above. Almost all of the packet is encrypted, including much of the transport's own metadata.
- Connection IDs, so a connection survives a change of IP address, for example a phone switching from Wi-Fi to mobile data, whereas TCP is tied to the 4-tuple of addresses and ports.
- Header compression with QPACK, the HTTP/3 counterpart of HPACK, designed to work with streams that arrive out of order.
It is not free. QUIC runs in user space and is more CPU-intensive than the kernel's TCP and its offloads, some networks block or throttle UDP, and 0-RTT data can be replayed by an attacker, so servers should accept it only for requests that are safe to repeat. Clients therefore discover HTTP/3 and fall back: a server announces it with an Alt-Svc header (or the HTTPS DNS record), the first visit usually uses HTTP/2 over TCP, and later ones try QUIC and race or fall back if UDP does not work. For you as an application developer, the semantics do not change; what changes is how you configure the edge and what you monitor.
4. CDNs
A content delivery network is a set of caching servers (edge or points of presence) spread across the world. Users connect to a nearby edge; if the edge has the object it answers at once (a hit), and if not it fetches it from the origin (a miss), keeps a copy and answers.
Diagram: A request through a CDN
Edges bring three benefits: latency (the TCP and TLS handshakes end at a server a few milliseconds away, and the long haul to the origin uses a warm, reused connection), offload (the origin sees only the misses) and absorption of traffic spikes and some attacks. An origin shield is an extra cache tier in front of the origin so that a hundred edges asking for the same missing object cause one request to the origin, not a hundred.
How the user reaches the nearest edge
Either DNS answers differently depending on where the resolver is (the CDN's name servers return the address of a nearby edge), or the CDN uses anycast, announcing the same IP address from many locations so that routing delivers the packets to the nearest. Many use both. A consequence of the DNS approach is that the "nearest" is chosen from the resolver's location, which is not always the user's.
What the hit ratio depends on
The mean latency is simple arithmetic: with a hit at 20 ms and a miss at 140 ms (20 ms to the edge and 120 ms to the origin), a 50% hit ratio gives 80 ms, 80% gives 44 ms, 95% gives 26 ms and 99% gives 21 ms. The origin load is the share of misses: 50%, 20%, 5%, 1%. The last few percentage points of hit ratio are worth a lot to the origin.
What determines the ratio? I simulated 200,000 requests to 10,000 objects where some are far more popular than others (a Zipf distribution), with an LRU cache at one edge:
| Edge cache holds | Hit ratio, strongly skewed popularity | Hit ratio, flatter popularity |
|---|---|---|
| 0.1% of the catalogue | 13.1% | not measured |
| 1% | 39.1% | 4.3% |
| 5% | 58.7% | 15.1% |
| 20% | 76.8% | 38.0% |
A cache holding 1% of the objects served 39% of the requests when popularity is concentrated, and only 4% when it is flat. This is why CDNs are excellent for popular static assets and much less so for the long tail (a huge catalogue of rarely requested objects), where the edge has to ask the origin almost every time, and why a shield tier and larger caches matter for that case. The same simulation with a time limit on cached objects (a TTL measured in requests, with a 20% cache) dropped the hit ratio from 76.8% to 69.1%, 46.2% and 23.3% for TTLs of 5000, 500 and 50 requests: expiry throws away useful copies. When the origin has not changed, a conditional request (If-None-Match with an ETag) lets the edge revalidate cheaply, with a 304 response and no body, rather than download again.
Controlling the cache
The edge obeys the HTTP caching headers your origin sends (RFC 9111):
Cache-Control: public, max-age=31536000, immutablefor fingerprinted assets (a file name with a hash of its content): cache for a year, never revalidate.Cache-Control: s-maxage=60to set a different lifetime for shared caches (CDNs) than for browsers (max-age).stale-while-revalidateandstale-if-errorto serve an expired copy while refreshing it in the background, or when the origin is failing. This is a cheap resilience measure.Varytells caches that the response depends on a request header (Accept-Encoding,Accept-Language). Each distinct value is a separate cached copy, soVary: CookieorVary: User-Agentdestroys the hit ratio.Cache-Control: privateorno-storefor personalised or sensitive responses; a shared cache that stores a response meant for one user can show it to another, a classic and serious bug. Watch also the cache key: if it ignores something the response depends on (a query parameter, a header), users get each other's variants.
Invalidation: purging is slow, partial and racy
Invalidation is the hard part (as ever with caches; see caching strategies). A CDN offers several tools, and each has limits:
- Purge by URL, by prefix or by tag (also called surrogate keys: the origin labels responses, and one purge by label removes them all). A purge is a message sent to every edge and shield, so it is eventually applied: from about a second to minutes, depending on the provider, and for a short time some locations still serve the old copy. Read what your provider promises, and do not assume that a purge is instant or that it succeeded everywhere.
- Soft purge (mark as stale) keeps the object but makes the edge revalidate or refetch it. It protects the origin from a stampede, and works well with
stale-while-revalidate. - The race with the origin. If you purge before the origin serves the new version (a replica that is still behind, a deploy that is still rolling out), an edge fetches the old content and caches it again. Update the origin first, make sure it is consistent, then purge.
- Several cache layers. The edge, the origin shield, your own reverse proxy and the user's browser each hold a copy. A purge does not reach browsers, so a page cached there for an hour stays for an hour. Layers also have to be purged in an order that does not refill one from another.
- Negative and error responses can be cached too (a 404 or an error page for a short time, which is usually good for the origin and bad if the error was transient). Check which status codes your CDN caches and for how long.
- Request collapsing. When a popular object expires, hundreds of requests can arrive at once; CDNs can collapse them into one origin request, but the others wait for it, and a slow origin means slow responses for all of them.
So the safest strategy is to make assets immutable by changing their URL for each version, keep HTML on short lifetimes (or stale-while-revalidate), and keep purge for the cases that really need it (an incorrect page, removed content, a legal request), with a check that it worked (request the object from several regions and compare Age and the CDN's cache-status header).
Personalised and private responses
A shared cache stores one response and serves it to many people, which is exactly wrong for anything that depends on who is asking. Be explicit:
- Mark responses with their audience.
Cache-Control: privateallows only the user's own browser to keep it;no-storemeans nobody should store it (use it for sensitive data such as account pages, tokens, health or financial details, becauseprivatestill lets the browser write it to disk). A response to a request with anAuthorizationheader is not stored by a shared cache unless the origin explicitly allows it (public,s-maxageormust-revalidate), and responses that set cookies should be treated with great care. Do not rely on defaults: many CDNs cache by file extension or path rules unless told otherwise, and some override the origin's headers through their own configuration (an "edge cache TTL" rule), which can silently cache something the origin marked private. - Make the cache key match the response. If the content depends on a cookie, a header, the language, the device type or a query parameter, either the key includes it (and every distinct value becomes a separate copy, so keep the set small and normalised), or the response must not be cached. A key that ignores something the response depends on gives users each other's content.
- Know the two classic attacks. In web cache deception an attacker gets a victim to open a URL such as
/account/settings/x.css; the origin ignores the extra segment and returns the victim's private page, while the CDN sees a "static" file and caches it, so the attacker then requests the same URL and receives the private page. In web cache poisoning a request header that the origin uses but the cache key leaves out lets an attacker store a harmful response that is then served to others. Defences: do not let path suffixes or extensions decide cacheability, make the origin return a 404 for paths that do not exist, sendCache-Control: no-storeon every authenticated page by default, and key on everything the response uses. - Personalise at the edge of the page, not the whole page. Cache the shared shell (or the shared API response) and load the personal part (a user name, a cart count) with a separate
no-storerequest or in the browser. Where private files are served through a CDN (invoices, downloads), use short-lived signed URLs or cookies and make the signature part of access control, not of the cache key alone, and keep the cache from serving them to anyone who guesses the URL. - Leaks happen outside the cache too. CDN and edge logs hold URLs, IP addresses and headers, so keep tokens and personal data out of URLs and query strings, and check the retention and access rules of those logs.
- Test it. Request the same URL as two different users and as an anonymous visitor; check that the second user never receives the first one's content, and look at the cache-status header (
Age,X-Cacheor the provider's equivalent) on authenticated routes: anyHITon a personal page is a bug to fix before it is an incident.
Many CDNs also terminate TLS, apply a web application firewall and rate limiting at the edge (compare designing a rate limiter), and run small functions at the edge. Terminating TLS means the CDN can read all the traffic, so it is part of your trust boundary and your privacy obligations. For dynamic, uncacheable requests the CDN still helps by terminating the handshake near the user and carrying the request to the origin over a persistent, optimised connection.
5. Common mistakes
- Setting a very long DNS TTL on a record you may need to move, or forgetting to lower it before a migration.
- A single DNS provider or a single nameserver as a hidden single point of failure.
- Relying on DNS failover for fast recovery.
- Keeping HTTP/1.1 workarounds (many domains, giant bundles) after moving to HTTP/2 or 3.
- Assuming HTTP/3 makes everything faster; it helps most on lossy and mobile networks and for new connections, and gains for a fast wired client are small.
- Caching personalised responses at the edge, or a cache key that omits a varying header; letting path suffixes decide that an authenticated page is "static"; using
privatewhereno-storeis needed. - Purging before the origin is updated, assuming a purge is instant, or forgetting that browsers keep their own copy.
Vary: Cookieor no caching headers at all, which makes the CDN a pass-through.- Purging as the only invalidation strategy, instead of versioned URLs.
- Measuring a CDN with one repeated request: the first request to each edge is a miss.
6. Things to mention in an interview
- The resolution path (stub, recursive resolver, root, TLD, authoritative) and that caching with TTL is what makes DNS scale; the trade-off between short and long TTLs.
- Round trips before the first request: TCP plus TLS compared with QUIC, and why connection reuse matters.
- What HTTP/2 fixes (multiplexing and header compression) and what it leaves (TCP head-of-line blocking), and how QUIC reduces that with per-stream ordering over UDP, with its caveats (shared congestion control, several streams in one packet).
- A CDN as a geographically spread cache: hit ratio, origin shield, cache keys and
Vary,Cache-Control, versioned URLs against purges, how purging really behaves, and how to keep personalised and private responses out of shared caches (including cache deception and poisoning). - That the benefit of caching depends on the popularity distribution, and that the long tail is the hard case.
Quick recap
- DNS is a cached hierarchy: four round trips cold (browser to resolver, root, TLD, authoritative), one when cached. TTL trades agility for load and speed (172,304 queries a day with no caching, 288 with 300 s), and a change takes up to the old TTL to be seen everywhere.
- A new connection costs 2 RTT with TCP and TLS 1.3, 1 with QUIC and 0 with resumption (replayable), as idealised minimums that assume no loss, no cached state beyond what is stated and a certificate that fits; connection reuse removes most handshakes in practice.
- HTTP/2 multiplexes streams on one connection (59 ms against 515 ms and 3,058 ms in the test) but TCP's single ordered byte stream lets one lost segment stall every stream; HTTP/3 over QUIC orders data per stream, so unrelated streams are not held back, though streams in one packet and the shared congestion control still link their fate.
- A CDN moves content close to users; its value is the hit ratio, which depends on popularity skew, cache size, TTLs and a correct cache key. Invalidation is eventual (so use versioned URLs), and personalised or private responses need
no-storeorprivate, a correct cache key and tests.
Appendix: the experiment code
Python 3 (standard library only) and Node.js 22 were used. The round-trip counts in section 2 are arithmetic on the protocol's message flights (see the caveats there), not measurements. The DNS, head-of-line and CDN scripts use fixed random seeds and give the same numbers each run; the HTTP/2 test measures real time on loopback, so its milliseconds vary a little between runs. The HTTP/2 server uses a fixed 50 ms delay per response in place of network latency, and cleartext HTTP/2 (which browsers do not use); the head-of-line and CDN scripts are simple models. They illustrate the mechanisms and are not measurements of any real network or CDN.
Names on the wire, resolver caching and TTL
import struct, random
# ---- 1. what a DNS query looks like on the wire (RFC 1035)
def encode_name(name):
return b"".join(bytes([len(p)]) + p.encode() for p in name.rstrip(".").split(".")) + b"\x00"
def build_query(name, qtype=1, txid=0x1234):
header = struct.pack("!HHHHHH", txid, 0x0100, 1, 0, 0, 0) # recursion desired, 1 question
return header + encode_name(name) + struct.pack("!HH", qtype, 1)
q = build_query("www.example.com")
print(len(q), "bytes:", q.hex(" "))
def build_response(name, ip, ttl):
header = struct.pack("!HHHHHH", 0x1234, 0x8180, 1, 1, 0, 0) # response, recursion available, 1 answer
question = encode_name(name) + struct.pack("!HH", 1, 1)
answer = b"\xc0\x0c" + struct.pack("!HHIH", 1, 1, ttl, 4) + bytes(map(int, ip.split("."))) # \xc0\x0c = pointer to offset 12
return header + question + answer
def parse_response(data):
txid, flags, qd, an, _, _ = struct.unpack("!HHHHHH", data[:12]); i = 12
while data[i]: i += data[i] + 1 # skip the question name
i += 1 + 4
out = []
for _ in range(an):
if data[i] & 0xC0 == 0xC0: i += 2 # compressed name: a pointer
else:
while data[i]: i += data[i] + 1
i += 1
typ, cls, ttl, rdlen = struct.unpack("!HHIH", data[i:i+10]); i += 10
out.append((typ, ttl, ".".join(map(str, data[i:i+rdlen])))); i += rdlen
return out
r = build_response("www.example.com", "93.184.216.34", 300)
print(len(r), "byte response ->", parse_response(r))
# ---- 2. a recursive resolver with a cache: how many upstream queries do a day of lookups cause?
def resolver_day(ttl, lookups_per_s=2, seconds=86400, seed=1):
rnd = random.Random(seed); cache_until = -1; upstream = 0; t = 0.0
while t < seconds:
t += rnd.expovariate(lookups_per_s)
if t >= cache_until: upstream += 1; cache_until = t + ttl
return upstream
for ttl in (0, 5, 60, 300, 3600):
print(f"TTL {ttl:>4} s: {resolver_day(ttl):>6} upstream queries in a day for one name at ~2 lookups/s")
# ---- 3. changing a record: how long until clients see it? worst case = the old TTL
def switch_time(old_ttl, clients=2000, seed=2):
rnd = random.Random(seed)
seen_old = [rnd.uniform(0, old_ttl) for _ in range(clients)] # remaining TTL of each resolver's cached copy
seen_old.sort()
return seen_old[int(0.5*clients)], seen_old[int(0.99*clients)], seen_old[-1]
for ttl in (60, 300, 3600, 86400):
med, p99, mx = switch_time(ttl); print(f"old TTL {ttl:>5} s: half of the resolvers update within {med:.0f} s, 99% within {p99:.0f} s, the last after {mx:.0f} s")
HTTP/1.1 against HTTP/2 (Node.js)
import http from 'node:http';
import http2 from 'node:http2';
const DELAY = 50; // each response takes 50 ms to produce (stands in for one network round trip)
const N = 60; // a page with 60 small resources
const handler = (req, res) => setTimeout(() => { res.writeHead(200, { 'content-type': 'text/plain' }); res.end('x'.repeat(2000)); }, DELAY);
const s1 = http.createServer(handler).listen(0);
const s2 = http2.createServer(handler).listen(0); // HTTP/2 over cleartext (h2c); browsers require TLS, the multiplexing is the same
await new Promise((r) => setTimeout(r, 100));
async function h1(maxSockets) {
const agent = new http.Agent({ keepAlive: true, maxSockets });
const t = performance.now();
await Promise.all(Array.from({ length: N }, () => new Promise((ok) => http.get({ port: s1.address().port, agent, path: '/' }, (r) => { r.resume(); r.on('end', ok); }))));
const ms = performance.now() - t; agent.destroy(); return Math.round(ms);
}
async function h2run() {
const c = http2.connect(`http://localhost:${s2.address().port}`);
await new Promise((ok) => c.on('connect', ok));
const t = performance.now();
await Promise.all(Array.from({ length: N }, () => new Promise((ok) => { const r = c.request({ ':path': '/' }); r.resume(); r.on('end', ok); r.end(); })));
const ms = performance.now() - t; c.close(); return Math.round(ms);
}
await h1(6); await h2run(); // warm up
console.log(`HTTP/1.1, 1 connection : ${await h1(1)} ms for ${N} requests`);
console.log(`HTTP/1.1, 6 connections : ${await h1(6)} ms for ${N} requests`);
console.log(`HTTP/2, 1 connection : ${await h2run()} ms for ${N} requests`);
s1.close(); s2.close(); process.exit(0);
Head-of-line blocking under packet loss
import random
def run(loss, independent, streams=3, packets=200, rtt=5, seed=0):
"""Each tick every stream sends one packet. A lost packet is detected and resent after `rtt` ticks.
Returns the average number of ticks a packet waits between arriving and being handed to the application."""
rnd = random.Random(seed); arrive = [] # (stream, seq, arrival tick, send order)
order = 0
for seq in range(packets):
for s in range(streams):
lost = rnd.random() < loss
arrive.append((s, seq, seq + (rtt if lost else 0), order)); order += 1
waits = []
if independent: # QUIC-like: each stream is ordered separately
for s in range(streams):
ready = 0
for (_, seq, at, _) in [a for a in arrive if a[0] == s]:
ready = max(ready, at); waits.append(ready - at)
else: # TCP-like: one byte stream; everything after a hole waits for it
ready = 0
for (_, seq, at, _) in sorted(arrive, key=lambda a: a[3]):
ready = max(ready, at); waits.append(ready - at)
return sum(waits) / len(waits), max(waits)
for loss in (0.0, 0.005, 0.01, 0.02, 0.05):
t = run(loss, False); q = run(loss, True)
print(f"loss {loss*100:>4.1f}%: ordered connection avg wait {t[0]:.2f} ticks (max {t[1]}), independent streams {q[0]:.2f} (max {q[1]})")
for streams in (1, 3, 10, 30):
t = run(0.01, False, streams=streams); q = run(0.01, True, streams=streams)
print(f"1% loss, {streams:>2} streams: ordered {t[0]:.2f} ticks on average, independent {q[0]:.2f}")
A CDN cache under skewed and flat popularity
import random, bisect
from collections import OrderedDict
def zipf_sampler(n, s, rnd):
w = [1 / (i ** s) for i in range(1, n + 1)]; tot = sum(w); cum = []; a = 0
for x in w: a += x / tot; cum.append(a)
return lambda: bisect.bisect_left(cum, rnd.random())
def cdn(catalogue=10000, cache_share=0.05, requests=200000, s=1.0, ttl=None, seed=5):
rnd = random.Random(seed); pick = zipf_sampler(catalogue, s, rnd)
cap = int(catalogue * cache_share); cache = OrderedDict(); hits = 0; origin = 0
for t in range(requests):
k = pick()
if k in cache and (ttl is None or t - cache[k] < ttl):
hits += 1; cache.move_to_end(k)
else:
origin += 1; cache[k] = t; cache.move_to_end(k)
if len(cache) > cap: cache.popitem(last=False)
return hits / requests, origin
print("popular-heavy traffic (Zipf, s=1.0), 10,000 objects:")
for share in (0.001, 0.01, 0.05, 0.2):
h, o = cdn(cache_share=share); print(f" edge cache holds {share*100:>5.1f}% of the catalogue: hit ratio {h*100:4.1f}%, origin sees {o} of 200000 requests")
print("flatter traffic (Zipf, s=0.6):")
for share in (0.01, 0.05, 0.2):
h, o = cdn(cache_share=share, s=0.6); print(f" edge cache holds {share*100:>5.1f}% of the catalogue: hit ratio {h*100:4.1f}%")
print("short TTLs cost hits (cache 20%, s=1.0, TTL in requests):")
for ttl in (None, 5000, 500, 50):
h, o = cdn(cache_share=0.2, ttl=ttl); print(f" TTL {str(ttl):>5}: hit ratio {h*100:4.1f}%")
# average latency: hit 20 ms, miss 20 + 120 ms to the origin
for h in (0.5, 0.8, 0.95, 0.99):
print(f"hit ratio {h*100:.0f}%: mean latency {20*h + 140*(1-h):.0f} ms (hit 20 ms, miss 140 ms); origin load x{1-h:.2f}")
Sources
- RFC 1034 and RFC 1035, Domain Names; RFC 2308, Negative Caching of DNS Queries; RFC 8767, Serving Stale Data to Improve DNS Resiliency; RFC 9460, Service Binding and Parameter Specification via the DNS (SVCB and HTTPS).
- RFC 9110 (HTTP Semantics), RFC 9111 (HTTP Caching), RFC 9113 (HTTP/2), RFC 9114 (HTTP/3), RFC 7541 (HPACK), RFC 9204 (QPACK); all at rfc-editor.org.
- RFC 9000 (QUIC: A UDP-Based Multiplexed and Secure Transport) and RFC 9001 (Using TLS to Secure QUIC).
- Ilya Grigorik, High Performance Browser Networking, the chapters on TCP, TLS, HTTP/2 and HTTP/3 background.
- Cloudflare and Fastly engineering blogs on anycast, tiered caching and origin shielding; MDN, HTTP caching.
- Omer Gil, Web Cache Deception Attack (2017), and James Kettle, Practical Web Cache Poisoning (PortSwigger Research, 2018); the OWASP Cache Poisoning and Web Cache Deception pages.
- Fastly, Purging (soft purge and surrogate keys), and Cloudflare, Purge cache: check each provider's current documentation for propagation behaviour.
- RFC 9000 section 8 (address validation and the anti-amplification limit) and RFC 8446 (TLS 1.3), for the handshake flights and 0-RTT.
Practise this topic: Networking & APIs