CS Fundamentals Trainer · Guides

Capacity estimation: load, storage, bandwidth and memory on the back of an envelope

The constants worth memorising, what one small machine really does (measured), a seven-step method with Littles law, worked examples for a URL shortener, photo sharing and chat, how long data takes to move, and the mistakes that make estimates wrong by ten.

34 min read

"How many users? How many requests per second? How much data?" Those are the first questions in almost every design discussion, and they are not small talk. A system for a hundred requests a day and a system for a hundred million need different architectures: the first fits on one cheap server, and the second needs caches, partitioned data and a CDN. Capacity estimation (the "back-of-the-envelope calculation") is the skill of getting from a few stated assumptions to the order of magnitude of load, storage, bandwidth and memory, quickly enough to do it out loud, and accurately enough to choose between designs.

It is not about precision. If the answer is 3 TB, nobody cares whether it is 2.7 or 3.4. They care whether it is 3 TB or 3 PB, because that is the difference between "one database" and "a distributed storage system". This guide gives you the constants worth memorising, a method that works every time, three worked examples, measured numbers for what one small machine can actually do, and the mistakes that make estimates wrong by a factor of ten.

A note on measurements. The numbers marked "measured" were produced for this guide on a small cloud VM (2 vCPUs of an Intel Xeon at 2.1 GHz, 8 GB of memory, Linux, Python 3.13, Go 1.24, SQLite 3.45); the code is in the appendix. Your hardware will differ, sometimes by a factor of ten, which is itself a lesson: use such numbers to know whether you are in the right range, and measure your own system before you rely on them. The arithmetic in the worked examples comes from the calculator in the appendix, so every figure can be checked.

1. What an estimate is for

An estimate answers a question about the design:

If the estimate says……the design question is
a few hundred requests per seconddoes a single well-built service already do it? (usually yes)
tens of thousands per seconda stateless tier behind a load balancer, plus a cache and read replicas
writes beyond what one database commitsbatching, queues, or splitting the data across nodes
data in the terabytesone node with good backups, or sharding? How long does a restore take?
data in the petabytes, or bandwidth in the hundreds of Gbit/sobject storage, tiering, and a CDN
millions of open connectionsa dedicated connection tier; memory per connection becomes the constraint

The estimate also finds the dominant constraint. Every system is limited by one thing first: queries per second (a search box), storage (photos), bandwidth (video), memory (a chat connection tier) or write rate (metrics). Knowing which one saves you from designing a beautiful solution to the wrong problem.

2. The numbers worth memorising

Time

SecondsMemorise as
1 day86,400about 10^5 (round to 100,000 when you need speed)
1 month (30 days)2,592,000about 2.6 million
1 year31,536,000about 3 × 10^7

Two conversions are worth knowing by heart: 1 million events a day is about 12 per second (1,000,000 ÷ 86,400 ≈ 11.6), and so 100 million a day is about 1,200 per second. Almost every "per day" figure in a problem needs turning into "per second", and these two cover most cases.

Sizes

Powers of two are close to powers of ten, which makes the arithmetic easy: 2^10 = 1,024 (about a thousand, 1 KB), 2^20 ≈ a million (1 MB), 2^30 ≈ a billion (1 GB), 2^40 ≈ a trillion (1 TB), 2^50 ≈ a quadrillion (1 PB). Typical item sizes: a character is 1 byte (more in UTF-8 for non-ASCII), a UUID is 16 bytes binary or 36 as text, a tweet-sized message with metadata about 200 to 500 bytes, a web page 1 to 3 MB, a phone photo 2 to 5 MB, a minute of compressed HD video tens of MB.

Bits are not bytes. Network speeds are quoted in bits per second, storage in bytes: a 1 Gbit/s link moves at most 125 MB/s, and 10 Gbit/s at most 1.25 GB/s. Mixing them up is a factor-of-eight error.

Availability

The "nines" translate to downtime you are allowed:

AvailabilityPer yearPer 30-day month
99%87.6 hours7.2 hours
99.9%8.8 hours43.2 minutes
99.99%52.6 minutes4.3 minutes
99.999%5.3 minutes26 seconds

A system made of several parts in series is only as available as the product of its parts: three components at 99.9% each give about 99.7%. That is why a requirement of four nines rules out a single database in a single zone, whatever else you do.

Latency

The classic list "Latency Numbers Every Programmer Should Know", credited to Jeff Dean and compiled by Peter Norvig (the widely copied version is dated about 2012), gives the shape of the hierarchy:

OperationTime (circa 2012)
L1 cache reference0.5 ns
Main memory reference100 ns
Compress 1 KB3 µs
Send 1 KB over a 1 Gbit/s network10 µs
Read 4 KB randomly from SSD150 µs
Read 1 MB sequentially from memory250 µs
Round trip within the same datacenter500 µs
Read 1 MB sequentially from SSD1 ms
Disk seek10 ms
Read 1 MB sequentially from a spinning disk20 ms
Packet from California to the Netherlands and back150 ms

Treat the individual values as outdated but the ratios as durable: memory is about a thousand times faster than a disk seek, a datacenter round trip is hundreds of times slower than a memory access, and a round trip across an ocean is a hundred or so milliseconds that no engineering can remove (the speed of light is the limit). Those ratios are what tell you to cache, to batch small operations and to put data near users.

What one machine does (measured)

The same numbers measured on the VM described above:

OperationMeasured
SHA-256, one core (Python's hashlib)about 1.2 GB/s
JSON encode / decode of a 620-byte object, Pythonabout 190,000 / 120,000 per second
zlib compression, JSON text, level 1 / level 6about 51 / 18 MB/s, output about 48% / 43% of the input
Sequential read from the page cacheabout 7 GB/s
Sequential read after dropping the cacheabout 2.9 GB/s
Random 4 KB read after dropping the cacheabout 37 µs each
write of 4 KB followed by fsyncabout 0.12 ms, about 8,400 per second
Loopback TCP round trip, 1 byte, Python at both endsabout 37 µs
Opening and closing a local TCP connectionabout 74 µs
Go net/http "hello world", 2 vCPUs shared with the load generator25,000 requests/s with 1 client, about 40,000 to 43,000 with 8 to 256 clients
Latency of that server at 64 concurrent clientsp50 1.3 ms, p99 5.1 ms
Memory per idle TCP connection in Go (a goroutine and a 4 KB buffer)about 4.6 KB of process memory

On the same machine, SQLite with a table of one million rows (an id, a number and 100 bytes of text, with two indexes, about 130 bytes per row on disk):

OperationMeasured
Bulk load in one transactionabout 205,000 rows/s
Primary-key lookups, one connection, warmabout 200,000 per second
Lookups through a secondary index (about 10 rows each)about 154,000 per second
One full scan with a LIKE '%…%' filterabout 119 ms for 1,000,000 rows
One insert per commit, synchronous=FULLabout 4,400 commits/s (0.23 ms each)
One insert per commit, WAL with synchronous=NORMALabout 27,000 commits/s

Read these carefully, because they hold several of the lessons of this guide:

3. The method

The same seven steps work for any system. Write down each assumption and its unit, and round aggressively.

  1. Clarify the scope and the mix. What does the product do, for how many users, with what read/write ratio, what retention, what quality targets? Ask, and when you cannot, state your assumption out loud. A wrong assumption is acceptable; a hidden one is not.
  2. Traffic. DAU × actions per user per day ÷ 86,400 gives the average requests per second. Multiply by a peak factor (2 to 3 for a steady consumer product, 5 to 10 or more for spiky ones such as ticket sales or news) to get the peak. Size for the peak, not the average.
  3. Storage. records × size per record × retention, then multiply for replication (typically 3 copies, or about 1.5 with erasure coding for cold objects) and for overhead (indexes, row headers and free space: databases are rarely run above 70 to 80% full). In my SQLite test the data and two indexes took 130 bytes for a row with about 105 bytes of fields, so count on a noticeable overhead; measure it for your schema.
  4. Bandwidth. requests per second × bytes per response for outgoing traffic, and the same for incoming. Convert to bits and compare against what a network interface carries (1, 10 or 25 Gbit/s).
  5. Memory. For caches, estimate the working set (the data that is actually hot) rather than the whole data set. A common assumption is that 20% of the items receive 80% of the reads. That is a rule of thumb and not a law; real access logs often show a different skew, so check them when you have them. For connection tiers, connections × memory per connection.
  6. Servers. peak load ÷ capacity of one server, where the capacity is a measured number and is derated to a target utilisation (50 to 70% leaves room for spikes, deploys and failures), and add redundancy (N+1, or N+2 for large pools, or enough to survive the loss of a whole zone).
  7. Sanity-check and find the bottleneck. Cross-check with a second route (for example, bottom-up from the machines you would need against top-down from the users), and name the resource that runs out first.

Little's law: how many requests are in flight

One formula is worth keeping beyond the seven steps. Little's law says that the average number of items in a stable system equals the arrival rate times the average time each spends in it: L = λ × W. For a service, that is requests in flight = requests per second × seconds per request.

RateTime per requestRequests in flight
1,000 per second50 ms50
12,000 per second50 ms600
12,000 per second500 ms6,000
100,000 per second20 ms2,000

It tells you how many concurrent connections, threads or database connections a given load needs, and it shows the cruel effect of slowness: if the same service gets ten times slower, ten times as many requests pile up in it, so the memory, threads and connection-pool slots it holds also grow ten times, which is how a slowdown becomes an outage.

4. Worked example 1: a URL shortener

Assumptions. 100 million new short links a month, 100 reads for every write, 500 bytes stored per link (the short code, the long URL, a creation time and a few counters), links kept 5 years, 3 copies, peak at 3 times the average.

QuantityCalculationResult
Writes per second, average100,000,000 ÷ (30 × 86,400)39 per second
Writes per second, peak× 3about 120
Reads per second, average× 100about 3,900
Reads per second, peak× 3about 11,600
Links after 5 years100 M × 12 × 56 billion
Raw storage6 × 10^9 × 500 B3 TB
With 3 copies× 39 TB
Reads per day3,858 × 86,400333 million
Cache for the hot linksassume 1 distinct link per 100 reads: 3.3 M distinct per day, the hottest 20% of them, at 500 Babout 333 MB
App servers11,600 ÷ 5,000 per server = 3, plus one spare4

What it tells you. The write rate is tiny (tens per second), so write scaling is not the problem. The read rate is moderate, and the hot data fits in a few hundred MB of cache, so a cache removes most database reads. Storage, 3 TB for the primary copy, fits on one modern database server's disks, so partitioning is optional at first and becomes a growth question, not a day-one requirement. The interesting design work is elsewhere (generating unique short codes without collisions, and keeping redirects fast and available). The estimate has told you where the effort should not go.

5. Worked example 2: photo sharing

Assumptions. 50 million daily active users; 10% post one photo a day; each upload is a 2 MB original plus 0.2 MB of thumbnails; each user views 50 photos a day at an average of 300 KB each (feed-sized images).

QuantityCalculationResult
Uploads per day50 M × 10%5 million (58 per second on average)
New storage per day5 M × 2.2 MB11 TB
New storage per year× 3654 PB
With 3 full copies× 312 PB per year
With erasure coding (about 1.5×)× 1.56 PB per year
Outgoing data per day50 M × 50 × 300 KB750 TB
Average outgoing bandwidth750 TB × 8 ÷ 86,40069 Gbit/s
At 3× peak× 3about 208 Gbit/s
10 Gbit/s links neededaverage, peak7, 21

What it tells you. The request rate (58 uploads a second) is trivial. Storage (petabytes a year) and above all bandwidth (hundreds of Gbit/s at peak) dominate. No origin server farm should serve that: the images belong in object storage, delivered by a CDN that absorbs most of the traffic near users, with tiering that moves old photos to cheaper storage. Compare with the previous example, where the same method pointed at a cache and a modest database. The method is the same, and the conclusion is entirely different.

6. Worked example 3: chat

Assumptions. 100 million daily active users, each sending 40 messages a day; 200 bytes stored per message (text and metadata); 20% of the users online at peak; each online user keeps one open connection costing about 8 KB in total; a gateway holds up to 100,000 connections.

QuantityCalculationResult
Messages per day100 M × 404 billion
Messages per second, average÷ 86,400about 46,000
Messages per second, peak× 3about 139,000
Storage per day4 B × 200 B800 GB
Storage per year× 365292 TB (876 TB with 3 copies)
Connections at peak20% of 100 M20 million
Memory to hold the connections20 M × 8 KB160 GB
Gateways20 M ÷ 100,000200 (plus spare capacity)
If one gateway fails100,000 clients reconnect; over 30 sabout 3,300 connections per second on the rest

The 8 KB per connection is a measured figure: holding 18,000 idle connections in a Go server took about 4.6 KB of process memory each (a goroutine and a 4 KB read buffer); the whole machine used about 12 KB per connection counting both the client and server ends of each one, which leaves roughly 4 KB of kernel memory per end (an estimate: I assumed both ends cost the same). A real chat connection also carries per-user state, so 8 KB is a floor.

What it tells you. The message rate is high but ordinary for a partitioned log or a sharded database. The distinctive constraint is connections: 20 million of them, 160 GB of memory just to hold them open, spread over hundreds of machines. That is the reason chat designs have a separate gateway tier that only holds connections, and a lookup of "which gateway holds this user". It is also why a failing gateway matters: its 100,000 clients reconnect at once, so reconnects need jitter and the remaining gateways need headroom. See the guide on designing a chat system for how those parts fit together.

7. Moving data takes time

Capacity is not only "does it fit" but also "how long does it take to copy". A new replica, a restore from backup or a migration to new hardware has to move the whole data set:

DataAt 1 Gbit/sAt 10 Gbit/s
1 TB2.2 hours13 minutes
10 TB22 hours2.2 hours
100 TB9.3 days22 hours

(Assuming the link is fully used, which it never quite is.) These are the numbers behind advice such as "keep each database node to a few terabytes". Past that, a failed replica takes a day to rebuild, during which you are running with less redundancy, and a restore after a disaster is measured in days. When an estimate gives you tens of terabytes in one place, ask not only whether it fits but also how you would rebuild it.

8. Rules of thumb and traps

9. Presenting an estimate

In an interview or a design review, the estimate is also communication.

10. Common mistakes

11. Things to mention in an interview

Quick recap

Appendix: the code

The numbers come from four programs. The calculator estimate.py prints every worked example above. bench.py produced the CPU, disk, SQLite and loopback figures (run python3 bench.py cpu disk db net; it needs a few GB of free disk space and tries to drop the page cache, which needs root). The two Go programs measured the HTTP rate and the cost of idle connections (build each with go build; start conns_bin server in one terminal and conns_bin client 18000 in another, with the file-descriptor limit raised if you want more than about 18,000 connections).

estimate.py

"""Back-of-the-envelope calculator for the capacity-estimation guide. Every number printed in the worked examples comes from here."""
import math

DAY = 86_400
def fmt(n, unit=""):
    for div, name in ((1e15, "P"), (1e12, "T"), (1e9, "G"), (1e6, "M"), (1e3, "k")):
        if abs(n) >= div: return f"{n / div:.3g} {name}{unit}"
    return f"{n:.3g} {unit}".strip()

def rule(title): print(f"\n== {title}")

def constants():
    rule("time")
    for name, s in (("minute", 60), ("hour", 3600), ("day", DAY), ("month (30 days)", 30 * DAY), ("year", 365 * DAY)):
        print(f"   {name:16s} {s:>12,} s   ~ {s:.1e}")
    rule("availability: downtime allowed per year and per 30-day month")
    for nines in (0.99, 0.999, 0.9999, 0.99999):
        y = (1 - nines) * 365 * DAY; m = (1 - nines) * 30 * DAY
        h = lambda s: f"{s / 3600:.1f} h" if s >= 3600 else (f"{s / 60:.1f} min" if s >= 60 else f"{s:.0f} s")
        print(f"   {nines * 100:g}%   {h(y):>9} per year   {h(m):>9} per month")
    rule("powers of two")
    for e, n in ((10, "KB"), (20, "MB"), (30, "GB"), (40, "TB"), (50, "PB")):
        print(f"   2^{e} = {2 ** e:>20,}  ~ 10^{e * 0.30103:.1f}  (1 {n})")

def littles_law():
    rule("Little's law: requests in flight = rate x time in the system")
    for rps, ms in ((1_000, 50), (12_000, 50), (12_000, 500), (100_000, 20)):
        print(f"   {rps:>7,} requests/s x {ms:>3} ms = {rps * ms / 1000:>7,.0f} requests in flight")

def example_url_shortener():
    rule("Example 1: URL shortener")
    new_per_month, read_write, record, years, replicas, peak = 100e6, 100, 500, 5, 3, 3
    w = new_per_month / (30 * DAY); r = w * read_write
    print(f"   writes: {new_per_month:,.0f} / month = {w:.1f} per second on average; peak x{peak} = {w * peak:.0f} per second")
    print(f"   reads:  x{read_write} = {r:,.0f} per second on average; peak = {r * peak:,.0f} per second")
    total = new_per_month * 12 * years
    raw = total * record
    print(f"   links after {years} years: {total:,.0f}; {record} bytes each = {fmt(raw, 'B')} raw, {fmt(raw * replicas, 'B')} with {replicas} copies")
    reads_day = r * DAY
    print(f"   reads per day: {reads_day:,.0f}; if 20% of links get 80% of the reads, cache the hot 20% of one day's links:")
    print(f"      assume 1 distinct link per 100 reads -> {reads_day / 100:,.0f} distinct per day; 20% of that = {reads_day / 100 * 0.2:,.0f} x {record} B = {fmt(reads_day / 100 * 0.2 * record, 'B')} of cache")
    per_server = 5_000
    print(f"   app servers: peak {r * peak:,.0f} / {per_server:,} per server = {math.ceil(r * peak / per_server)}, plus one spare = {math.ceil(r * peak / per_server) + 1}")

def example_photos():
    rule("Example 2: photo sharing")
    dau, post_share, orig, thumb, views, avg_view = 50e6, 0.10, 2e6, 0.2e6, 50, 0.3e6
    posts = dau * post_share
    per_day = posts * (orig + thumb)
    print(f"   uploads: {dau:,.0f} DAU x {post_share:.0%} post once = {posts:,.0f} photos/day = {posts / DAY:.0f} per second on average")
    print(f"   storage: {posts:,.0f} x ({orig / 1e6:g} MB + {thumb / 1e6:g} MB thumbnails) = {fmt(per_day, 'B')} per day, {fmt(per_day * 365, 'B')} per year")
    print(f"      with 3 full copies {fmt(per_day * 365 * 3, 'B')} per year; with erasure coding ~1.5x: {fmt(per_day * 365 * 1.5, 'B')} per year")
    egress = dau * views * avg_view
    print(f"   reads: {dau:,.0f} x {views} views x {avg_view / 1e3:.0f} KB = {fmt(egress, 'B')} per day = {egress * 8 / DAY / 1e9:.0f} Gbit/s on average, {egress * 8 / DAY / 1e9 * 3:.0f} Gbit/s at 3x peak")
    print(f"   one 10 Gbit/s server link carries {10 / 8:.2f} GB/s; the average needs {egress * 8 / DAY / 1e9 / 10:.0f} of them, the peak {egress * 8 / DAY / 1e9 * 3 / 10:.0f}")

def example_chat():
    rule("Example 3: chat")
    dau, msgs, size, online, bytes_conn, per_gw = 100e6, 40, 200, 0.2, 8_000, 100_000
    per_day = dau * msgs; rate = per_day / DAY
    print(f"   messages: {dau:,.0f} x {msgs} = {per_day:,.0f} per day = {rate:,.0f} per second on average, {rate * 3:,.0f} at 3x peak")
    print(f"   storage:  {per_day:,.0f} x {size} B = {fmt(per_day * size, 'B')} per day, {fmt(per_day * size * 365, 'B')} per year, x3 copies = {fmt(per_day * size * 365 * 3, 'B')}")
    conns = dau * online
    print(f"   connections at peak: {online:.0%} of DAU = {conns:,.0f}; at {bytes_conn / 1e3:g} KB each = {fmt(conns * bytes_conn, 'B')} of memory just to hold them")
    print(f"   gateways: {conns:,.0f} / {per_gw:,} per gateway = {math.ceil(conns / per_gw)} (plus spare capacity so one lost gateway's {per_gw:,} reconnects can be absorbed)")
    print(f"   if a gateway fails, {per_gw:,} clients reconnect at once; spread over 30 s that is {per_gw / 30:,.0f} connections per second to the rest")

def copy_time():
    rule("how long it takes to move data (a new replica, a restore, a migration)")
    for size_tb in (1, 10, 100):
        row = []
        for name, gbit in (("1 Gbit/s", 1), ("10 Gbit/s", 10)):
            secs = size_tb * 1e12 / (gbit * 1e9 / 8)
            row.append(f"{name}: {secs / 3600:6.1f} h" if secs < 3 * DAY else f"{name}: {secs / DAY:6.1f} days")
        print(f"   {size_tb:>4} TB   " + "   ".join(row) + "   (link fully used, no overhead)")

def headroom():
    rule("headroom: how many servers for a load, at a target utilisation")
    peak, per_server = 12_000, 4_000
    for util in (1.0, 0.7, 0.5):
        n = math.ceil(peak / (per_server * util))
        print(f"   peak {peak:,} rps, {per_server:,} rps per server at 100% -> at {util:.0%} target utilisation: {n} servers; with N+1: {n + 1}; losing one of {n + 1} leaves {(n) * 100 / (n + 1):.0f}% of the pool")

if __name__ == "__main__":
    constants(); littles_law(); example_url_shortener(); example_photos(); example_chat(); copy_time(); headroom()

bench.py

"""Calibration numbers for the capacity-estimation guide: what one small machine does. Standard library only."""
import hashlib, json, os, random, socket, sqlite3, threading, time, uuid, zlib

D = "/var/tmp/g3/bench"
os.makedirs(D, exist_ok=True)

def timeit(f, n):
    t = time.perf_counter(); f(n); return time.perf_counter() - t

def part_cpu():
    print("CPU-bound work in one Python thread (the interpreter is slower than C or Go, so read these as floors)")
    buf = os.urandom(1 << 20)
    n = 200; dt = timeit(lambda n: [hashlib.sha256(buf).digest() for _ in range(n)], n)
    print(f"   SHA-256 of 1 MB blocks:           {n / dt:7.0f} MB/s")
    random.seed(5)
    words = [hashlib.md5(str(i).encode()).hexdigest()[:random.randint(3, 9)] for i in range(3000)]
    def make():
        return {"id": str(uuid.UUID(int=random.getrandbits(128), version=4)), "user": random.randrange(10**7),
                "text": " ".join(random.choice(words) for _ in range(60)), "tags": random.sample(words, 3), "n": [random.randrange(1000) for _ in range(20)]}
    doc = make(); raw = json.dumps(doc); n = 100_000
    print(f"   JSON object of {len(raw)} bytes:        {n / timeit(lambda n: [json.dumps(doc) for _ in range(n)], n):7.0f} encodes/s, "
          f"{n / timeit(lambda n: [json.loads(raw) for _ in range(n)], n):7.0f} decodes/s")
    text = ("\n".join(json.dumps(make()) for _ in range(3000))).encode()
    for lvl in (1, 6):
        t = time.perf_counter(); c = zlib.compress(text, lvl); dt = time.perf_counter() - t
        print(f"   zlib level {lvl} on JSON text:       {len(text) / dt / 1e6:7.0f} MB/s, size {len(c) / len(text):.0%} of the original")

def part_disk():
    print("Disk and memory")
    p = f"{D}/seq.bin"; size = 1 << 30
    with open(p, "wb") as f:
        blk = os.urandom(1 << 20)
        t = time.perf_counter()
        for _ in range(size >> 20): f.write(blk)
        f.flush(); os.fsync(f.fileno())
    print(f"   write 1 GB sequentially + fsync:  {size / (time.perf_counter() - t) / 1e6:7.0f} MB/s")
    for label in ("page cache (warm)", "after dropping caches (cold)"):
        if "cold" in label:
            try:
                os.sync(); open("/proc/sys/vm/drop_caches", "w").write("3")
            except Exception as e:
                print("   (could not drop caches:", e, ")"); continue
        t = time.perf_counter(); got = 0
        with open(p, "rb", buffering=0) as f:
            while True:
                b = f.read(1 << 20)
                if not b: break
                got += len(b)
        print(f"   read 1 GB sequentially, {label:28s}: {got / (time.perf_counter() - t) / 1e6:7.0f} MB/s")
    # random 4 KB reads, cold
    try:
        os.sync(); open("/proc/sys/vm/drop_caches", "w").write("3")
    except Exception: pass
    fd = os.open(p, os.O_RDONLY); n = 2000; random.seed(1)
    t = time.perf_counter()
    for _ in range(n): os.pread(fd, 4096, random.randrange(0, size - 4096) & ~4095)
    dt = time.perf_counter() - t
    print(f"   random 4 KB reads, cold:          {dt / n * 1e6:7.0f} us each ({n / dt:,.0f} per second, one at a time)")
    os.close(fd); os.remove(p)
    # fsync latency
    fd = os.open(f"{D}/sync.bin", os.O_WRONLY | os.O_CREAT); n = 500
    t = time.perf_counter()
    for i in range(n): os.pwrite(fd, b"x" * 4096, 0); os.fsync(fd)
    dt = time.perf_counter() - t; os.close(fd)
    print(f"   write 4 KB + fsync:               {dt / n * 1e3:7.2f} ms each ({n / dt:,.0f} per second)")

def part_db():
    print("SQLite, one table of 1,000,000 rows (id, user, 100-byte payload) with indexes on id and user")
    p = f"{D}/db.sqlite"
    for f in (p, p + "-wal", p + "-shm"):
        if os.path.exists(f): os.remove(f)
    c = sqlite3.connect(p, isolation_level=None); c.execute("PRAGMA journal_mode=WAL")
    c.execute("CREATE TABLE t (id INTEGER PRIMARY KEY, user INTEGER, payload TEXT)")
    c.execute("CREATE INDEX t_user ON t (user)")
    pay = "p" * 100; N = 1_000_000
    t = time.perf_counter(); c.execute("BEGIN")
    c.executemany("INSERT INTO t (user, payload) VALUES (?, ?)", ((random.randrange(100_000), pay) for _ in range(N)))
    c.execute("COMMIT"); dt = time.perf_counter() - t
    size = os.path.getsize(p)
    print(f"   bulk load in one transaction:     {N / dt:8,.0f} rows/s; file {size / 1e6:.0f} MB = {size / N:.0f} bytes per row (data + 2 indexes)")
    n = 100_000; ids = [random.randrange(1, N) for _ in range(n)]
    t = time.perf_counter()
    for i in ids: c.execute("SELECT payload FROM t WHERE id=?", (i,)).fetchone()
    print(f"   primary-key lookups, warm:        {n / (time.perf_counter() - t):8,.0f} per second, one connection")
    n = 20_000; t = time.perf_counter()
    for _ in range(n): c.execute("SELECT COUNT(*) FROM t WHERE user=?", (random.randrange(100_000),)).fetchone()
    print(f"   lookups via the user index:       {n / (time.perf_counter() - t):8,.0f} per second (about {N // 100_000} rows each)")
    n = 50; t = time.perf_counter()
    for _ in range(n): c.execute("SELECT COUNT(*) FROM t WHERE payload LIKE '%zzz%'").fetchone()
    dt = (time.perf_counter() - t) / n
    print(f"   one full scan (no index):         {dt * 1e3:8.0f} ms for {N:,} rows")
    # commits with fsync
    c.execute("PRAGMA synchronous=FULL"); n = 300; t = time.perf_counter()
    for _ in range(n): c.execute("INSERT INTO t (user, payload) VALUES (1, 'x')")
    dt = time.perf_counter() - t
    print(f"   one insert per commit (synchronous=FULL): {n / dt:6,.0f} commits/s ({dt / n * 1e3:.2f} ms each)")
    c.execute("PRAGMA synchronous=NORMAL"); t = time.perf_counter()
    for _ in range(n * 10): c.execute("INSERT INTO t (user, payload) VALUES (1, 'x')")
    dt = time.perf_counter() - t
    print(f"   one insert per commit (synchronous=NORMAL, WAL): {n * 10 / dt:6,.0f} commits/s")

def part_net():
    print("Loopback TCP, one byte back and forth (this is the floor: a real network adds distance)")
    srv = socket.socket(); srv.bind(("127.0.0.1", 0)); srv.listen()
    def echo():
        s, _ = srv.accept(); s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
        while True:
            b = s.recv(1)
            if not b: return
            s.sendall(b)
    threading.Thread(target=echo, daemon=True).start()
    c = socket.create_connection(srv.getsockname()); c.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
    n = 20_000; t = time.perf_counter()
    for _ in range(n): c.sendall(b"x"); c.recv(1)
    dt = time.perf_counter() - t
    print(f"   round trip: {dt / n * 1e6:6.1f} us (includes Python's own overhead on both ends)")
    srv2 = socket.socket(); srv2.bind(("127.0.0.1", 0)); srv2.listen(1024)
    threading.Thread(target=lambda: [srv2.accept()[0].close() for _ in range(2000)], daemon=True).start()
    t = time.perf_counter()
    for _ in range(2000):
        s = socket.create_connection(srv2.getsockname()); s.close()
    print(f"   opening and closing a connection: {(time.perf_counter() - t) / 2000 * 1e6:6.0f} us")

if __name__ == "__main__":
    import sys
    for p in (sys.argv[1:] or ["cpu", "disk", "db", "net"]): globals()["part_" + p]()

http/main.go

// A "hello world" HTTP server and a closed-loop client on the same machine: how many requests per second one small VM serves.
package main

import (
	"fmt"
	"io"
	"net"
	"net/http"
	"runtime"
	"sort"
	"sync"
	"sync/atomic"
	"time"
)

func main() {
	ln, _ := net.Listen("tcp", "127.0.0.1:0")
	go http.Serve(ln, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "ok") }))
	url := "http://" + ln.Addr().String() + "/"
	for _, workers := range []int{1, 8, 64, 256} {
		client := &http.Client{Transport: &http.Transport{MaxIdleConnsPerHost: workers, MaxConnsPerHost: workers}}
		var n atomic.Int64
		var mu sync.Mutex
		var lat []time.Duration
		stop := time.Now().Add(4 * time.Second)
		var wg sync.WaitGroup
		for i := 0; i < workers; i++ {
			wg.Add(1)
			go func() {
				defer wg.Done()
				var mine []time.Duration
				for time.Now().Before(stop) {
					t := time.Now()
					resp, err := client.Get(url)
					if err != nil {
						continue
					}
					io.Copy(io.Discard, resp.Body)
					resp.Body.Close()
					mine = append(mine, time.Since(t))
					n.Add(1)
				}
				mu.Lock()
				lat = append(lat, mine...)
				mu.Unlock()
			}()
		}
		wg.Wait()
		sort.Slice(lat, func(i, j int) bool { return lat[i] < lat[j] })
		fmt.Printf("  %3d concurrent clients: %7.0f requests/s, p50 %v, p99 %v\n", workers, float64(n.Load())/4, lat[len(lat)/2].Round(time.Microsecond), lat[len(lat)*99/100].Round(time.Microsecond))
		client.CloseIdleConnections()
	}
	fmt.Println("  (GOMAXPROCS =", runtime.GOMAXPROCS(0), "; the client and the server share the same CPUs)")
}

conns/main.go

// Memory cost of holding many idle TCP connections, one goroutine per connection (the usual Go server shape).
// Run "server" in one process and "client N" in another.
package main

import (
	"bufio"
	"fmt"
	"net"
	"os"
	"strconv"
	"strings"
	"time"
)

func meminfo(key string) int64 {
	b, _ := os.ReadFile("/proc/meminfo")
	for _, l := range strings.Split(string(b), "\n") {
		if strings.HasPrefix(l, key+":") {
			f := strings.Fields(l)
			v, _ := strconv.ParseInt(f[1], 10, 64)
			return v // kB
		}
	}
	return 0
}
func rss() int64 {
	b, _ := os.ReadFile("/proc/self/status")
	for _, l := range strings.Split(string(b), "\n") {
		if strings.HasPrefix(l, "VmRSS:") {
			v, _ := strconv.ParseInt(strings.Fields(l)[1], 10, 64)
			return v
		}
	}
	return 0
}

const ports = 6

func main() {
	if os.Args[1] == "server" {
		base := int(meminfo("MemAvailable"))
		r0 := rss()
		var count int64
		for p := 0; p < ports; p++ {
			ln, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", 9100+p))
			if err != nil {
				panic(err)
			}
			go func() {
				for {
					c, err := ln.Accept()
					if err != nil {
						return
					}
					count++
					go func() { // one goroutine and one 4 KB read buffer per connection
						r := bufio.NewReaderSize(c, 4096)
						r.ReadByte()
						c.Close()
					}()
				}
			}()
		}
		fmt.Println("server ready")
		last := int64(-1)
		for {
			time.Sleep(2 * time.Second)
			if count != last && count > 0 {
				last = count
				fmt.Printf("server: %6d connections, process RSS +%6.1f MB (%.2f KB each), system memory used +%6.1f MB (%.2f KB each)\n",
					count, float64(rss()-r0)/1024, float64(rss()-r0)/float64(count),
					float64(int64(base)-meminfo("MemAvailable"))/1024, float64(int64(base)-meminfo("MemAvailable"))/float64(count))
			}
		}
	}
	n, _ := strconv.Atoi(os.Args[2])
	var keep []net.Conn
	for i := 0; i < n; i++ {
		c, err := net.Dial("tcp", fmt.Sprintf("127.0.0.1:%d", 9100+i%ports))
		if err != nil {
			fmt.Println("client stopped at", i, err)
			break
		}
		keep = append(keep, c)
	}
	fmt.Println("client holds", len(keep), "connections")
	time.Sleep(6 * time.Second)
}

Sources

Practise this topic: System Design System Design › Reliability and scale

More guides