CS Fundamentals Trainer · Guides
Go runtime in practice: goroutines, channels, the scheduler, memory and garbage collection
How the Go runtime behaves and what it costs, with measurements: goroutine memory and start-up cost, the G, M and P scheduler and preemption, channels against mutexes and atomics, data races and the memory model, goroutine leaks and context cancellation with a worker pool, escape analysis and allocation costs, GOGC and GOMEMLIMIT, and the language behaviours that surprise people.
39 min read
Go became popular for servers because concurrency feels cheap: you write go f() and a new goroutine runs, you pass values over channels, and the runtime spreads the work over the CPUs. That impression is mostly true, and it hides the places where the cost is real. A goroutine takes memory, a channel takes time, a forgotten goroutine never goes away, a pointer that escapes becomes garbage that the collector has to scan, and a loop that looks innocent can hold a processor for a whole time slice.
This guide explains how the Go runtime works well enough to predict those costs: what a goroutine is and how the scheduler runs it, what channels, mutexes and atomics cost against each other, how goroutine leaks happen and how to prevent them, how the compiler decides between stack and heap, how the garbage collector trades memory for CPU, and the language behaviours that most often surprise people. Almost every claim comes with a measurement, and the programs are in the appendix.
A note on measurements. The numbers marked "measured" come from Go 1.24 programs on a small cloud VM (2 vCPUs of an Intel Xeon at 2.1 GHz, 8 GB of memory, Linux). Timings vary between runs and between machines, so read them for the ratios and orders of magnitude; where a result depends on noise, the guide gives the median of three runs. The benchmarks use
testing.Benchmark, and the output of the compiler's escape analysis is quoted fromgo build -gcflags='-m -l'.
1. Goroutines
A goroutine is a function running concurrently, scheduled by the Go runtime and not by the operating system. Three properties make it cheap:
- It starts with a small stack that grows. The runtime begins with a stack of a couple of kilobytes and copies it to a larger one when it runs out, so a goroutine pays only for what it uses. An operating system thread, by contrast, reserves a large fixed stack of address space (commonly megabytes) and costs a kernel object.
- Creating and switching are done in user space, without a system call.
- Waiting is cheap. A goroutine blocked on a channel, a mutex or network I/O is parked in a queue and costs memory but no CPU.
Measured: goroutines were started and parked on a channel, and the memory taken was read from the runtime.
| Goroutines | Time to start all | Per goroutine | Stack in use per goroutine | Memory from the system, per goroutine |
|---|---|---|---|---|
| 1,000 | 1.7 ms | 1.7 µs | 1.98 KB | |
| 100,000 | 241 ms | 2.4 µs | 2.02 KB | 2.65 KB (271 MB total) |
| 1,000,000 | 3.4 s | 3.4 µs | 2.20 KB | 2.56 KB (2.6 GB total) |
So a goroutine costs about 2 KB of stack plus a few hundred bytes of bookkeeping, about 2.6 KB in total, and a couple of microseconds to start. A million of them fit in a few gigabytes. That is why Go can give each connection or request its own goroutine, and why the old advice to "pool your threads" mostly does not apply.
Cheap is not free, and it is not bounded. Two rules follow:
- Bound the concurrency of work that touches something finite: a database with 100 connections, an API with a rate limit, memory per job. Start a fixed number of workers, or use a semaphore (a buffered channel), instead of one goroutine per item of an unbounded input.
- Every goroutine needs an end. Know, when you write
go, what makes it stop (section 5).
2. The scheduler: G, M and P
The runtime schedules goroutines (G) onto operating-system threads (M), and a thread needs a processor (P), a token that holds a queue of runnable goroutines and the permission to run Go code. The number of Ps is GOMAXPROCS, which defaults to the number of CPUs the process sees.
Diagram: Goroutines, threads and processors
What the diagram means in practice:
- A new goroutine goes onto the local queue of the current P. An idle P steals work from another P or takes from the global queue, so the load spreads without a central lock.
- Network I/O does not hold a thread. A goroutine waiting for a socket is parked and registered with the network poller; when the socket is ready it becomes runnable again. Ten thousand idle connections are ten thousand parked goroutines, not ten thousand threads.
- A goroutine in a blocking system call (a file read, for instance) keeps its thread, but the runtime hands the P to another thread so the other goroutines go on running.
- Since Go 1.14 the scheduler can preempt a goroutine that runs for too long (about 10 ms) even if it never calls a function or waits, using a signal.
Measured: 400,000 rounds of SHA-256, split among goroutines, with different GOMAXPROCS on a 2-CPU machine.
| GOMAXPROCS | Goroutines sharing the work | Time |
|---|---|---|
| 1 | 1, 2 or 8 | 40 ms, 42 ms, 40 ms |
| 2 | 1 | 40 ms (one goroutine can use only one CPU) |
| 2 | 2 or 8 | 22 ms, 21 ms (about twice as fast) |
| 4 | 2 or 8 | 22 ms, 21 ms (no gain over 2: there are only 2 CPUs) |
Three conclusions: parallelism needs both several goroutines and several Ps; more goroutines than CPUs do not make CPU-bound work faster (they cost nothing extra, either); and GOMAXPROCS beyond the number of real CPUs buys nothing. In a container limited by a CPU quota, the runtime may see more CPUs than the quota allows, which causes throttling; set GOMAXPROCS explicitly to the limit (or use a library that reads it).
Concurrency is not parallelism. Measured: with GOMAXPROCS=1, 10,000 goroutines that each slept 100 ms finished in 134 ms, because waiting overlaps even on one processor.
And preemption is not instant. Measured: with GOMAXPROCS=1 and one goroutine spinning in a loop with no function calls, a 1 ms sleep in another goroutine took 21.7 ms on average and 40.8 ms at worst. The spinner is preempted, which is why the program does not hang, but the sleeper waits for the spinner's time slice to end. On a real server, a CPU-heavy loop on all Ps (compression, parsing a huge input, a tight computation) shows up as latency spikes for unrelated requests, not as a failure.
3. Channels
A channel is a typed queue with built-in synchronisation. The rules worth knowing exactly:
| Operation | Unbuffered channel | Buffered channel (capacity n) |
|---|---|---|
| Send | waits until a receiver is ready (a hand-off) | waits only if the buffer is full |
| Receive | waits until a sender is ready | waits only if the buffer is empty |
| Receive from a closed channel | returns the zero value at once (v, ok gives ok == false), after buffered values are drained | same |
| Send on a closed channel | panics | panics |
| Close a closed channel, or a nil channel | panics | panics |
| Send or receive on a nil channel | blocks forever | blocks forever |
Consequences: only the sender closes, and only when there is a single owner of the closing decision (several senders need a coordinator); close is a broadcast ("no more values", seen by every receiver); and a select over channels chooses randomly among the ready cases, so it gives no priority. A nil channel in a select is a useful trick: it disables that case.
The classic patterns are the pipeline (stages connected by channels), the worker pool (several goroutines reading one channel of jobs) and fan-in (merging several channels into one). Section 5 gives a worker pool with correct cancellation.
Channels, mutexes and atomics: what they cost
Measured: 4 goroutines each add 1 to a shared counter 1,000,000 times, on 2 CPUs, four ways.
| Technique | Time per increment |
|---|---|
| One counter per goroutine, summed at the end | 1.4 ns |
atomic.Int64 | 16.4 ns |
sync.Mutex | 57.6 ns |
| Send every increment over a channel to one owner goroutine | 78.0 ns |
And for passing messages between two goroutines:
| Operation | Cost |
|---|---|
| Unbuffered round trip (send, then receive back) | 431 ns |
| One-way stream, unbuffered | 245 ns per message |
| One-way stream, buffer of 1 | 204 ns per message |
| One-way stream, buffer of 128 | 92 ns per message |
Read these carefully:
- The right design beat the fastest primitive by a factor of ten: giving each goroutine its own counter and combining at the end removes contention altogether. The cheapest synchronisation is none.
- A channel send costs on the order of 100 to 250 ns: very cheap for passing a job or a result, but too expensive for a per-item operation inside a hot loop (a counter, a lock around a map lookup). The mutex cost here comes from contention between goroutines on different CPUs; an uncontended lock is much cheaper.
- A buffer reduces the cost per message by letting sender and receiver run without meeting each other every time.
The practical rule, often stated as "share memory by communicating": use a channel to transfer ownership or to coordinate (jobs, results, events, cancellation), and a mutex or atomic to protect state that several goroutines must read and write (a cache, a counter, a configuration). A channel standing in for a lock is slower and harder to read than the lock.
4. Shared state, data races and the memory model
When two goroutines access the same variable, at least one of them writes, and nothing orders the accesses, the program has a data race, and its behaviour is undefined in practice. The problem is not only lost updates: the compiler and the CPU are allowed to reorder and cache, so one goroutine may never see another's write.
Measured: two goroutines each incremented a plain int one million times. The result should be 2,000,000. Three runs gave 1,716,216, 1,716,891 and 1,442,407: between 14% and 28% of the updates vanished, and a different number each time. (With only 1,000 increments each, three runs gave the right answer, because the first goroutine finished before the second started. A race that does not show in a small test is still a race.)
Go's tool for this is the race detector, a build flag that instruments memory accesses: go test -race and go run -race. On the same program it reports:
WARNING: DATA RACE
Read at 0x00c000112078 by goroutine 8:
main.main.func3()
main.go:85 +0x99
Previous write at 0x00c000112078 by goroutine 7:
main.main.func3()
main.go:85 +0xab
with the two conflicting accesses and where the goroutines were created. It finds only races that actually happen during the run, and it makes the program several times slower, so run your tests with -race in CI (it is the single most useful concurrency tool in the language) and do not rely on it in production.
The Go memory model says what ordering you can rely on. The guarantees come from synchronisation events, each of which creates a happens-before edge:
- A send on a channel happens before the corresponding receive completes; closing a channel happens before a receive that returns because of the close.
Unlockof a mutex happens before a laterLockof the same mutex.wg.Donehappens beforewg.Waitreturns;sync.Oncefinishes before anyDoreturns.- Atomic operations are sequentially consistent and synchronise.
Without such an edge you may not assume that one goroutine sees what another did, even if the code "obviously" runs in sequence. Typical bugs: a flag read in a loop without synchronisation (the compiler may hoist the read out of the loop), a lazily initialised global without sync.Once, a map written by several goroutines.
A map is the sharpest case. Concurrent writes are detected on a best-effort basis and end the whole process: four goroutines writing to one map crashed this experiment with
fatal error: concurrent map writes
which is a fatal error, not a panic, so recover cannot catch it. Protect a map with a mutex (or use sync.Map for the two narrow cases it was made for: keys written once and read many times, or goroutines working on disjoint sets of keys).
5. Cancellation and goroutine leaks
A goroutine that blocks forever is a leak: it never frees its stack, and everything its stack refers to stays alive. Leaks do not fail a test; they show up as slowly rising memory and goroutine counts in production. The most common cause is a goroutine blocked on a channel operation that nobody will ever complete.
The experiment calls a function 10,000 times; each call starts a worker that takes 5 ms and gives up after 1 ms. In the leaking version the worker sends its result over an unbuffered channel; after the timeout nobody receives, so the send blocks forever.
| Version | Goroutines before → after 10,000 calls | Extra memory from the system |
|---|---|---|
| Unbuffered result channel | 1 → 10,001 | +29.9 MB |
| Result channel with a buffer of one | 1 → 1 | +4.8 MB |
The fix was a single character, make(chan int, 1): a buffer of one lets the worker complete its send and exit whether or not anyone is listening. About 3 KB per leaked goroutine, multiplied by every timed-out request for the life of the process.
The general tool is the context.Context: a value that carries a cancellation signal (and a deadline) down a call tree. The rules:
- Pass
ctxas the first parameter of every function that may block or take long, and pass it down. Do not store it in a struct. - Every
gostatement needs an answer to "what stops it?": the channel it reads gets closed, or it watchesctx.Done(), or its work is bounded. - Every blocking channel operation in a goroutine that can be cancelled should be in a
selectwithctx.Done(). - The side that creates a context with
WithCancelorWithTimeoutmust call thecancelfunction (usuallydefer cancel()). - Wait for goroutines to finish (
sync.WaitGroup) before returning from the function that started them, so no work outlives its caller.
The listing below puts these rules into a worker pool that stops at the first error and when its parent is cancelled, and that returns only after all workers have exited. (It compiles and was run under the race detector with a failing job and with a clean run.)
// process runs handle for every job with a fixed number of workers. The first error stops the rest;
// cancelling the parent context does too. It returns only after every worker has exited.
func process(parent context.Context, jobs []string, workers int, handle func(context.Context, string) error) error {
ctx, cancel := context.WithCancel(parent)
defer cancel()
in := make(chan string)
firstErr := make(chan error, 1) // buffered, so a failing worker never blocks while reporting
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range in {
if err := handle(ctx, job); err != nil {
select {
case firstErr <- err:
default: // another worker already reported
}
cancel() // tell the producer and the other workers to stop
return
}
}
}()
}
feed:
for _, job := range jobs {
select {
case in <- job:
case <-ctx.Done(): // never block on a send without also watching for cancellation
break feed
}
}
close(in) // the sender closes, so the workers' range loops end
wg.Wait()
select {
case err := <-firstErr:
return err
default:
return parent.Err()
}
}
To find leaks in a running service, look at the goroutine count over time (runtime.NumGoroutine() as a metric) and at the profile of goroutine stacks (pprof's goroutine profile groups them by where they are blocked, which usually points straight at the leak).
6. Where values live: stack, heap and escape analysis
Go has no new versus stack distinction in the syntax. The compiler decides, by escape analysis, whether a variable can live on the goroutine's stack (freed for nothing when the function returns, never seen by the garbage collector) or must go on the heap (allocated by the runtime, and freed later by the garbage collector). A value escapes when the compiler cannot prove that nothing refers to it after the function returns. You can ask the compiler what it decided with go build -gcflags='-m -l' (the -l disables inlining so that the output is easier to read). For the functions in the appendix:
./main.go:15:2: moved to heap: p (byPointer: returns &p)
./main.go:21:13: make([]int, 8) does not escape (localSum: small, constant size, stays inside)
./main.go:34:13: make([]int, n) escapes to heap (dynamic: size unknown at compile time)
./main.go:40:14: n escapes to heap (printIt: stored in an interface for fmt.Println)
./main.go:47:24: func literal escapes to heap (capture: a closure that outlives the call)
The usual reasons a value escapes: its address is returned or stored somewhere that outlives the function; it is put in an interface (boxing); it is captured by a closure that escapes; its size is not a compile-time constant or it is large; or it is passed to a function the compiler cannot see into (an interface method or a function value).
Why care? Heap allocation costs time at allocation, and work for the garbage collector later. Measured with testing.Benchmark on one goroutine:
| Operation | Time per operation | Memory per operation | Allocations |
|---|---|---|---|
| Return a small struct by value | 1.0 ns | 0 | 0 |
| Return a pointer that outlives the call | 26 ns | 24 B | 1 |
Put an int greater than 255 into an interface | 14 ns | 8 B | 1 |
| Append 1,000 ints to a nil slice | 9,150 ns | 25,208 B | 12 |
Append 1,000 ints to make([]int, 0, 1000) | 542 ns | 0 | 0 |
Build a string of 1,000 parts with += | 224,025 ns | 530,275 B | 999 |
The same with strings.Builder | 2,591 ns | 3,320 B | 9 |
A 4 KB buffer per call that escapes: make | 1,424 ns | 4,096 B | 1 |
The same buffer from a sync.Pool | 27 ns | 0 | 0 |
The lessons, in the order of how often they matter:
- Preallocate slices and maps when you know the size (
make([]T, 0, n)). Growing by append copies the data again and again: 12 allocations and 17 times the time here. (The preallocated slice of 1,000 ints did not even reach the heap: it is constant-sized and does not escape.) - Build strings with
strings.Builder(orbytes.Buffer), never with+=in a loop, which is quadratic: each step copies the whole string again. - A
sync.Poolreuses temporary objects (buffers, encoders), taking an allocation of 4 KB from 1.4 µs down to 27 ns. The pool may drop its contents at any collection, so it is a cache of scratch objects, never a place to keep state; and always reset what you take out. - Pointers are not "faster". Passing a small struct by value is often cheaper than passing a pointer, because the value stays on the stack and the collector never sees it. Use pointers for large structs, for things that must be mutated, and when identity matters.
- Measure before optimising, using
go test -bench -benchmemand the escape output, and look at the heap withpprof. An allocation inside a function that runs twice a second is not the problem.
7. The garbage collector
Go's collector is concurrent, non-moving, tri-colour mark and sweep, not generational. It marks the objects reachable from the roots (stacks and globals) while your goroutines keep running, with brief stop-the-world pauses at the start and the end of a cycle (usually well under a millisecond), and then frees the rest. Because it runs concurrently, its cost is not mostly pauses: it is CPU. The collector uses about a quarter of the CPUs while marking, and when a goroutine allocates faster than the collector can keep up, that goroutine is made to help ("assist"), which slows it down.
The central setting is GOGC (default 100): a new cycle starts when the heap has grown by GOGC percent over the live heap measured after the previous cycle. With 200 MB of live data and GOGC=100, the heap may grow to about 400 MB before the next collection; with GOGC=400, to about 1 GB. A higher value means fewer collections and more memory; a lower value, the reverse. Since Go 1.19 there is also GOMEMLIMIT, a soft limit on total memory: as the process nears it, the collector runs more often, so the limit prevents an out-of-memory kill but can make the program spend most of its time collecting.
Measured: a program holds 200 MB of live data (200,000 objects of 1 KB) and two goroutines allocate 4 million short-lived 1 KB objects (about 4 GB in total). Medians of three runs:
| Setting | Wall time | Collections | Total stop-the-world pause | Peak memory from the system |
|---|---|---|---|---|
GOGC=50 | 1,061 ms | 36 | 3.4 ms | 386 MB |
GOGC=100 (default) | 888 ms | 17 | 1.7 ms | 558 MB |
GOGC=200 | 793 ms | 9 | 0.8 ms | 813 MB |
GOGC=400 | 936 ms | 5 | 0.6 ms | 1,275 MB |
GOGC=800 | 1,030 ms | 3 | 0.3 ms | 2,181 MB |
GOGC=off, GOMEMLIMIT=300MiB | 1,351 ms | 63 | 5.9 ms | 330 MB |
What this shows:
- Pauses are not the cost. Even with 63 collections, the total pause was a few milliseconds over more than a second of work. A Go service is far more likely to suffer from the CPU spent collecting than from stop-the-world pauses.
- There is a trade of memory for time, and it flattens out: going from 50 to 200 gave a 25% shorter run for 2.1 times the memory, and past 200 the extra memory bought nothing (fresh memory has its own cost: page faults). The default is a sensible middle.
- A tight memory limit is expensive: capping the process only 100 MB above its live data made the collector run 63 times and the run took 50% longer than the default. Leave headroom of at least a fraction of the live heap, or reduce what you allocate.
The best way to speed up the collector is therefore the previous section: allocate less and keep fewer pointers. Fewer objects mean less to mark. Large arrays of pointer-free data ([]int, []byte) cost far less to scan than slices of pointers or of structs that contain pointers. To look at what the collector is doing, run with GODEBUG=gctrace=1 (one line per cycle, with the heap sizes and the CPU time) and use runtime.ReadMemStats or the runtime/metrics package for numbers in production.
8. Language behaviours that surprise people
The small program in the appendix prints each of these.
- The loop variable. Before Go 1.22 all iterations of a
forloop shared one variable, so goroutines started in a loop all saw its last value. Since Go 1.22 (when thegoline ofgo.modis 1.22 or later), each iteration has its own copy. This experiment started three goroutines in a loop and they saw[0 1 2]. If your module declares an older Go version, the old behaviour still applies: check thego.mod. - Slices share memory. A slice is a small header (pointer, length, capacity) over an array.
b := a[:2]sharesa's array, andappend(b, 99)writes intoa's memory if the capacity allows. The experiment printeda = [0 0 99] b = [0 0 99]for a slice with spare capacity, andc = [1 2 99] d = [1 2 99]for a plain three-element slice, because the sub-slicec[:2]still has room up to the end ofc's array. When a function stores or returns a sub-slice, copy if the caller must not see later changes:append([]int(nil), c[:2]...)printed an independent[1 2 100]and leftcalone. A retained small sub-slice also keeps the whole large array alive. - A nil pointer in an interface is not a nil interface. An interface value holds a type and a value, and it equals
nilonly if both are empty. A function that declaresvar e *myErrand returns it aserrorreturns a non-nil error even when the pointer is nil:mayFail(false) == nilprintedfalse. Return the literalnil. deferevaluates its arguments when thedeferstatement runs, not when the function exits, and deferred calls run last-in first-out. Adeferinside a loop runs when the function returns, not when the iteration ends, so deferringClosein a loop holds every file open until the end.- Maps cannot be used concurrently (section 4), the iteration order is deliberately randomised, and reading a missing key returns the zero value: use
v, ok := m[k]when the difference matters. - Errors are values. Wrap them with
fmt.Errorf("...: %w", err)and test witherrors.Isanderrors.As; do not compare error strings. Usepanicfor programmer errors and impossible states, not for expected failures, andrecoveronly at the boundary of a goroutine or a request.
9. Which tool for which question
| Question | Tool |
|---|---|
| Is there a data race? | go test -race, go run -race |
| Where does the time go (CPU)? | pprof CPU profile (net/http/pprof, or go test -cpuprofile) |
| Who allocates, and how much? | pprof heap and allocation profiles; go test -bench -benchmem |
| Did the compiler move this to the heap? | go build -gcflags='-m -l' |
| Are goroutines piling up, and where are they stuck? | pprof goroutine profile; runtime.NumGoroutine() as a metric |
| Why is latency uneven (scheduling, collection, blocking)? | go tool trace (the execution tracer) |
| What is the collector doing? | GODEBUG=gctrace=1, runtime/metrics |
| Is this change really faster? | a benchmark, run several times, compared with benchstat |
10. Common mistakes
- Starting goroutines without a way to stop them, or with a channel nobody will receive from (section 5). Look for every
goand ask what ends it. - Unbounded concurrency: one goroutine per item of a large input, hammering a database or an API. Use a worker pool or a semaphore.
- Sharing a variable between goroutines without synchronisation, including a "harmless" flag or counter. Run the tests with
-race. - Using channels where a mutex is simpler, or a mutex held across a slow call (a network request) so that every other goroutine waits.
- Closing a channel from the receiver, or from several senders, causing a panic.
- Forgetting that a goroutine's panic ends the program, unless recovered inside that goroutine:
recoverin the parent does not help. - Capturing a loop variable in old-style code, or ignoring the
goversion ingo.mod. - Leaking through
time.Afterin a hot loop, timers that are not stopped, tickers that are not stopped (ticker.Stop()), response bodies that are not closed. - Building strings with
+=and slices without capacity in hot paths. - Tuning the collector first. Reduce allocations before turning
GOGC, and useGOMEMLIMITas a safety net rather than a way of life. - Reading a benchmark as a law. The ratios here are specific to this hardware and these programs. Measure your own case.
11. Things to mention in an interview
- Explain the G, M, P model in two sentences: goroutines are multiplexed onto OS threads, a thread needs a P to run Go code, idle Ps steal work, and blocked network I/O parks the goroutine, not the thread.
- Give the cost of a goroutine (about 2 KB of stack, microseconds to start), and say that you still bound concurrency.
- State the channel rules (unbuffered synchronises, send on closed panics, only the sender closes, nil blocks forever,
selectis random) and when to prefer a mutex or atomic. - Describe a goroutine leak, how you would find it (goroutine profile,
NumGoroutine) and howcontextand buffered channels prevent it. - Mention the race detector and the happens-before rules of the memory model.
- Explain escape analysis with an example (returning
&x, interfaces, closures) and how you would check it (-gcflags=-m). - Describe the collector: concurrent mark and sweep, short pauses, the cost is CPU;
GOGCtrades memory for time,GOMEMLIMITis a soft ceiling; allocate less first. - Know the gotchas: loop variable (Go 1.22), slice aliasing, the nil interface, concurrent map writes being fatal.
- Quote numbers from your own measurements; it shows that you check.
Quick recap
- Goroutines cost about 2.6 KB and a few microseconds. Cheap, but not free and not bounded: bound the concurrency and give every goroutine an end.
- The scheduler runs Gs on Ms with a P each; network waits park the goroutine, idle Ps steal work, long loops are preempted after about 10 ms (measured: a 1 ms sleep took 22 ms behind a spinner).
- Channels transfer ownership and coordinate (about 100 to 250 ns per message); mutexes and atomics protect state (57 and 16 ns under contention); no sharing at all was 1.4 ns.
- Data races lose updates silently (measured: 14 to 28% of them); run
go test -race. - Leaks: an unbuffered send nobody receives leaked 10,001 goroutines and 30 MB; a buffer of one fixed it. Use
contextandselectonctx.Done(). - Escape analysis decides stack or heap; preallocate (17 times faster), use
strings.Builder(86 times faster), and async.Poolfor scratch buffers (27 ns instead of 1.4 µs). - The collector's cost is CPU, not pauses.
GOGCtrades memory for time; a tightGOMEMLIMITmade the run 50% slower. Allocate less first. - Gotchas: loop variables (fixed in Go 1.22), slices share memory, a nil pointer in an interface is not nil, concurrent map writes are fatal.
Appendix: the code
The numbers come from four programs, all using only the standard library and Go 1.24. goexp holds the runtime experiments, one per command: go run . gcost, sched, sync, leak leaky, leak fixed, alloc, and gc (run the last with the settings in the environment, for example GOGC=200 go run . gc, or GOGC=off GOMEMLIMIT=300MiB go run . gc). escape is the file whose escape analysis is quoted in section 6 (go build -gcflags='-m -l'). gotchas prints the behaviours of section 8 (go run . loopvar, slice, nilerr, mapcrash, race, and go run -race . race for the detector). Note that the module's go.mod must say go 1.22 or later for the loop variable result.
goexp/main.go
// goexp: small experiments on the Go runtime. Usage: go run . gcost|sched|sync|leak leaky|leak fixed|alloc|gc
package main
import (
"crypto/sha256"
"fmt"
"io"
"os"
"runtime"
"runtime/debug"
"strconv"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
func ms(d time.Duration) float64 { return float64(d.Microseconds()) / 1000 }
// ---------- 1. What a goroutine costs ----------
func gcost() {
fmt.Println("goroutines parked on a channel: memory taken and time to start them")
for _, n := range []int{1_000, 100_000, 1_000_000} {
runtime.GC()
var a, b runtime.MemStats
runtime.ReadMemStats(&a)
stop := make(chan struct{})
var started sync.WaitGroup
started.Add(n)
t := time.Now()
for i := 0; i < n; i++ {
go func() { started.Done(); <-stop }()
}
started.Wait()
el := time.Since(t)
runtime.ReadMemStats(&b)
fmt.Printf(" %9d goroutines: %7.1f ms to start all (%.2f us each), %6.1f MB from the system = %.2f KB each (stacks in use: %.2f KB each)\n",
n, ms(el), float64(el.Nanoseconds())/float64(n)/1000, float64(b.Sys-a.Sys)/1e6, float64(b.Sys-a.Sys)/float64(n)/1024,
float64(b.StackInuse-a.StackInuse)/float64(n)/1024)
close(stop)
time.Sleep(200 * time.Millisecond)
}
}
// ---------- 2. The scheduler ----------
func burn(rounds int) {
var buf [32]byte
for i := 0; i < rounds; i++ {
buf = sha256.Sum256(buf[:])
}
}
func sched() {
fmt.Printf("this machine: %d logical CPUs\n", runtime.NumCPU())
const total = 400_000 // SHA-256 rounds, split over goroutines
for _, procs := range []int{1, 2, 4} {
runtime.GOMAXPROCS(procs)
for _, g := range []int{1, 2, 8} {
var wg sync.WaitGroup
t := time.Now()
for i := 0; i < g; i++ {
wg.Add(1)
go func() { defer wg.Done(); burn(total / g) }()
}
wg.Wait()
fmt.Printf(" GOMAXPROCS=%d, %d goroutine(s) sharing the same CPU work: %7.1f ms\n", procs, g, ms(time.Since(t)))
}
}
runtime.GOMAXPROCS(1)
// Concurrency without parallelism: 10,000 goroutines that each wait 100 ms
var wg sync.WaitGroup
t := time.Now()
for i := 0; i < 10_000; i++ {
wg.Add(1)
go func() { defer wg.Done(); time.Sleep(100 * time.Millisecond) }()
}
wg.Wait()
fmt.Printf("GOMAXPROCS=1: 10,000 goroutines that each sleep 100 ms finish in %.0f ms (waiting overlaps)\n", ms(time.Since(t)))
// Preemption: a goroutine in a loop with no function calls and no waiting, on a single P
var spin atomic.Bool
spin.Store(true)
go func() {
x := 0
for spin.Load() { // plain loop, no blocking call
x++
}
}()
time.Sleep(10 * time.Millisecond) // let the spinner start
var worst, sum time.Duration
const reps = 20
for i := 0; i < reps; i++ {
t = time.Now()
time.Sleep(time.Millisecond)
d := time.Since(t)
sum += d
if d > worst {
worst = d
}
}
fmt.Printf("GOMAXPROCS=1 with a goroutine spinning in a loop: a 1 ms sleep took %.1f ms on average, %.1f ms at worst (the spinner is preempted, but only after its time slice)\n",
ms(sum)/reps, ms(worst))
spin.Store(false)
runtime.GOMAXPROCS(runtime.NumCPU())
}
// ---------- 3. Channels, mutexes and atomics ----------
func syncCosts() {
const workers, perWorker = 4, 1_000_000
total := workers * perWorker
run := func(name string, f func(inc func())) {
t := time.Now()
f(nil)
el := time.Since(t)
fmt.Printf(" %-44s %8.1f ms %6.1f ns per increment\n", name, ms(el), float64(el.Nanoseconds())/float64(total))
}
fmt.Printf("%d goroutines each add 1 to a shared counter %d times (%d CPUs)\n", workers, perWorker, runtime.NumCPU())
run("sync.Mutex", func(func()) {
var mu sync.Mutex
var n int
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < perWorker; i++ {
mu.Lock()
n++
mu.Unlock()
}
}()
}
wg.Wait()
if n != total {
panic("lost updates")
}
})
run("atomic.Int64", func(func()) {
var n atomic.Int64
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < perWorker; i++ {
n.Add(1)
}
}()
}
wg.Wait()
if n.Load() != int64(total) {
panic("lost updates")
}
})
run("one counter per goroutine, summed at the end", func(func()) {
parts := make([]struct {
n int
_ [56]byte // padding so each counter has its own cache line
}, workers)
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < perWorker; i++ {
parts[w].n++
}
}()
}
wg.Wait()
sum := 0
for _, p := range parts {
sum += p.n
}
if sum != total {
panic("lost updates")
}
})
run("send each increment over a channel to one owner", func(func()) {
ch := make(chan int, 256)
done := make(chan int)
go func() {
n := 0
for v := range ch {
n += v
}
done <- n
}()
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < perWorker; i++ {
ch <- 1
}
}()
}
wg.Wait()
close(ch)
if <-done != total {
panic("lost updates")
}
})
fmt.Println("channel hand-off between two goroutines")
const msgs = 1_000_000
{ // unbuffered ping-pong
ping, pong := make(chan int), make(chan int)
go func() {
for v := range ping {
pong <- v
}
close(pong)
}()
t := time.Now()
for i := 0; i < msgs/10; i++ {
ping <- i
<-pong
}
el := time.Since(t)
close(ping)
fmt.Printf(" unbuffered round trip (send and receive back): %6.0f ns per round trip\n", float64(el.Nanoseconds())/float64(msgs/10))
}
for _, size := range []int{0, 1, 128} { // one-way stream
ch := make(chan int, size)
done := make(chan struct{})
go func() {
for range ch {
}
close(done)
}()
t := time.Now()
for i := 0; i < msgs; i++ {
ch <- i
}
close(ch)
<-done
el := time.Since(t)
fmt.Printf(" one-way stream, buffer %3d: %6.0f ns per message (%.1f million per second)\n", size, float64(el.Nanoseconds())/msgs, msgs/el.Seconds()/1e6)
}
}
// ---------- 4. A goroutine leak ----------
func slowCompute() int { time.Sleep(5 * time.Millisecond); return 42 }
// fetchLeaky gives up after 1 ms, but the worker is left blocked on an unbuffered send nobody will receive.
func fetchLeaky() int {
ch := make(chan int)
go func() { ch <- slowCompute() }()
select {
case v := <-ch:
return v
case <-time.After(time.Millisecond):
return -1
}
}
// fetchFixed is the same with a buffer of one, so the worker can always finish its send and exit.
func fetchFixed() int {
ch := make(chan int, 1)
go func() { ch <- slowCompute() }()
select {
case v := <-ch:
return v
case <-time.After(time.Millisecond):
return -1
}
}
func leak(variant string) {
f, name := fetchLeaky, "unbuffered channel (leaks)"
if variant == "fixed" {
f, name = fetchFixed, "channel with a buffer of one"
}
runtime.GC()
var a, b runtime.MemStats
runtime.ReadMemStats(&a)
before := runtime.NumGoroutine()
for i := 0; i < 10_000; i++ {
f()
}
time.Sleep(100 * time.Millisecond) // let finished workers exit
runtime.GC()
runtime.ReadMemStats(&b)
fmt.Printf(" %-30s after 10,000 timed-out calls: goroutines %6d -> %6d, memory from the system +%.1f MB\n",
name, before, runtime.NumGoroutine(), float64(b.Sys-a.Sys)/1e6)
}
// ---------- 5. Allocation: stack, heap, and what to do about it ----------
type point struct{ x, y, z float64 }
//go:noinline
func newPointValue(i int) point { return point{float64(i), 1, 2} }
var sink *point
//go:noinline
func newPointPtr(i int) *point { return &point{float64(i), 1, 2} } // escapes: the caller keeps the pointer
var sinkAny any
func bench(name string, f func(b *testing.B)) {
r := testing.Benchmark(f)
fmt.Printf(" %-52s %9.1f ns/op %6d B/op %4d allocs/op\n", name, float64(r.NsPerOp()), r.AllocedBytesPerOp(), r.AllocsPerOp())
}
func alloc() {
fmt.Println("allocation costs (testing.Benchmark, one goroutine)")
bench("return a struct by value", func(b *testing.B) {
b.ReportAllocs()
var s float64
for i := 0; i < b.N; i++ {
s += newPointValue(i).x
}
_ = s
})
bench("return a pointer that outlives the call", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
sink = newPointPtr(i)
}
})
const n = 1000
bench("append 1,000 ints, no preallocation", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var s []int
for j := 0; j < n; j++ {
s = append(s, j)
}
_ = s
}
})
bench("append 1,000 ints, make([]int, 0, 1000)", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
s := make([]int, 0, n)
for j := 0; j < n; j++ {
s = append(s, j)
}
_ = s
}
})
bench("build a string of 1,000 parts with +=", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
s := ""
for j := 0; j < n; j++ {
s += "x"
}
_ = s
}
})
bench("build a string of 1,000 parts with strings.Builder", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var sb strings.Builder
for j := 0; j < n; j++ {
sb.WriteByte('x')
}
_ = sb.String()
}
})
bench("put an int > 255 in an interface (boxing)", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
sinkAny = i + 1000
}
})
pool := sync.Pool{New: func() any { b := make([]byte, 0, 4096); return &b }}
bench("a 4 KB scratch buffer per call: make (escapes)", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
buf := make([]byte, 0, 4096)
buf = strconv.AppendInt(buf, int64(i), 10)
out.Write(buf) // an interface call: the compiler cannot prove the buffer is not kept, so it goes to the heap
}
})
bench("a 4 KB scratch buffer per call: sync.Pool", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
bp := pool.Get().(*[]byte)
buf := strconv.AppendInt((*bp)[:0], int64(i), 10)
out.Write(buf)
*bp = buf
pool.Put(bp)
}
})
}
var out io.Writer = io.Discard
// ---------- 6. The garbage collector: GOGC and memory ----------
var keep [][]byte
func gcRun() {
// A live heap of about 200 MB that stays, plus a stream of short-lived 1 KB objects.
for i := 0; i < 200_000; i++ {
keep = append(keep, make([]byte, 1024))
}
var before runtime.MemStats
runtime.ReadMemStats(&before)
t := time.Now()
var wg sync.WaitGroup
for w := 0; w < 2; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 2_000_000; i++ {
b := make([]byte, 1024)
b[0] = byte(i)
out.Write(b[:1]) // escapes: the object is allocated on the heap
}
}()
}
wg.Wait()
el := time.Since(t)
var after runtime.MemStats
runtime.ReadMemStats(&after)
fmt.Printf("GOGC=%-4s GOMEMLIMIT=%-6s %7.0f ms collections %4d total pause %6.1f ms peak memory from the system %5.0f MB\n",
os.Getenv("GOGC"), orDash(os.Getenv("GOMEMLIMIT")), ms(el), after.NumGC-before.NumGC,
float64(after.PauseTotalNs-before.PauseTotalNs)/1e6, float64(after.Sys)/1e6)
_ = debug.SetGCPercent
}
func orDash(s string) string {
if s == "" {
return "-"
}
return s
}
func main() {
if len(os.Args) < 2 {
fmt.Println("usage: goexp gcost|sched|sync|leak leaky|leak fixed|alloc|gc")
return
}
switch os.Args[1] {
case "gcost":
gcost()
case "sched":
sched()
case "sync":
syncCosts()
case "leak":
leak(os.Args[2]) // "leaky" or "fixed"
case "alloc":
alloc()
case "gc":
gcRun()
}
}
escape/main.go
package main
import "fmt"
type point struct{ x, y int }
// stays on the stack: the value is copied to the caller
func byValue() point {
p := point{1, 2}
return p
}
// escapes: the caller receives a pointer to a variable that must outlive this call
func byPointer() *point {
p := point{1, 2}
return &p
}
// does not escape: the slice is small, of constant size, and does not leave the function
func localSum() int {
buf := make([]int, 8)
for i := range buf {
buf[i] = i
}
s := 0
for _, v := range buf {
s += v
}
return s
}
// escapes: the size is not known at compile time
func dynamic(n int) []int {
buf := make([]int, n)
return buf[:1]
}
// escapes: the value is stored in an interface that is passed on (fmt.Println takes ...any)
func printIt(n int) {
fmt.Println(n)
}
var saved []func() int
// escapes: the closure outlives the call and captures i
func capture(i int) {
saved = append(saved, func() int { return i })
}
func main() {
_ = byValue()
_ = byPointer()
_ = localSum()
_ = dynamic(100)
printIt(7)
capture(1)
}
gotchas/main.go
// gotchas: five Go behaviours that surprise people. Usage: go run . loopvar|slice|nilerr|mapcrash|race
package main
import (
"fmt"
"os"
"sort"
"sync"
)
type myErr struct{}
func (*myErr) Error() string { return "my error" }
func mayFail(fail bool) error {
var e *myErr // a nil pointer of a concrete type
if fail {
e = &myErr{}
}
return e // BUG: a non-nil interface holding a nil pointer when fail is false
}
func mayFailFixed(fail bool) error {
if fail {
return &myErr{}
}
return nil
}
func main() {
switch os.Args[1] {
case "loopvar":
var wg sync.WaitGroup
var mu sync.Mutex
var seen []int
for i := 0; i < 3; i++ {
wg.Add(1)
go func() {
defer wg.Done()
mu.Lock()
seen = append(seen, i) // since Go 1.22 each iteration has its own i
mu.Unlock()
}()
}
wg.Wait()
sort.Ints(seen)
fmt.Println("goroutines saw i =", seen)
case "slice":
a := make([]int, 3, 10)
b := a[:2]
b = append(b, 99) // fits in the capacity of a, so it overwrites a[2]
fmt.Println("a =", a, "b =", b)
c := []int{1, 2, 3}
d := c[:2]
d = append(d, 99) // also fits: capacity of c is 3
fmt.Println("c =", c, "d =", d)
e := append([]int(nil), c[:2]...) // a copy: independent memory
e = append(e, 100)
fmt.Println("c =", c, "e =", e)
case "nilerr":
fmt.Println("mayFail(false) == nil:", mayFail(false) == nil)
fmt.Println("mayFailFixed(false) == nil:", mayFailFixed(false) == nil)
case "mapcrash":
m := map[int]int{}
var wg sync.WaitGroup
for g := 0; g < 4; g++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 100000; i++ {
m[i%1000] = i
}
}()
}
wg.Wait()
fmt.Println("finished without a crash (not guaranteed)")
case "race":
var n int
var wg sync.WaitGroup
for g := 0; g < 2; g++ {
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < 1_000_000; i++ {
n++
}
}()
}
wg.Wait()
fmt.Println("n =", n, "(expected 2000000)")
}
}
Sources
- The Go Authors, The Go Memory Model, the happens-before rules for channels, mutexes,
sync.Once,WaitGroupand atomics. - The Go Authors, A Guide to the Go Garbage Collector:
GOGC,GOMEMLIMIT, and how the cost of collection is modelled. - Dmitry Vyukov, Scalable Go Scheduler Design Doc (the G, M, P model and work stealing), and the Go 1.14 release notes (asynchronous preemption).
- The Go Authors, Frequently Asked Questions: how do I know whether a variable is allocated on the heap or the stack?
- The Go Authors, Data Race Detector.
- Sameer Ajmani, Go Concurrency Patterns: Context and Go Concurrency Patterns: Pipelines and cancellation.
- David Chase and Russ Cox, Fixing For Loops in Go 1.22, and Andrew Gerrand, Go Slices: usage and internals.
- The Go Authors, package runtime (
GOMAXPROCS,GOGC,GOMEMLIMIT,GODEBUG) and Effective Go. - Katherine Cox-Buday, Concurrency in Go (O'Reilly, 2017), and Teiva Harsanyi, 100 Go Mistakes and How to Avoid Them (Manning, 2022).
Practise this topic: Go