CS Fundamentals Trainer · Guides
Garbage collection and memory management
Reachability, reference counting against tracing, generational collection, what heap headroom and collector choice cost (measured in Java, Go and Python), leaks in managed languages and sizing a heap in a container.
38 min read
Programs create objects constantly and something has to give the memory back. In C you do it by hand with free, and the bugs are famous: leaks (never freed), use after free and double free (freed too early or twice, a major source of security vulnerabilities). Most other languages take the job away from the programmer with automatic memory management, usually called garbage collection (GC). It works well enough that many developers never think about it, until a service shows latency spikes, a container is killed for exceeding its memory limit, or a heap grows without bound. This guide explains the two basic ideas (reference counting and tracing), the generational trick that makes collectors fast, and measures what they cost, using Python, Java and Go on a real machine. The scripts are in the appendix.
It builds on the memory hierarchy and operating systems, and the limits it mentions are the cgroup limits from containers.
1. Garbage is what you can no longer reach
The collector cannot know what the program will use; it can only know what the program can still reach. Memory is live if it can be reached by following references from the roots: global variables, the variables on each thread's stack, and CPU registers. Everything else is garbage, since no code can ever touch it again.
Diagram: Reachability: live objects can be reached from a root, garbage cannot
Here everything is reachable at first. If the program drops A→B and E→F, then B, C, D, F and G become garbage, even though F and G still point at each other. Notice that "reachable" is not the same as "needed": an object the program will never use again, but still has a reference to, is live as far as the collector is concerned. That is the source of every memory leak in a garbage-collected language.
There are two families of algorithms.
2. Reference counting
Every object carries a count of references to it. When a reference is added the count goes up, and when one is removed it goes down; at zero the object is freed at once. CPython, Swift (ARC), Objective-C and C++ shared_ptr work this way.
I checked the behaviour in Python:
| Experiment | Result |
|---|---|
a = Node(); del a | the destructor ran immediately at del |
| two nodes pointing at each other, then both names deleted | the objects were still alive (checked with a weak reference) |
gc.collect() | the cycle detector found the unreachable objects (a count of 9 here) and freed them |
Strengths: memory is returned the moment it becomes unused, there is no long pause, and destructors run at a predictable time (useful for files and locks). Weaknesses:
- Cycles are never freed by counting alone, as the second row shows. CPython therefore adds a separate cycle collector, and Swift and Rust's
Rcneedweakreferences where an object points back to its owner. - Every reference change costs work, including writes to shared counts, which is expensive across threads (atomic operations).
- Counting adds bookkeeping to each object. In Python building a list of a million small lists took about 0.22 s with the cycle detector enabled and 0.08 s with it disabled in my run: the detector's periodic scans of newly created container objects are measurable overhead (the default thresholds are
(2000, 10, 10): a scan of the youngest generation after 2,000 net new container objects).
3. Tracing: mark and sweep
A tracing collector does not count. Occasionally it stops to find out what is live: it marks every object reachable from the roots, then sweeps the heap and frees everything unmarked. Because it works from the roots, cycles are no problem. Step through the same seven objects with both approaches:
Interactive animation: mark sweep
Tracing found and freed the five garbage objects, including the F↔G cycle. With the option on, reference counting freed three of them as soon as their counts reached zero (B, D, then C) and leaked the cycle. The price of tracing is that the cost depends on how much is live (the collector walks every live object), not on how much garbage there is, and that unreachable memory is not returned at the moment it becomes unreachable but at the next collection.
Variations on the theme:
- Mark-sweep leaves the live objects where they are, so free memory is scattered in small holes (fragmentation), and a large allocation can fail although enough memory is free in total.
- Mark-compact slides the live objects together to remove the holes. Allocation becomes a pointer bump (fast), at the cost of moving objects and updating every reference to them.
- Copying collectors copy the live objects from one region to another and discard the whole old region, so the cost is proportional to the live data only. They need free space to copy into.
4. The generational hypothesis
Measurements of real programs show that most objects die young: temporary strings, request objects, iterators. A few live a long time (caches, configuration, sessions). Generational collectors use this: they split the heap into a small young generation, collected often and cheaply, and a large old generation, collected rarely.
Diagram: A generational heap: objects are born in the young generation and promoted if they survive
A minor collection only visits the live objects in the young generation (a few, since most are dead) and the references from old objects into it, so its cost is small and does not grow with the size of the whole heap. I measured it on Java 21: a program with 200 MB of long-lived data allocated 8,000 MB of short-lived objects (1 KB each) on a 1 GB heap with the Serial collector:
| Quantity | Result |
|---|---|
| garbage allocated | 8,000 MB |
| collections | 30, all young |
| total pause time | 57 ms, in a run of 1,631 ms |
| longest pause | 5.6 ms |
Thirty small pauses, and the 200 MB of old data were never touched. (The first version of this test allocated 4 GB in 63 ms, because the JIT compiler's escape analysis saw that the arrays never left the loop and removed them. The program has to keep each object reachable for a moment, here in a small ring buffer, before there is anything to collect; see compilers and runtimes.) This is why allocating many small short-lived objects is cheap on the JVM, and why "avoid creating objects" is advice for hot paths only, not a general rule. It is also the reason that medium-lived objects are the expensive kind: they survive the young collections, get promoted, and then die in the old generation where collecting them costs more (for example a request-scoped cache that lives for a few seconds).
5. When the old generation fills: heap headroom and collectors
To make the old data die too, I held 600 MB of live data and kept replacing entries at random while allocating 8,000 MB of garbage, and ran four Java collectors with three heap sizes (single runs; the machine has two cores):
| Heap | Serial | Parallel | G1 | ZGC |
|---|---|---|---|---|
| 700 MB | 25.5 s | 28.0 s | 14.9 s | OutOfMemoryError |
| 1 GB | 10.4 s | 6.8 s | 6.7 s | 5.4 s |
| 2 GB | 5.9 s | 5.6 s | 4.8 s | 3.3 s |
These are total run times of the same work. The effect that stands out is headroom, not the choice of collector: with only about 14% of the 700 MB heap free around the 600 MB of live data, Serial spent 304 full collections (23 of its 25.5 seconds in pauses); at 2 GB it took 4.3 times less time for identical work, with 7 full collections. When there is little free memory, the collector runs again and again, each time freeing little: the program spends its time collecting (the "GC death spiral", which ends in OutOfMemoryError: GC overhead limit exceeded or in an out-of-memory kill).
The pauses the application would feel in the 2 GB runs (the longest gap observed by the allocation loop):
| Collector | Longest gap at 2 GB | Design |
|---|---|---|
| Serial | 310 ms | single thread, stop the world; small heaps and small containers |
| Parallel | 228 ms | several threads, stop the world; best throughput for batch jobs |
| G1 | 71 ms | heap in regions; collects the regions with the most garbage first; the default; aims at a pause-time goal (200 ms by default) |
| ZGC | 60 ms in my run | does almost all its work concurrently with the application; pauses are designed to stay under a few milliseconds regardless of heap size, at the cost of some throughput and of needing extra memory |
On this small machine under memory pressure, ZGC's gap was larger than its design target because allocation stalls when the heap is nearly full: a concurrent collector must be able to free memory faster than the program allocates it, which again needs headroom. The summary of the trade-off is the classic triangle: throughput (how much work gets done), latency (how long the pauses are) and footprint (how much memory is used). You can optimise two of them, so a batch job picks Parallel, a latency-sensitive service picks G1 or ZGC, and a tiny container picks Serial.
5.1 Latency distributions: what a request actually experiences
Total run time and "time spent in GC" describe throughput. Users feel latency, and an average hides exactly the events a collector causes. I measured it directly: a request loop at a fixed 1,000 requests per second on top of 600 MB of long-lived data (each request allocates about 400 KB of short-lived objects, replaces four long-lived entries and reads fifty), for 40 seconds on a 2 GB heap, with the first 5 seconds discarded. Latency is measured from the time each request should have started, not from when the program got round to it (an "open loop"). Otherwise a pause that stops the program also stops the clock, and the stall is counted as one slow request instead of the hundreds that were due during it (the problem called coordinated omission). Java 21, two cores, one run per collector (script in the appendix); in brackets, the range over two runs I made:
| Collector | p50 | p99 | p99.9 | max | Requests over 10 ms | Application stopped (total / longest) |
|---|---|---|---|---|---|---|
| Serial | 0.14 ms | 62.6 ms (63 to 79) | 135.5 ms | 152.7 ms | 1,338 of 35,000 | 29 times, 1,485 ms / 152 ms |
| Parallel | 0.14 ms | 66.0 ms (65 to 66) | 221.5 ms | 252.4 ms | 1,044 | 35 times, 1,361 ms / 252 ms |
| G1 | 0.14 ms | 28.1 ms (28 to 33) | 75.5 ms | 92.2 ms | 778 | 30 times, 903 ms / 91 ms |
| ZGC | 0.15 ms | 0.35 ms (0.35 to 0.35) | 2.3 ms | 6.3 ms (6 to 23) | 0 (14 in the first run) | 39 times, 1 ms in total / 0.1 ms |
What the table shows:
- The median is the same in every row (0.14 to 0.15 ms). A benchmark that reports average or median latency cannot tell these collectors apart, although their p99 differs by a factor of 180.
- The tail follows the longest stop-the-world pause. The "longest stop" column is the time the application threads were halted (taken from the JVM's safepoint log), and the maximum latency is almost the same number: when the program stops for 152 ms, every request due in that window waits for it.
- Throughput loss and latency damage are different things. Serial stopped the application for 1.5 s of a 35-second window (about 4%), a modest throughput cost, but that 4% is concentrated in 29 stops, and the requests that hit them are 450 times slower than the median. A collector can look fine on throughput and be unacceptable at p99.
- Concurrent collection removes the pause but has a price. ZGC's stops were 0.1 ms; its work moved onto other threads, which on a two-core machine compete with the application. On a bigger machine there is more room; on a small container it can become CPU throttling (containers) or allocation stalls when the heap is nearly full. Measure on the hardware you will use.
- Report percentiles, over a long enough window. The p99.99 of a 35,000-request run is only 3 requests, so the right-hand columns are noisy (note the 6 to 23 ms range of ZGC's maximum). Run longer, repeat, and keep the raw distribution.
The same loop in Go (1.24, GOGC 25, 100 and 400 on 600 MB of live data, one 40-second run each; a fourth run repeated GOGC=100):
GOGC | p50 | p99 | p99.9 | max | Collections | GC CPU |
|---|---|---|---|---|---|---|
| 25 | 0.56 ms | 1.18 ms | 2.7 ms | 7.3 ms | 109 | 0.4% |
| 100 | 0.55 ms | 1.17 ms | 4.9 ms | 19.4 ms | 27 | 0.1% |
| 100 (second run) | 0.57 ms | 1.41 ms | 11.9 ms | 27.8 ms | 27 | 0.1% |
| 400 | 0.54 ms | 1.30 ms | 3.1 ms | 8.2 ms | 6 | 0.2% |
Go's collector is concurrent, so there is no long stop-the-world pause: the p99 stays near 1.2 to 1.4 ms in every run, and the tail of a few to 28 ms is not a long stop-the-world pause. I did not trace it to a single cause; likely contributors are allocating goroutines being asked to help with marking and the collector's threads sharing two cores with the application. The differences between GOGC settings are within the run-to-run noise (compare the two GOGC=100 rows), so the earlier table's advice stands: GOGC trades memory for CPU, and here it did not visibly change the latency. The comparison between the languages is not a ranking (different allocators, a different request cost, one machine): the lesson is how to measure, and that stop-the-world pauses, not the average cost of collection, set the tail.
6. Go: one knob
Go has a concurrent, non-generational mark-sweep collector and one main setting, GOGC (default 100): start the next collection when the heap has grown by this percentage over the live data after the last one. So the heap peaks at about live × (1 + GOGC/100). I ran a program with 200 MB of live data and 8 GB of short-lived allocation:
| GOGC | Collections | Peak heap in use | GC CPU fraction | Total pause |
|---|---|---|---|---|
| 25 | 189 | 255 MB | 2.1% | 12.6 ms |
| 50 | 95 | 309 MB | 1.4% | 7.6 ms |
| 100 | 47 | 407 MB | 0.7% | 3.1 ms |
| 200 | 24 | 609 MB | 0.5% | 4.5 ms |
| 400 | 12 | 1,032 MB | 0.2% | 1.3 ms |
| 800 | 6 | 1,878 MB | 0.1% | 0.5 ms |
Each doubling of GOGC halves the number of collections and roughly doubles the extra memory: memory buys CPU. It is a direct trade you can make deliberately. Since Go 1.19 there is also GOMEMLIMIT, a soft limit on the total memory that makes the collector work harder as the process approaches it, which is the right way to run in a container: set it a little below the container limit (see below).
7. Memory leaks in a language with garbage collection
A collector frees unreachable memory, so a "leak" means keeping a reference you no longer need. The usual causes:
- An unbounded cache or map (a static
Mapkeyed by user id that is only ever added to). Use a size limit and an eviction policy, or a cache library. - Listeners and callbacks registered and never removed; a closure that captures a large object.
- Collections that live as long as the process (a list of "recent requests" that is never trimmed).
- Thread-locals and thread pools that hold per-request data; sessions that are never expired.
- Native or off-heap memory, and file handles, which the collector does not account for. Close them deterministically (
with,try-with-resources,defer) instead of waiting for a finalizer. - Finalizers are unreliable: they run late, or never. Avoid them.
Weak references (WeakReference, Python's weakref) point to an object without keeping it alive, which is how caches and observer lists avoid leaks. To find a leak, look at the heap usage after collections over hours (a staircase that never comes down), then take a heap dump or profile and look at what retains the largest objects.
7.1 What the collector does not see, or sees late
"The collector freed it" and "the process gave the memory back" are different statements, and several things sit between them. All of the following were measured (Java 21, Python 3.13; scripts in the appendix) unless a sentence says it comes from the documentation.
Native memory is invisible to the heap. A Java ByteBuffer.allocateDirect allocates outside the heap; the small Java object that owns it is what the collector tracks, and the native block is released only when that object is collected. I allocated 4,000 one-megabyte direct buffers and dropped each at once, with a 256 MB heap:
| Setting | Result |
|---|---|
| limit of 128 MB for direct memory, default flags | completed (peak RSS 188 MB): when the limit is reached the JVM calls System.gc() itself and frees the dropped buffers |
the same, with -XX:+DisableExplicitGC (a common production flag) | OutOfMemoryError: Cannot reserve 1048576 bytes of direct buffer memory with only 5 MB of the heap in use |
| a 4 GB limit and a quiet heap | completed, but process RSS peaked at 1,909 MB while the Java heap held 45 MB: no heap pressure, so no collection, so no release |
A service that uses direct buffers heavily (network libraries, off-heap caches) can exhaust container memory with an almost empty heap. Bound the direct memory (-XX:MaxDirectMemorySize), reuse buffers from a pool, close or release native resources explicitly, and monitor RSS and native memory (-XX:NativeMemoryTracking) next to the heap.
Finalizers delay reclamation. An object with a finalize() method cannot be freed at its first collection: the collector queues it, a finalizer thread runs the method later, and only the next collection frees the memory. I created 100,000 unreachable 1 KB objects, once with and once without a finalizer, and called System.gc() twice:
| Objects | Heap in use after the first gc | After the second gc |
|---|---|---|
| plain | 1 MB | 1 MB |
with finalize() | 85 MB (19,229 finalizers run) | 1 MB (all 100,000 run) |
Finalizers also run at an unpredictable time, on a thread you do not control, and they can resurrect the object; Java has deprecated finalization for removal. For releasing a resource use try-with-resources (or with, defer) deterministically, and for a safety net use java.lang.ref.Cleaner, which does not keep the object alive for another cycle. The same advice holds for __del__ in Python and destructors elsewhere: never depend on them to free scarce things such as file handles, sockets or locks.
Weak and soft references have different rules. After one collection the weak reference was cleared (true), while the soft reference was still alive (false) and a WeakHashMap kept only the entry whose key was still held strongly (size 1 of 2). Soft references were cleared only when I filled the heap, just before an OutOfMemoryError would have happened (about 240 MB of allocation later). So: a weak reference is for "do not keep this alive" (an observer list, a canonicalising map), and it can disappear at the next collection, which makes it a poor cache; a soft reference is a memory-sensitive cache in principle, but its lifetime depends on heap size and collector policy, so a bounded cache with an explicit size and eviction policy (an LRU) is more predictable.
Why the process's memory stays high after the data is gone. Three numbers are involved: what the program is using (live data), what the runtime has committed (reserved from the OS for its heap), and what the OS reports as the process's resident memory (RSS). The collector lowers the first. The runtime decides about the other two. I allocated 800 MB of live arrays, dropped them, called System.gc() twice and waited:
| Collector (heap from 64 MB to 2 GB) | Heap in use after | Heap committed after | Process RSS after |
|---|---|---|---|
| Serial | 20 MB | 1,134 MB | 844 MB |
| Parallel | 9 MB | 1,213 MB | 1,130 MB |
| G1 | 1 MB | 64 MB | 80 MB (it shrinks the heap after a full collection) |
| ZGC | 6 MB | 896 MB | 922 MB (it uncommits memory only after a delay, 300 seconds by default) |
(G1 showed 1,601 MB in use with 800 MB of live data because 1 MB arrays with their header do not fit in one 1 MB region and each takes two: "humongous object" waste, a reminder that heap in use is not the same as data size.) The same effect without a managed heap: CPython allocates small objects in its own arenas and can return an arena to the OS only when every object in it is gone. I created 2,000,000 small objects (485 MB of RSS), then freed all of them (RSS back to 12 MB), but when I kept every twentieth object, scattered through all the arenas, and freed the other 95%, RSS stayed at 485 MB; malloc_trim brought it to 470 MB. Only when the survivors were freed too did it fall to 14 MB. An 800 MB single buffer, by contrast, was returned immediately (814 MB to 14 MB), because large blocks are mapped separately. Go's runtime returns memory to the OS in the background (the scavenger) and debug.FreeOSMemory() forces it; this I did not measure.
Why it matters: the container's limit counts resident memory, not live data. A process that peaked at 1.1 GB and holds it will be OOM-killed in a 1 GB container, or crowd out its neighbours, even though it is "using" 20 MB. What to do:
- Size limits from the peak resident memory, not from the steady state, and watch RSS and the heap side by side, with the trend. A heap that fell and an RSS that did not is this effect, not a leak.
- Choose and tune the runtime for the environment: on the JVM, a collector that returns memory (G1 with its periodic collection options, ZGC's uncommit settings, or the heap-shrinking flags of your collector, per the documentation), a smaller maximum heap, and the right percentage of the container limit.
- For fragmentation in long-running processes: recycle workers (for example a web server that restarts each worker after N requests or a memory threshold), process batches in a short-lived child process, avoid scattering a few long-lived small objects among many short-lived ones (build the long-lived structure first, or use compact arrays), and use arrays or memory-mapped files for bulk data.
8. Memory management in other languages
- Rust has no collector: ownership and borrowing rules checked at compile time decide where each value is freed, with
Rc/Arcwhere sharing is needed. You get deterministic freeing and no pauses, and you pay in learning the rules. - C++ uses RAII: destructors free resources when a scope ends, plus smart pointers.
- Swift uses automatic reference counting; cycles are the programmer's problem (
weak,unowned). - JavaScript (V8) uses a generational tracing collector with parallel and concurrent phases; .NET and Java use generational collectors with several modes; PHP and Ruby combine ideas.
- Arena (region) allocation frees a whole group of objects at once, such as everything for one request. It is a technique inside any language where you control allocation.
9. Containers and sizing
The heap is not the only memory a process uses: there is also thread stacks, code, metaspace, buffers and native allocations. A container's memory limit (see containers, where a 50 MB limit killed a process with exit status 137) counts all of it. So:
- Make the runtime read the container limit. Modern JVMs size the heap as a percentage of it (
-XX:MaxRAMPercentage, commonly 50 to 75), and Go can be givenGOMEMLIMIT. A heap set equal to the container limit leaves no room for anything else and ends in an OOM kill. - Leave headroom for the collector, as the 700 MB row of the table shows. A rule of thumb for tracing collectors is to give the heap 1.5 to 3 times the live data, depending on the collector and the allocation rate.
- Watch the right number: heap usage after collection (the live set) and its trend, the time spent in GC, and the pause distribution (p99), alongside the container's resident memory (observability).
10. Practical guidance
- Measure first. Turn on GC logging (
-Xlog:gc,GODEBUG=gctrace=1,gc.set_debug) or use the metrics; most programs never need tuning. - Reduce allocation only where a profile shows it matters: reuse buffers in hot loops, avoid building big intermediate collections, pre-size collections. Do not contort ordinary code.
- Give the heap room before reaching for exotic flags.
- Choose the collector for the goal: throughput (Parallel), balanced default (G1), very low latency or huge heaps (ZGC and similar), tiny footprint (Serial).
- Watch tail latency: a stop-the-world pause is felt by every in-flight request at once and looks like a latency spike on every instance at the same time.
- Keep objects small and lifetimes short where you can: short-lived is nearly free; medium-lived is the expensive case.
- For caches, bound the size.
11. Common mistakes
- Believing that garbage collection means leaks are impossible.
- Setting the heap equal to the container's memory limit.
- A heap that is barely larger than the live set, so that the program is always collecting.
- Tuning flags copied from a blog post for another collector and workload.
- Unbounded caches, maps and queues.
- Relying on finalizers or destructors to release scarce resources (files, connections, locks).
- Averaging latency: GC pauses show up in the tail, not in the mean.
- Allocating big short-lived objects (large arrays) in a loop, which may be allocated straight into the old generation and defeat the young-generation trick.
12. Things to mention in an interview
- Live means reachable from roots; garbage is what is not; "needed" is not the same as "reachable", which is why leaks exist.
- Reference counting against tracing, with cycles as the key difference, and how CPython combines both.
- Mark-sweep, mark-compact and copying, and fragmentation.
- The generational hypothesis, minor against major collections, promotion, and why medium-lived objects are costly.
- The throughput, latency and footprint triangle, and the main Java collectors (Serial, Parallel, G1, ZGC) with Go's
GOGCandGOMEMLIMIT. - Why heap headroom matters (the death spiral), and sizing the heap inside a container.
- Typical leaks in managed languages and how to find them.
- Rust's ownership as the alternative.
Quick recap
- A collector frees what the program can no longer reach from its roots; a program that keeps a reference keeps the memory, which is how leaks happen in managed languages.
- Reference counting frees at once but cannot free cycles (the model and the Python test both leaked them until the cycle detector ran); tracing (mark and sweep) handles cycles but its cost grows with the live data.
- Generational collectors make allocation cheap: 8,000 MB of short-lived garbage cost 30 small pauses totalling 57 ms.
- Headroom beats tuning: the same work took 25.5 s on a tight heap and 5.9 s on a roomy one with Serial, and ZGC failed outright on the tight one. In Go each doubling of
GOGChalved the collections and doubled the extra memory. - In containers, make the runtime read the limit, leave room for non-heap memory, and watch the live set after collection.
Appendix: the experiment code
Four files: a shell script that runs the Java workload (gc.sh), the Java program it runs (GcDemo.java, JDK 21), a Go program (gogc.go, Go 1.24) and a Python script (refs.py, Python 3.13). The figures come from single runs on a small two-core virtual machine, so absolute times vary from run to run and small differences between collectors should not be read as rankings; the large effects (heap headroom, cycles, the number of collections falling as GOGC rises) are repeatable. ZGC's pauses are not logged as Pause lines, so for every collector the guide uses the longest gap the allocation loop itself observed, which is what an application would feel. The animation is a model of seven objects, not a measurement.
gc.sh: four Java collectors and three heap sizes
#!/bin/bash
# Java: the same allocation workload under four collectors and three heap sizes (needs a JDK 21; GcDemo.java is the next file).
javac GcDemo.java
run() { name=$1; shift
java "$@" -Xlog:gc:file=gc.log GcDemo "$LIVE_MB" 8000 $CHURN 2>&1 | sed "s/^/$name: /"
grep -o 'Pause.* [0-9.]*ms$' gc.log | grep -o '[0-9.]*ms$' | tr -d ms |
awk -v n="$name" 'NF{c++; s+=$1; if($1>m)m=$1} END{printf "%s: %d pauses, total %.0f ms, longest %.1f ms\n", n, c, s, m}'
grep -c "Pause Full" gc.log | sed "s/^/$name: full collections: /"
}
# 1. a live set that never changes: young collections are all that is needed
LIVE_MB=200 CHURN=
run "Serial 1g" -XX:+UseSerialGC -Xmx1g -Xms1g
# 2. a live set of 600 MB whose entries are replaced all the time, so old data dies too
LIVE_MB=600 CHURN=churn
for x in 700m 1g 2g; do
run "Serial $x" -XX:+UseSerialGC -Xmx$x -Xms$x
run "Parallel $x" -XX:+UseParallelGC -Xmx$x -Xms$x
run "G1 $x" -XX:+UseG1GC -Xmx$x -Xms$x
run "ZGC $x" -XX:+UseZGC -Xmx$x -Xms$x
done
GcDemo.java: short-lived garbage on top of long-lived data
import java.util.*;
/** Allocates lots of short-lived garbage while holding a long-lived set; records the worst pause seen by the loop. */
public class GcDemo {
public static void main(String[] args) {
int liveMb = Integer.parseInt(args[0]); // size of the long-lived data
int totalMb = Integer.parseInt(args[1]); // total garbage to allocate
boolean churn = args.length > 2; // replace long-lived entries now and then, so old data dies too
Random rnd = new Random(1);
List<byte[]> live = new ArrayList<>();
for (int i = 0; i < liveMb * 1024 / 16; i++) live.add(new byte[16 * 1024]);
byte[][] ring = new byte[4096][]; // keeps each object alive for a moment so the JIT cannot remove it
long maxGap = 0, sink = 0, t0 = System.nanoTime(), last = t0;
for (long allocated = 0; allocated < totalMb * 1024L * 1024; allocated += 1024) {
byte[] garbage = new byte[1024]; // dies soon: overwritten 4096 allocations later
ring[(int) ((allocated >> 10) & 4095)] = garbage;
sink += garbage.length;
if (churn && (allocated >> 10) % 16 == 0) live.set(rnd.nextInt(live.size()), new byte[16 * 1024]);
if ((allocated & 0xFFFF) == 0) { // look at the clock every 64 KB allocated
long now = System.nanoTime(); maxGap = Math.max(maxGap, now - last); last = now;
}
}
long total = System.nanoTime() - t0;
System.out.printf("live=%d MB, garbage=%d MB, total %d ms, longest gap between checks %.1f ms (%d)%n",
liveMb, totalMb, total / 1_000_000, maxGap / 1e6, sink % 2);
}
}
gogc.go: how Go's GOGC trades memory for CPU
// gogc.go: how Go's GOGC setting trades memory for CPU. A 200 MB live heap, 8 GB of short-lived garbage.
package main
import (
"fmt"
"os"
"runtime"
"runtime/debug"
"strconv"
"time"
)
var sink [][]byte
func main() {
pct, _ := strconv.Atoi(os.Args[1])
debug.SetGCPercent(pct)
live := make([][]byte, 0, 12800)
for i := 0; i < 12800; i++ {
live = append(live, make([]byte, 16*1024))
}
sink = make([][]byte, 4096)
start := time.Now()
var peak uint64
var m runtime.MemStats
for i := 0; i < 8*1024*1024; i++ { // 8 million allocations of 1 KB = 8 GB
sink[i&4095] = make([]byte, 1024)
if i%100000 == 0 {
runtime.ReadMemStats(&m)
if m.HeapAlloc > peak {
peak = m.HeapAlloc
}
}
}
runtime.ReadMemStats(&m)
fmt.Printf("GOGC=%-4d run %5d ms, %4d collections, GC pauses total %5.1f ms, GC CPU fraction %.1f%%, peak heap in use %4d MB (live data %d MB)\n",
pct, time.Since(start).Milliseconds(), m.NumGC, float64(m.PauseTotalNs)/1e6, m.GCCPUFraction*100, peak>>20, len(live)*16/1024)
runtime.KeepAlive(live)
}
refs.py: reference counting, a cycle and the cycle collector (Python)
import gc, sys, weakref, time
class Node:
def __init__(self, name): self.name = name; self.other = None
def __del__(self): print(f" freed {self.name}")
print("1. reference counting frees an object the moment the last reference goes")
a = Node("a"); print(" refcount of a:", sys.getrefcount(a) - 1); del a
print("2. a cycle keeps its members alive, although nothing can reach them")
gc.disable()
x, y = Node("x"), Node("y"); x.other, y.other = y, x
w = weakref.ref(x); del x, y
print(" after del: object still alive?", w() is not None)
print(" collector run, unreachable objects found:", gc.collect())
print(" alive afterwards?", w() is not None)
print("3. cost of the cycle detector: 1,000,000 small objects in lists")
gc.enable()
for enabled in (True, False):
(gc.enable if enabled else gc.disable)()
t = time.perf_counter(); data = [[i] for i in range(1_000_000)]; dt = time.perf_counter() - t
print(f" cyclic collector {'on ' if enabled else 'off'}: building took {dt:.2f} s"); del data
gc.enable()
LatencyDemo.java and latency.sh: percentiles of a request loop under four Java collectors
JDK 21. The loop is open (latency is measured from when each request was due), and the script totals the time the application threads were stopped, from the JVM's safepoint log, after the warm-up.
import java.util.*;
import java.util.concurrent.locks.LockSupport;
/** A request loop at a fixed rate on top of a large live data set. Latency is measured from the time each request SHOULD have started
* (open loop), so a pause that stops the program also delays every request that was due during it, as users would experience. */
public class LatencyDemo {
public static void main(String[] args) {
int liveMb = Integer.parseInt(args[0]), rate = Integer.parseInt(args[1]), seconds = Integer.parseInt(args[2]);
Random rnd = new Random(1);
byte[][] live = new byte[liveMb * 1024 / 16][];
for (int i = 0; i < live.length; i++) live[i] = new byte[16 * 1024];
long interval = 1_000_000_000L / rate, warm = 5L * rate;
int n = rate * seconds; long[] lat = new long[n]; long sink = 0;
byte[][] ring = new byte[64][];
long start = System.nanoTime();
long measureFromMs = java.lang.management.ManagementFactory.getRuntimeMXBean().getUptime() + 5000; // pauses before this time (start-up, warm-up) are not counted
System.out.println("MEASURE_FROM_MS=" + measureFromMs);
for (int i = 0; i < n; i++) {
long due = start + i * interval;
long now = System.nanoTime();
while (now < due) { LockSupport.parkNanos(Math.min(due - now, 200_000)); now = System.nanoTime(); }
for (int k = 0; k < 100; k++) { byte[] b = new byte[4096]; b[k] = (byte) i; ring[(i + k) & 63] = b; sink += b[k]; } // about 400 KB of short-lived garbage
for (int k = 0; k < 4; k++) live[rnd.nextInt(live.length)] = new byte[16 * 1024]; // long-lived entries replaced: old data dies too
for (int k = 0; k < 50; k++) sink += live[rnd.nextInt(live.length)][k]; // reads of the live set
lat[i] = System.nanoTime() - due;
}
long[] s = Arrays.copyOfRange(lat, (int) warm, n); Arrays.sort(s);
java.util.function.DoubleUnaryOperator q = p -> s[(int) Math.min(s.length - 1, Math.floor(s.length * p))] / 1e6;
long over10 = Arrays.stream(s).filter(x -> x > 10_000_000L).count();
System.out.printf("latency ms: p50 %.2f p90 %.2f p99 %.2f p99.9 %.2f p99.99 %.2f max %.1f | requests over 10 ms: %d of %d (%d)%n",
q.applyAsDouble(.5), q.applyAsDouble(.9), q.applyAsDouble(.99), q.applyAsDouble(.999), q.applyAsDouble(.9999), s[s.length - 1] / 1e6, over10, s.length, sink % 2);
}
}
#!/bin/bash
# The same request loop under four Java collectors: latency percentiles, and the time the application was actually stopped. Needs JDK 21.
export JAVA_TOOL_OPTIONS=
javac LatencyDemo.java
for gc in Serial Parallel G1 Z; do
java -XX:+Use${gc}GC -Xmx2g -Xms2g -Xlog:safepoint:file=sp_$gc.log:uptime LatencyDemo 600 1000 40 > out_$gc.txt
from=$(grep -o 'MEASURE_FROM_MS=[0-9]*' out_$gc.txt | cut -d= -f2)
printf '%-9s %s\n' $gc "$(grep latency out_$gc.txt)"
# every safepoint after the warm-up: the time the application threads were stopped (in ms)
sed -n 's/^\[\([0-9.]*\)s\].*Safepoint "\([A-Za-z0-9_]*\)".*Total: \([0-9]*\) ns.*/\1 \2 \3/p' sp_$gc.log |
awk -v f=$from -v n=$gc 'BEGIN{FS=" "} $1*1000>=f {c++; t+=$3/1e6; if($3/1e6>m)m=$3/1e6; if($2 ~ /Collect|GC|Z|Cleanup|Pause/) g++} END{printf "%-9s application stopped %d times after warm-up, %.0f ms in total, longest %.1f ms\n", n, c, t, m}'
done
latency.go: the same loop in Go
Go 1.24. Run with GOGC=100 ./latency_go 600 1000 40 (600 MB of live data, 1,000 requests per second, 40 seconds).
package main
// The same open-loop request loop as LatencyDemo.java, in Go: latency is measured from the time each request should have started.
import (
"fmt"
"math/rand"
"os"
"runtime"
"sort"
"strconv"
"time"
)
func main() {
liveMB, _ := strconv.Atoi(os.Args[1])
rate, _ := strconv.Atoi(os.Args[2])
seconds, _ := strconv.Atoi(os.Args[3])
rnd := rand.New(rand.NewSource(1))
live := make([][]byte, liveMB*1024/16)
for i := range live {
live[i] = make([]byte, 16*1024)
}
interval := time.Second / time.Duration(rate)
n := rate * seconds
lat := make([]time.Duration, n)
ring := make([][]byte, 64)
var sink int
var ms0 runtime.MemStats
start := time.Now()
for i := 0; i < n; i++ {
due := start.Add(time.Duration(i) * interval)
if d := time.Until(due); d > 200*time.Microsecond {
time.Sleep(d - 100*time.Microsecond)
}
for time.Now().Before(due) {
}
if i == 5*rate {
runtime.ReadMemStats(&ms0)
}
for k := 0; k < 100; k++ {
b := make([]byte, 4096)
b[k] = byte(i)
ring[(i+k)&63] = b
sink += int(b[k])
}
for k := 0; k < 4; k++ {
live[rnd.Intn(len(live))] = make([]byte, 16*1024)
}
for k := 0; k < 50; k++ {
sink += int(live[rnd.Intn(len(live))][k])
}
lat[i] = time.Since(due)
}
var ms runtime.MemStats
runtime.ReadMemStats(&ms)
s := append([]time.Duration(nil), lat[5*rate:]...)
sort.Slice(s, func(a, b int) bool { return s[a] < s[b] })
q := func(p float64) float64 { return float64(s[int(float64(len(s))*p)].Microseconds()) / 1000 }
over := 0
for _, x := range s {
if x > 10*time.Millisecond {
over++
}
}
fmt.Printf("GOGC=%s: latency ms: p50 %.2f p90 %.2f p99 %.2f p99.9 %.2f p99.99 %.2f max %.1f | over 10 ms: %d of %d | GCs %d, GC CPU %.1f%%, peak heap %d MB (%d)\n",
os.Getenv("GOGC"), q(.5), q(.9), q(.99), q(.999), q(.9999), float64(s[len(s)-1].Microseconds())/1000, over, len(s),
ms.NumGC-ms0.NumGC, 100*ms.GCCPUFraction, ms.HeapSys>>20, sink%2)
}
Surprises.java: direct buffers, finalizers, references and RSS after collection
JDK 21. Run, for example, java -Xmx256m -XX:MaxDirectMemorySize=128m -XX:+DisableExplicitGC Surprises direct; the other modes are final, refs and rss (try it with -XX:+UseSerialGC, -XX:+UseG1GC and -XX:+UseZGC).
import java.lang.ref.*;
import java.nio.ByteBuffer;
import java.nio.file.*;
import java.util.*;
/** What the collector does not see, or does late: native (direct) buffers, finalizers, weak and soft references, and memory the process keeps after collection. */
public class Surprises {
static long rssMb() throws Exception {
for (String l : Files.readAllLines(Path.of("/proc/self/status"))) if (l.startsWith("VmRSS:")) return Long.parseLong(l.replaceAll("[^0-9]", "")) / 1024;
return -1;
}
static long heapUsedMb() { Runtime r = Runtime.getRuntime(); return (r.totalMemory() - r.freeMemory()) >> 20; }
static long heapCommittedMb() { return Runtime.getRuntime().totalMemory() >> 20; }
@SuppressWarnings({"removal", "deprecation"})
static class WithFinalizer { final byte[] data = new byte[1024]; static int ran; @Override protected void finalize() { ran++; } }
static class Plain { final byte[] data = new byte[1024]; }
public static void main(String[] a) throws Exception {
switch (a[0]) {
case "direct" -> { // 4,000 direct buffers of 1 MB, each dropped at once; the Java heap is almost untouched
long peak = 0;
try {
for (int i = 0; i < 4000; i++) { ByteBuffer.allocateDirect(1 << 20).put(0, (byte) 1); peak = Math.max(peak, rssMb()); }
System.out.printf("4,000 direct buffers of 1 MB: completed. Heap in use %d MB, process RSS peaked at %d MB%n", heapUsedMb(), peak);
} catch (OutOfMemoryError e) { System.out.printf("OutOfMemoryError: %s (heap in use only %d MB, RSS %d MB)%n", e.getMessage(), heapUsedMb(), rssMb()); }
}
case "final" -> {
for (boolean fin : new boolean[]{false, true}) {
WithFinalizer.ran = 0; Object[] keep = new Object[1];
for (int i = 0; i < 100_000; i++) keep[0] = fin ? new WithFinalizer() : new Plain(); // 100,000 objects of about 1 KB, all unreachable at once
long before = heapUsedMb(); System.gc(); long after1 = heapUsedMb(); int ran1 = WithFinalizer.ran;
Thread.sleep(300); System.gc(); long after2 = heapUsedMb();
System.out.printf("%-26s heap in use: %3d MB before gc, %3d MB after the first gc (finalizers run: %6d), %3d MB after the second (finalizers run: %6d)%n",
fin ? "objects with finalize()" : "plain objects", before, after1, ran1, after2, WithFinalizer.ran);
}
}
case "refs" -> {
Object strong = new byte[1 << 20]; WeakReference<Object> weak = new WeakReference<>(new byte[1 << 20]); SoftReference<Object> soft = new SoftReference<>(new byte[1 << 20]);
WeakHashMap<Object, String> whm = new WeakHashMap<>(); Object key = new Object(); whm.put(key, "kept"); whm.put(new Object(), "dropped");
System.gc(); Thread.sleep(100);
System.out.printf("after one gc: weak reference cleared=%b, soft reference cleared=%b, WeakHashMap size %d (one key still held strongly)%n", weak.get() == null, soft.get() == null, whm.size());
List<byte[]> pressure = new ArrayList<>();
try { while (soft.get() != null) pressure.add(new byte[8 << 20]); } catch (OutOfMemoryError e) { }
int n = pressure.size(); pressure.clear();
System.out.printf("under memory pressure (%d MB allocated until the heap was full): soft reference cleared=%b -- it is cleared before an OutOfMemoryError, but only then%n", n * 8, soft.get() == null);
Reference.reachabilityFence(strong); Reference.reachabilityFence(key);
}
case "rss" -> {
System.out.printf("start: RSS %4d MB, heap in use %4d MB, heap committed %4d MB%n", rssMb(), heapUsedMb(), heapCommittedMb());
List<byte[]> big = new ArrayList<>(); for (int i = 0; i < 800; i++) { byte[] b = new byte[1 << 20]; Arrays.fill(b, (byte) 1); big.add(b); }
System.out.printf("800 MB of live data: RSS %4d MB, heap in use %4d MB, heap committed %4d MB%n", rssMb(), heapUsedMb(), heapCommittedMb());
big = null; System.gc(); Thread.sleep(500); System.gc(); Thread.sleep(500);
System.out.printf("all of it dropped, after gc: RSS %4d MB, heap in use %4d MB, heap committed %4d MB%n", rssMb(), heapUsedMb(), heapCommittedMb());
}
}
}
}
rss_py.py: memory that Python frees but keeps
Python 3.13 on Linux (it reads /proc/self/status).
"""Why the memory of a Python process stays high after objects are freed: small objects live in arenas that can only be returned when completely empty."""
import ctypes, gc, os
def rss_mb():
for line in open("/proc/self/status"):
if line.startswith("VmRSS:"): return int(line.split()[1]) // 1024
class Rec: # a small object with a few fields, like a row or a parsed message
__slots__ = ("a", "b", "c")
def __init__(self, i): self.a = i; self.b = str(i); self.c = (i, i + 1)
N = 2_000_000
print(f"baseline RSS {rss_mb()} MB")
objs = [Rec(i) for i in range(N)]
print(f"{N:,} small objects alive: RSS {rss_mb()} MB")
objs = None; gc.collect()
print(f"all freed (refcount + gc.collect): RSS {rss_mb()} MB <- returned: the arenas became completely empty")
objs = [Rec(i) for i in range(N)]
keep = objs[::20] # 5% survive, scattered through every arena
objs = None; gc.collect()
print(f"95% freed, 5% kept (scattered): RSS {rss_mb()} MB <- the memory is free inside Python but not returned to the OS")
try: ctypes.CDLL("libc.so.6").malloc_trim(0); print(f"after malloc_trim(0): RSS {rss_mb()} MB <- helps very little: these small objects live in Python's own arenas, not in malloc blocks")
except OSError: pass
keep = None; gc.collect()
print(f"survivors freed too: RSS {rss_mb()} MB")
big = bytearray(800 * 1024 * 1024)
for i in range(0, len(big), 4096): big[i] = 1
print(f"one 800 MB buffer: RSS {rss_mb()} MB")
big = None
print(f"buffer freed: RSS {rss_mb()} MB <- large blocks are mapped separately and returned at once")
Sources
- Richard Jones, Antony Hosking and Eliot Moss, The Garbage Collection Handbook: The Art of Automatic Memory Management (CRC Press, 2nd ed., 2023).
- Oracle, HotSpot Virtual Machine Garbage Collection Tuning Guide (Serial, Parallel, G1, ZGC) and the OpenJDK ZGC wiki.
- The Go team, A Guide to the Go Garbage Collector (GOGC and GOMEMLIMIT) and Go's runtime.MemStats.
- Python documentation: gc, the garbage collector interface, weakref, and the CPython developer guide, Garbage collector design.
- Apple, The Swift Programming Language, Automatic Reference Counting; Steve Klabnik and Carol Nichols, The Rust Programming Language, chapter 4 "Understanding Ownership".
- Paul R. Wilson, Uniprocessor Garbage Collection Techniques (1992); David Ungar, Generation Scavenging (1984), the origin of the generational idea.
- Node.js and V8, Trash talk: the Orinoco garbage collector.
- Gil Tene, How NOT to Measure Latency (coordinated omission); the Go team's A Guide to the Go Garbage Collector.
- OpenJDK: JEP 421: Deprecate Finalization for Removal, JEP 377: ZGC and the
ByteBuffer.allocateDirectandjava.lang.ref.Cleanerdocumentation; Oracle, Java Garbage Collection Tuning Guide. - Python documentation, Memory management and the CPython source notes on
obmalloc(arenas and pools).
Practise this topic: CS Fundamentals