CS Fundamentals Trainer · Guides

Garbage collection and memory management

Reachability, reference counting against tracing, generational collection, what heap headroom and collector choice cost (measured in Java, Go and Python), leaks in managed languages and sizing a heap in a container.

38 min read

Programs create objects constantly and something has to give the memory back. In C you do it by hand with free, and the bugs are famous: leaks (never freed), use after free and double free (freed too early or twice, a major source of security vulnerabilities). Most other languages take the job away from the programmer with automatic memory management, usually called garbage collection (GC). It works well enough that many developers never think about it, until a service shows latency spikes, a container is killed for exceeding its memory limit, or a heap grows without bound. This guide explains the two basic ideas (reference counting and tracing), the generational trick that makes collectors fast, and measures what they cost, using Python, Java and Go on a real machine. The scripts are in the appendix.

It builds on the memory hierarchy and operating systems, and the limits it mentions are the cgroup limits from containers.

1. Garbage is what you can no longer reach

The collector cannot know what the program will use; it can only know what the program can still reach. Memory is live if it can be reached by following references from the roots: global variables, the variables on each thread's stack, and CPU registers. Everything else is garbage, since no code can ever touch it again.

Diagram: Reachability: live objects can be reached from a root, garbage cannot

Here everything is reachable at first. If the program drops A→B and E→F, then B, C, D, F and G become garbage, even though F and G still point at each other. Notice that "reachable" is not the same as "needed": an object the program will never use again, but still has a reference to, is live as far as the collector is concerned. That is the source of every memory leak in a garbage-collected language.

There are two families of algorithms.

2. Reference counting

Every object carries a count of references to it. When a reference is added the count goes up, and when one is removed it goes down; at zero the object is freed at once. CPython, Swift (ARC), Objective-C and C++ shared_ptr work this way.

I checked the behaviour in Python:

ExperimentResult
a = Node(); del athe destructor ran immediately at del
two nodes pointing at each other, then both names deletedthe objects were still alive (checked with a weak reference)
gc.collect()the cycle detector found the unreachable objects (a count of 9 here) and freed them

Strengths: memory is returned the moment it becomes unused, there is no long pause, and destructors run at a predictable time (useful for files and locks). Weaknesses:

3. Tracing: mark and sweep

A tracing collector does not count. Occasionally it stops to find out what is live: it marks every object reachable from the roots, then sweeps the heap and frees everything unmarked. Because it works from the roots, cycles are no problem. Step through the same seven objects with both approaches:

Interactive animation: mark sweep

Tracing found and freed the five garbage objects, including the F↔G cycle. With the option on, reference counting freed three of them as soon as their counts reached zero (B, D, then C) and leaked the cycle. The price of tracing is that the cost depends on how much is live (the collector walks every live object), not on how much garbage there is, and that unreachable memory is not returned at the moment it becomes unreachable but at the next collection.

Variations on the theme:

4. The generational hypothesis

Measurements of real programs show that most objects die young: temporary strings, request objects, iterators. A few live a long time (caches, configuration, sessions). Generational collectors use this: they split the heap into a small young generation, collected often and cheaply, and a large old generation, collected rarely.

Diagram: A generational heap: objects are born in the young generation and promoted if they survive

A minor collection only visits the live objects in the young generation (a few, since most are dead) and the references from old objects into it, so its cost is small and does not grow with the size of the whole heap. I measured it on Java 21: a program with 200 MB of long-lived data allocated 8,000 MB of short-lived objects (1 KB each) on a 1 GB heap with the Serial collector:

QuantityResult
garbage allocated8,000 MB
collections30, all young
total pause time57 ms, in a run of 1,631 ms
longest pause5.6 ms

Thirty small pauses, and the 200 MB of old data were never touched. (The first version of this test allocated 4 GB in 63 ms, because the JIT compiler's escape analysis saw that the arrays never left the loop and removed them. The program has to keep each object reachable for a moment, here in a small ring buffer, before there is anything to collect; see compilers and runtimes.) This is why allocating many small short-lived objects is cheap on the JVM, and why "avoid creating objects" is advice for hot paths only, not a general rule. It is also the reason that medium-lived objects are the expensive kind: they survive the young collections, get promoted, and then die in the old generation where collecting them costs more (for example a request-scoped cache that lives for a few seconds).

5. When the old generation fills: heap headroom and collectors

To make the old data die too, I held 600 MB of live data and kept replacing entries at random while allocating 8,000 MB of garbage, and ran four Java collectors with three heap sizes (single runs; the machine has two cores):

HeapSerialParallelG1ZGC
700 MB25.5 s28.0 s14.9 sOutOfMemoryError
1 GB10.4 s6.8 s6.7 s5.4 s
2 GB5.9 s5.6 s4.8 s3.3 s

These are total run times of the same work. The effect that stands out is headroom, not the choice of collector: with only about 14% of the 700 MB heap free around the 600 MB of live data, Serial spent 304 full collections (23 of its 25.5 seconds in pauses); at 2 GB it took 4.3 times less time for identical work, with 7 full collections. When there is little free memory, the collector runs again and again, each time freeing little: the program spends its time collecting (the "GC death spiral", which ends in OutOfMemoryError: GC overhead limit exceeded or in an out-of-memory kill).

The pauses the application would feel in the 2 GB runs (the longest gap observed by the allocation loop):

CollectorLongest gap at 2 GBDesign
Serial310 mssingle thread, stop the world; small heaps and small containers
Parallel228 msseveral threads, stop the world; best throughput for batch jobs
G171 msheap in regions; collects the regions with the most garbage first; the default; aims at a pause-time goal (200 ms by default)
ZGC60 ms in my rundoes almost all its work concurrently with the application; pauses are designed to stay under a few milliseconds regardless of heap size, at the cost of some throughput and of needing extra memory

On this small machine under memory pressure, ZGC's gap was larger than its design target because allocation stalls when the heap is nearly full: a concurrent collector must be able to free memory faster than the program allocates it, which again needs headroom. The summary of the trade-off is the classic triangle: throughput (how much work gets done), latency (how long the pauses are) and footprint (how much memory is used). You can optimise two of them, so a batch job picks Parallel, a latency-sensitive service picks G1 or ZGC, and a tiny container picks Serial.

5.1 Latency distributions: what a request actually experiences

Total run time and "time spent in GC" describe throughput. Users feel latency, and an average hides exactly the events a collector causes. I measured it directly: a request loop at a fixed 1,000 requests per second on top of 600 MB of long-lived data (each request allocates about 400 KB of short-lived objects, replaces four long-lived entries and reads fifty), for 40 seconds on a 2 GB heap, with the first 5 seconds discarded. Latency is measured from the time each request should have started, not from when the program got round to it (an "open loop"). Otherwise a pause that stops the program also stops the clock, and the stall is counted as one slow request instead of the hundreds that were due during it (the problem called coordinated omission). Java 21, two cores, one run per collector (script in the appendix); in brackets, the range over two runs I made:

Collectorp50p99p99.9maxRequests over 10 msApplication stopped (total / longest)
Serial0.14 ms62.6 ms (63 to 79)135.5 ms152.7 ms1,338 of 35,00029 times, 1,485 ms / 152 ms
Parallel0.14 ms66.0 ms (65 to 66)221.5 ms252.4 ms1,04435 times, 1,361 ms / 252 ms
G10.14 ms28.1 ms (28 to 33)75.5 ms92.2 ms77830 times, 903 ms / 91 ms
ZGC0.15 ms0.35 ms (0.35 to 0.35)2.3 ms6.3 ms (6 to 23)0 (14 in the first run)39 times, 1 ms in total / 0.1 ms

What the table shows:

The same loop in Go (1.24, GOGC 25, 100 and 400 on 600 MB of live data, one 40-second run each; a fourth run repeated GOGC=100):

GOGCp50p99p99.9maxCollectionsGC CPU
250.56 ms1.18 ms2.7 ms7.3 ms1090.4%
1000.55 ms1.17 ms4.9 ms19.4 ms270.1%
100 (second run)0.57 ms1.41 ms11.9 ms27.8 ms270.1%
4000.54 ms1.30 ms3.1 ms8.2 ms60.2%

Go's collector is concurrent, so there is no long stop-the-world pause: the p99 stays near 1.2 to 1.4 ms in every run, and the tail of a few to 28 ms is not a long stop-the-world pause. I did not trace it to a single cause; likely contributors are allocating goroutines being asked to help with marking and the collector's threads sharing two cores with the application. The differences between GOGC settings are within the run-to-run noise (compare the two GOGC=100 rows), so the earlier table's advice stands: GOGC trades memory for CPU, and here it did not visibly change the latency. The comparison between the languages is not a ranking (different allocators, a different request cost, one machine): the lesson is how to measure, and that stop-the-world pauses, not the average cost of collection, set the tail.

6. Go: one knob

Go has a concurrent, non-generational mark-sweep collector and one main setting, GOGC (default 100): start the next collection when the heap has grown by this percentage over the live data after the last one. So the heap peaks at about live × (1 + GOGC/100). I ran a program with 200 MB of live data and 8 GB of short-lived allocation:

GOGCCollectionsPeak heap in useGC CPU fractionTotal pause
25189255 MB2.1%12.6 ms
5095309 MB1.4%7.6 ms
10047407 MB0.7%3.1 ms
20024609 MB0.5%4.5 ms
400121,032 MB0.2%1.3 ms
80061,878 MB0.1%0.5 ms

Each doubling of GOGC halves the number of collections and roughly doubles the extra memory: memory buys CPU. It is a direct trade you can make deliberately. Since Go 1.19 there is also GOMEMLIMIT, a soft limit on the total memory that makes the collector work harder as the process approaches it, which is the right way to run in a container: set it a little below the container limit (see below).

7. Memory leaks in a language with garbage collection

A collector frees unreachable memory, so a "leak" means keeping a reference you no longer need. The usual causes:

Weak references (WeakReference, Python's weakref) point to an object without keeping it alive, which is how caches and observer lists avoid leaks. To find a leak, look at the heap usage after collections over hours (a staircase that never comes down), then take a heap dump or profile and look at what retains the largest objects.

7.1 What the collector does not see, or sees late

"The collector freed it" and "the process gave the memory back" are different statements, and several things sit between them. All of the following were measured (Java 21, Python 3.13; scripts in the appendix) unless a sentence says it comes from the documentation.

Native memory is invisible to the heap. A Java ByteBuffer.allocateDirect allocates outside the heap; the small Java object that owns it is what the collector tracks, and the native block is released only when that object is collected. I allocated 4,000 one-megabyte direct buffers and dropped each at once, with a 256 MB heap:

SettingResult
limit of 128 MB for direct memory, default flagscompleted (peak RSS 188 MB): when the limit is reached the JVM calls System.gc() itself and frees the dropped buffers
the same, with -XX:+DisableExplicitGC (a common production flag)OutOfMemoryError: Cannot reserve 1048576 bytes of direct buffer memory with only 5 MB of the heap in use
a 4 GB limit and a quiet heapcompleted, but process RSS peaked at 1,909 MB while the Java heap held 45 MB: no heap pressure, so no collection, so no release

A service that uses direct buffers heavily (network libraries, off-heap caches) can exhaust container memory with an almost empty heap. Bound the direct memory (-XX:MaxDirectMemorySize), reuse buffers from a pool, close or release native resources explicitly, and monitor RSS and native memory (-XX:NativeMemoryTracking) next to the heap.

Finalizers delay reclamation. An object with a finalize() method cannot be freed at its first collection: the collector queues it, a finalizer thread runs the method later, and only the next collection frees the memory. I created 100,000 unreachable 1 KB objects, once with and once without a finalizer, and called System.gc() twice:

ObjectsHeap in use after the first gcAfter the second gc
plain1 MB1 MB
with finalize()85 MB (19,229 finalizers run)1 MB (all 100,000 run)

Finalizers also run at an unpredictable time, on a thread you do not control, and they can resurrect the object; Java has deprecated finalization for removal. For releasing a resource use try-with-resources (or with, defer) deterministically, and for a safety net use java.lang.ref.Cleaner, which does not keep the object alive for another cycle. The same advice holds for __del__ in Python and destructors elsewhere: never depend on them to free scarce things such as file handles, sockets or locks.

Weak and soft references have different rules. After one collection the weak reference was cleared (true), while the soft reference was still alive (false) and a WeakHashMap kept only the entry whose key was still held strongly (size 1 of 2). Soft references were cleared only when I filled the heap, just before an OutOfMemoryError would have happened (about 240 MB of allocation later). So: a weak reference is for "do not keep this alive" (an observer list, a canonicalising map), and it can disappear at the next collection, which makes it a poor cache; a soft reference is a memory-sensitive cache in principle, but its lifetime depends on heap size and collector policy, so a bounded cache with an explicit size and eviction policy (an LRU) is more predictable.

Why the process's memory stays high after the data is gone. Three numbers are involved: what the program is using (live data), what the runtime has committed (reserved from the OS for its heap), and what the OS reports as the process's resident memory (RSS). The collector lowers the first. The runtime decides about the other two. I allocated 800 MB of live arrays, dropped them, called System.gc() twice and waited:

Collector (heap from 64 MB to 2 GB)Heap in use afterHeap committed afterProcess RSS after
Serial20 MB1,134 MB844 MB
Parallel9 MB1,213 MB1,130 MB
G11 MB64 MB80 MB (it shrinks the heap after a full collection)
ZGC6 MB896 MB922 MB (it uncommits memory only after a delay, 300 seconds by default)

(G1 showed 1,601 MB in use with 800 MB of live data because 1 MB arrays with their header do not fit in one 1 MB region and each takes two: "humongous object" waste, a reminder that heap in use is not the same as data size.) The same effect without a managed heap: CPython allocates small objects in its own arenas and can return an arena to the OS only when every object in it is gone. I created 2,000,000 small objects (485 MB of RSS), then freed all of them (RSS back to 12 MB), but when I kept every twentieth object, scattered through all the arenas, and freed the other 95%, RSS stayed at 485 MB; malloc_trim brought it to 470 MB. Only when the survivors were freed too did it fall to 14 MB. An 800 MB single buffer, by contrast, was returned immediately (814 MB to 14 MB), because large blocks are mapped separately. Go's runtime returns memory to the OS in the background (the scavenger) and debug.FreeOSMemory() forces it; this I did not measure.

Why it matters: the container's limit counts resident memory, not live data. A process that peaked at 1.1 GB and holds it will be OOM-killed in a 1 GB container, or crowd out its neighbours, even though it is "using" 20 MB. What to do:

8. Memory management in other languages

9. Containers and sizing

The heap is not the only memory a process uses: there is also thread stacks, code, metaspace, buffers and native allocations. A container's memory limit (see containers, where a 50 MB limit killed a process with exit status 137) counts all of it. So:

10. Practical guidance

  1. Measure first. Turn on GC logging (-Xlog:gc, GODEBUG=gctrace=1, gc.set_debug) or use the metrics; most programs never need tuning.
  2. Reduce allocation only where a profile shows it matters: reuse buffers in hot loops, avoid building big intermediate collections, pre-size collections. Do not contort ordinary code.
  3. Give the heap room before reaching for exotic flags.
  4. Choose the collector for the goal: throughput (Parallel), balanced default (G1), very low latency or huge heaps (ZGC and similar), tiny footprint (Serial).
  5. Watch tail latency: a stop-the-world pause is felt by every in-flight request at once and looks like a latency spike on every instance at the same time.
  6. Keep objects small and lifetimes short where you can: short-lived is nearly free; medium-lived is the expensive case.
  7. For caches, bound the size.

11. Common mistakes

12. Things to mention in an interview

Quick recap

Appendix: the experiment code

Four files: a shell script that runs the Java workload (gc.sh), the Java program it runs (GcDemo.java, JDK 21), a Go program (gogc.go, Go 1.24) and a Python script (refs.py, Python 3.13). The figures come from single runs on a small two-core virtual machine, so absolute times vary from run to run and small differences between collectors should not be read as rankings; the large effects (heap headroom, cycles, the number of collections falling as GOGC rises) are repeatable. ZGC's pauses are not logged as Pause lines, so for every collector the guide uses the longest gap the allocation loop itself observed, which is what an application would feel. The animation is a model of seven objects, not a measurement.

gc.sh: four Java collectors and three heap sizes

#!/bin/bash
# Java: the same allocation workload under four collectors and three heap sizes (needs a JDK 21; GcDemo.java is the next file).
javac GcDemo.java
run() { name=$1; shift
  java "$@" -Xlog:gc:file=gc.log GcDemo "$LIVE_MB" 8000 $CHURN 2>&1 | sed "s/^/$name: /"
  grep -o 'Pause.* [0-9.]*ms$' gc.log | grep -o '[0-9.]*ms$' | tr -d ms |
    awk -v n="$name" 'NF{c++; s+=$1; if($1>m)m=$1} END{printf "%s: %d pauses, total %.0f ms, longest %.1f ms\n", n, c, s, m}'
  grep -c "Pause Full" gc.log | sed "s/^/$name: full collections: /"
}
# 1. a live set that never changes: young collections are all that is needed
LIVE_MB=200 CHURN=
run "Serial   1g" -XX:+UseSerialGC -Xmx1g -Xms1g
# 2. a live set of 600 MB whose entries are replaced all the time, so old data dies too
LIVE_MB=600 CHURN=churn
for x in 700m 1g 2g; do
  run "Serial   $x" -XX:+UseSerialGC   -Xmx$x -Xms$x
  run "Parallel $x" -XX:+UseParallelGC -Xmx$x -Xms$x
  run "G1       $x" -XX:+UseG1GC       -Xmx$x -Xms$x
  run "ZGC      $x" -XX:+UseZGC        -Xmx$x -Xms$x
done

GcDemo.java: short-lived garbage on top of long-lived data

import java.util.*;

/** Allocates lots of short-lived garbage while holding a long-lived set; records the worst pause seen by the loop. */
public class GcDemo {
    public static void main(String[] args) {
        int liveMb = Integer.parseInt(args[0]);          // size of the long-lived data
        int totalMb = Integer.parseInt(args[1]);         // total garbage to allocate
        boolean churn = args.length > 2;                 // replace long-lived entries now and then, so old data dies too
        Random rnd = new Random(1);
        List<byte[]> live = new ArrayList<>();
        for (int i = 0; i < liveMb * 1024 / 16; i++) live.add(new byte[16 * 1024]);
        byte[][] ring = new byte[4096][];                  // keeps each object alive for a moment so the JIT cannot remove it
        long maxGap = 0, sink = 0, t0 = System.nanoTime(), last = t0;
        for (long allocated = 0; allocated < totalMb * 1024L * 1024; allocated += 1024) {
            byte[] garbage = new byte[1024];                // dies soon: overwritten 4096 allocations later
            ring[(int) ((allocated >> 10) & 4095)] = garbage;
            sink += garbage.length;
            if (churn && (allocated >> 10) % 16 == 0) live.set(rnd.nextInt(live.size()), new byte[16 * 1024]);
            if ((allocated & 0xFFFF) == 0) {               // look at the clock every 64 KB allocated
                long now = System.nanoTime(); maxGap = Math.max(maxGap, now - last); last = now;
            }
        }
        long total = System.nanoTime() - t0;
        System.out.printf("live=%d MB, garbage=%d MB, total %d ms, longest gap between checks %.1f ms (%d)%n",
            liveMb, totalMb, total / 1_000_000, maxGap / 1e6, sink % 2);
    }
}

gogc.go: how Go's GOGC trades memory for CPU

// gogc.go: how Go's GOGC setting trades memory for CPU. A 200 MB live heap, 8 GB of short-lived garbage.
package main

import (
	"fmt"
	"os"
	"runtime"
	"runtime/debug"
	"strconv"
	"time"
)

var sink [][]byte

func main() {
	pct, _ := strconv.Atoi(os.Args[1])
	debug.SetGCPercent(pct)
	live := make([][]byte, 0, 12800)
	for i := 0; i < 12800; i++ {
		live = append(live, make([]byte, 16*1024))
	}
	sink = make([][]byte, 4096)
	start := time.Now()
	var peak uint64
	var m runtime.MemStats
	for i := 0; i < 8*1024*1024; i++ { // 8 million allocations of 1 KB = 8 GB
		sink[i&4095] = make([]byte, 1024)
		if i%100000 == 0 {
			runtime.ReadMemStats(&m)
			if m.HeapAlloc > peak {
				peak = m.HeapAlloc
			}
		}
	}
	runtime.ReadMemStats(&m)
	fmt.Printf("GOGC=%-4d run %5d ms, %4d collections, GC pauses total %5.1f ms, GC CPU fraction %.1f%%, peak heap in use %4d MB (live data %d MB)\n",
		pct, time.Since(start).Milliseconds(), m.NumGC, float64(m.PauseTotalNs)/1e6, m.GCCPUFraction*100, peak>>20, len(live)*16/1024)
	runtime.KeepAlive(live)
}

refs.py: reference counting, a cycle and the cycle collector (Python)

import gc, sys, weakref, time

class Node:
    def __init__(self, name): self.name = name; self.other = None
    def __del__(self): print(f"  freed {self.name}")

print("1. reference counting frees an object the moment the last reference goes")
a = Node("a"); print("  refcount of a:", sys.getrefcount(a) - 1); del a

print("2. a cycle keeps its members alive, although nothing can reach them")
gc.disable()
x, y = Node("x"), Node("y"); x.other, y.other = y, x
w = weakref.ref(x); del x, y
print("  after del: object still alive?", w() is not None)
print("  collector run, unreachable objects found:", gc.collect())
print("  alive afterwards?", w() is not None)

print("3. cost of the cycle detector: 1,000,000 small objects in lists")
gc.enable()
for enabled in (True, False):
    (gc.enable if enabled else gc.disable)()
    t = time.perf_counter(); data = [[i] for i in range(1_000_000)]; dt = time.perf_counter() - t
    print(f"  cyclic collector {'on ' if enabled else 'off'}: building took {dt:.2f} s"); del data
gc.enable()

LatencyDemo.java and latency.sh: percentiles of a request loop under four Java collectors

JDK 21. The loop is open (latency is measured from when each request was due), and the script totals the time the application threads were stopped, from the JVM's safepoint log, after the warm-up.

import java.util.*;
import java.util.concurrent.locks.LockSupport;

/** A request loop at a fixed rate on top of a large live data set. Latency is measured from the time each request SHOULD have started
 *  (open loop), so a pause that stops the program also delays every request that was due during it, as users would experience. */
public class LatencyDemo {
    public static void main(String[] args) {
        int liveMb = Integer.parseInt(args[0]), rate = Integer.parseInt(args[1]), seconds = Integer.parseInt(args[2]);
        Random rnd = new Random(1);
        byte[][] live = new byte[liveMb * 1024 / 16][];
        for (int i = 0; i < live.length; i++) live[i] = new byte[16 * 1024];
        long interval = 1_000_000_000L / rate, warm = 5L * rate;
        int n = rate * seconds; long[] lat = new long[n]; long sink = 0;
        byte[][] ring = new byte[64][];
        long start = System.nanoTime();
        long measureFromMs = java.lang.management.ManagementFactory.getRuntimeMXBean().getUptime() + 5000;      // pauses before this time (start-up, warm-up) are not counted
        System.out.println("MEASURE_FROM_MS=" + measureFromMs);
        for (int i = 0; i < n; i++) {
            long due = start + i * interval;
            long now = System.nanoTime();
            while (now < due) { LockSupport.parkNanos(Math.min(due - now, 200_000)); now = System.nanoTime(); }
            for (int k = 0; k < 100; k++) { byte[] b = new byte[4096]; b[k] = (byte) i; ring[(i + k) & 63] = b; sink += b[k]; }   // about 400 KB of short-lived garbage
            for (int k = 0; k < 4; k++) live[rnd.nextInt(live.length)] = new byte[16 * 1024];                                       // long-lived entries replaced: old data dies too
            for (int k = 0; k < 50; k++) sink += live[rnd.nextInt(live.length)][k];                                                 // reads of the live set
            lat[i] = System.nanoTime() - due;
        }
        long[] s = Arrays.copyOfRange(lat, (int) warm, n); Arrays.sort(s);
        java.util.function.DoubleUnaryOperator q = p -> s[(int) Math.min(s.length - 1, Math.floor(s.length * p))] / 1e6;
        long over10 = Arrays.stream(s).filter(x -> x > 10_000_000L).count();
        System.out.printf("latency ms: p50 %.2f  p90 %.2f  p99 %.2f  p99.9 %.2f  p99.99 %.2f  max %.1f | requests over 10 ms: %d of %d (%d)%n",
            q.applyAsDouble(.5), q.applyAsDouble(.9), q.applyAsDouble(.99), q.applyAsDouble(.999), q.applyAsDouble(.9999), s[s.length - 1] / 1e6, over10, s.length, sink % 2);
    }
}
#!/bin/bash
# The same request loop under four Java collectors: latency percentiles, and the time the application was actually stopped. Needs JDK 21.
export JAVA_TOOL_OPTIONS=
javac LatencyDemo.java
for gc in Serial Parallel G1 Z; do
  java -XX:+Use${gc}GC -Xmx2g -Xms2g -Xlog:safepoint:file=sp_$gc.log:uptime LatencyDemo 600 1000 40 > out_$gc.txt
  from=$(grep -o 'MEASURE_FROM_MS=[0-9]*' out_$gc.txt | cut -d= -f2)
  printf '%-9s %s\n' $gc "$(grep latency out_$gc.txt)"
  # every safepoint after the warm-up: the time the application threads were stopped (in ms)
  sed -n 's/^\[\([0-9.]*\)s\].*Safepoint "\([A-Za-z0-9_]*\)".*Total: \([0-9]*\) ns.*/\1 \2 \3/p' sp_$gc.log |
    awk -v f=$from -v n=$gc 'BEGIN{FS=" "} $1*1000>=f {c++; t+=$3/1e6; if($3/1e6>m)m=$3/1e6; if($2 ~ /Collect|GC|Z|Cleanup|Pause/) g++} END{printf "%-9s application stopped %d times after warm-up, %.0f ms in total, longest %.1f ms\n", n, c, t, m}'
done

latency.go: the same loop in Go

Go 1.24. Run with GOGC=100 ./latency_go 600 1000 40 (600 MB of live data, 1,000 requests per second, 40 seconds).

package main

// The same open-loop request loop as LatencyDemo.java, in Go: latency is measured from the time each request should have started.
import (
	"fmt"
	"math/rand"
	"os"
	"runtime"
	"sort"
	"strconv"
	"time"
)

func main() {
	liveMB, _ := strconv.Atoi(os.Args[1])
	rate, _ := strconv.Atoi(os.Args[2])
	seconds, _ := strconv.Atoi(os.Args[3])
	rnd := rand.New(rand.NewSource(1))
	live := make([][]byte, liveMB*1024/16)
	for i := range live {
		live[i] = make([]byte, 16*1024)
	}
	interval := time.Second / time.Duration(rate)
	n := rate * seconds
	lat := make([]time.Duration, n)
	ring := make([][]byte, 64)
	var sink int
	var ms0 runtime.MemStats
	start := time.Now()
	for i := 0; i < n; i++ {
		due := start.Add(time.Duration(i) * interval)
		if d := time.Until(due); d > 200*time.Microsecond {
			time.Sleep(d - 100*time.Microsecond)
		}
		for time.Now().Before(due) {
		}
		if i == 5*rate {
			runtime.ReadMemStats(&ms0)
		}
		for k := 0; k < 100; k++ {
			b := make([]byte, 4096)
			b[k] = byte(i)
			ring[(i+k)&63] = b
			sink += int(b[k])
		}
		for k := 0; k < 4; k++ {
			live[rnd.Intn(len(live))] = make([]byte, 16*1024)
		}
		for k := 0; k < 50; k++ {
			sink += int(live[rnd.Intn(len(live))][k])
		}
		lat[i] = time.Since(due)
	}
	var ms runtime.MemStats
	runtime.ReadMemStats(&ms)
	s := append([]time.Duration(nil), lat[5*rate:]...)
	sort.Slice(s, func(a, b int) bool { return s[a] < s[b] })
	q := func(p float64) float64 { return float64(s[int(float64(len(s))*p)].Microseconds()) / 1000 }
	over := 0
	for _, x := range s {
		if x > 10*time.Millisecond {
			over++
		}
	}
	fmt.Printf("GOGC=%s: latency ms: p50 %.2f  p90 %.2f  p99 %.2f  p99.9 %.2f  p99.99 %.2f  max %.1f | over 10 ms: %d of %d | GCs %d, GC CPU %.1f%%, peak heap %d MB (%d)\n",
		os.Getenv("GOGC"), q(.5), q(.9), q(.99), q(.999), q(.9999), float64(s[len(s)-1].Microseconds())/1000, over, len(s),
		ms.NumGC-ms0.NumGC, 100*ms.GCCPUFraction, ms.HeapSys>>20, sink%2)
}

Surprises.java: direct buffers, finalizers, references and RSS after collection

JDK 21. Run, for example, java -Xmx256m -XX:MaxDirectMemorySize=128m -XX:+DisableExplicitGC Surprises direct; the other modes are final, refs and rss (try it with -XX:+UseSerialGC, -XX:+UseG1GC and -XX:+UseZGC).

import java.lang.ref.*;
import java.nio.ByteBuffer;
import java.nio.file.*;
import java.util.*;

/** What the collector does not see, or does late: native (direct) buffers, finalizers, weak and soft references, and memory the process keeps after collection. */
public class Surprises {
    static long rssMb() throws Exception {
        for (String l : Files.readAllLines(Path.of("/proc/self/status"))) if (l.startsWith("VmRSS:")) return Long.parseLong(l.replaceAll("[^0-9]", "")) / 1024;
        return -1;
    }
    static long heapUsedMb() { Runtime r = Runtime.getRuntime(); return (r.totalMemory() - r.freeMemory()) >> 20; }
    static long heapCommittedMb() { return Runtime.getRuntime().totalMemory() >> 20; }

    @SuppressWarnings({"removal", "deprecation"})
    static class WithFinalizer { final byte[] data = new byte[1024]; static int ran; @Override protected void finalize() { ran++; } }
    static class Plain { final byte[] data = new byte[1024]; }

    public static void main(String[] a) throws Exception {
        switch (a[0]) {
            case "direct" -> {                       // 4,000 direct buffers of 1 MB, each dropped at once; the Java heap is almost untouched
                long peak = 0;
                try {
                    for (int i = 0; i < 4000; i++) { ByteBuffer.allocateDirect(1 << 20).put(0, (byte) 1); peak = Math.max(peak, rssMb()); }
                    System.out.printf("4,000 direct buffers of 1 MB: completed. Heap in use %d MB, process RSS peaked at %d MB%n", heapUsedMb(), peak);
                } catch (OutOfMemoryError e) { System.out.printf("OutOfMemoryError: %s  (heap in use only %d MB, RSS %d MB)%n", e.getMessage(), heapUsedMb(), rssMb()); }
            }
            case "final" -> {
                for (boolean fin : new boolean[]{false, true}) {
                    WithFinalizer.ran = 0; Object[] keep = new Object[1];
                    for (int i = 0; i < 100_000; i++) keep[0] = fin ? new WithFinalizer() : new Plain();          // 100,000 objects of about 1 KB, all unreachable at once
                    long before = heapUsedMb(); System.gc(); long after1 = heapUsedMb(); int ran1 = WithFinalizer.ran;
                    Thread.sleep(300); System.gc(); long after2 = heapUsedMb();
                    System.out.printf("%-26s heap in use: %3d MB before gc, %3d MB after the first gc (finalizers run: %6d), %3d MB after the second (finalizers run: %6d)%n",
                        fin ? "objects with finalize()" : "plain objects", before, after1, ran1, after2, WithFinalizer.ran);
                }
            }
            case "refs" -> {
                Object strong = new byte[1 << 20]; WeakReference<Object> weak = new WeakReference<>(new byte[1 << 20]); SoftReference<Object> soft = new SoftReference<>(new byte[1 << 20]);
                WeakHashMap<Object, String> whm = new WeakHashMap<>(); Object key = new Object(); whm.put(key, "kept"); whm.put(new Object(), "dropped");
                System.gc(); Thread.sleep(100);
                System.out.printf("after one gc: weak reference cleared=%b, soft reference cleared=%b, WeakHashMap size %d (one key still held strongly)%n", weak.get() == null, soft.get() == null, whm.size());
                List<byte[]> pressure = new ArrayList<>();
                try { while (soft.get() != null) pressure.add(new byte[8 << 20]); } catch (OutOfMemoryError e) { }
                int n = pressure.size(); pressure.clear();
                System.out.printf("under memory pressure (%d MB allocated until the heap was full): soft reference cleared=%b -- it is cleared before an OutOfMemoryError, but only then%n", n * 8, soft.get() == null);
                Reference.reachabilityFence(strong); Reference.reachabilityFence(key);
            }
            case "rss" -> {
                System.out.printf("start:                          RSS %4d MB, heap in use %4d MB, heap committed %4d MB%n", rssMb(), heapUsedMb(), heapCommittedMb());
                List<byte[]> big = new ArrayList<>(); for (int i = 0; i < 800; i++) { byte[] b = new byte[1 << 20]; Arrays.fill(b, (byte) 1); big.add(b); }
                System.out.printf("800 MB of live data:            RSS %4d MB, heap in use %4d MB, heap committed %4d MB%n", rssMb(), heapUsedMb(), heapCommittedMb());
                big = null; System.gc(); Thread.sleep(500); System.gc(); Thread.sleep(500);
                System.out.printf("all of it dropped, after gc:    RSS %4d MB, heap in use %4d MB, heap committed %4d MB%n", rssMb(), heapUsedMb(), heapCommittedMb());
            }
        }
    }
}

rss_py.py: memory that Python frees but keeps

Python 3.13 on Linux (it reads /proc/self/status).

"""Why the memory of a Python process stays high after objects are freed: small objects live in arenas that can only be returned when completely empty."""
import ctypes, gc, os

def rss_mb():
    for line in open("/proc/self/status"):
        if line.startswith("VmRSS:"): return int(line.split()[1]) // 1024

class Rec:                                                  # a small object with a few fields, like a row or a parsed message
    __slots__ = ("a", "b", "c")
    def __init__(self, i): self.a = i; self.b = str(i); self.c = (i, i + 1)

N = 2_000_000
print(f"baseline RSS {rss_mb()} MB")
objs = [Rec(i) for i in range(N)]
print(f"{N:,} small objects alive:           RSS {rss_mb()} MB")
objs = None; gc.collect()
print(f"all freed (refcount + gc.collect):    RSS {rss_mb()} MB   <- returned: the arenas became completely empty")

objs = [Rec(i) for i in range(N)]
keep = objs[::20]                                            # 5% survive, scattered through every arena
objs = None; gc.collect()
print(f"95% freed, 5% kept (scattered):       RSS {rss_mb()} MB   <- the memory is free inside Python but not returned to the OS")
try: ctypes.CDLL("libc.so.6").malloc_trim(0); print(f"after malloc_trim(0):                 RSS {rss_mb()} MB   <- helps very little: these small objects live in Python's own arenas, not in malloc blocks")
except OSError: pass
keep = None; gc.collect()
print(f"survivors freed too:                  RSS {rss_mb()} MB")

big = bytearray(800 * 1024 * 1024)
for i in range(0, len(big), 4096): big[i] = 1
print(f"one 800 MB buffer:                    RSS {rss_mb()} MB")
big = None
print(f"buffer freed:                         RSS {rss_mb()} MB   <- large blocks are mapped separately and returned at once")

Sources

Practise this topic: CS Fundamentals

More guides