sync.Pool survives one GC, not two

sync.Pool gets described as a cache often enough that the description sticks, and then someone is surprised when their pool is empty. It is not a cache. It is a set of objects the garbage collector is explicitly allowed to take back, and the rule for when it takes them is both simple and rarely stated precisely.

Here it is, measured.

All numbers: go1.27.0 darwin/arm64, Apple M4 Pro, 12 cores, macOS 26.6.2. Benchmarks with -benchmem -count=10, summarised by benchstat.

The eviction rule

Counting evictions directly is easier than inferring them. Give the pool a New that increments a counter, put a known number of objects in, then take the same number out. Every New call is an object the pool could not supply.

var misses int

func newPool() *sync.Pool {
	misses = 0
	return &sync.Pool{New: func() any { misses++; return new(buf) }}
}

func drain(p *sync.Pool) int {
	misses = 0
	for i := 0; i < objects; i++ {
		p.Get()
	}
	return misses
}

sync.Pool is per-P, so the test pins itself to a single P with runtime.GOMAXPROCS(1). Otherwise the answer depends on where the goroutine happened to be running, which is a different article.

$ go run ./pool
no GC:        0/100 Gets missed
after 1 GC:   0/100 Gets missed
after 2 GC: 100/100 Gets missed

One collection: nothing lost. Two: everything lost. Not a proportion, not a heuristic — a cliff.

The victim cache

sync.Pool registers a cleanup that runs at the start of every collection, before the world restarts. It does two things in this order: it discards whatever is currently in the victim cache, then it moves the live pool contents into it.

So an object that was in the pool when a collection began is not freed. It is demoted. Get still finds it — the lookup falls through to the victim cache when the primary is empty — and a Get that hits the victim promotes the object back to the primary pool.

The next collection then discards that victim. An object survives exactly one collection, and a pool that receives no Put between two collections is empty after the second.

The victim cache was added in Go 1.13 for exactly the pathology the naive design produced: a service with a two-millisecond GC period had its pools flattened every two milliseconds, so the pool spent most of its life cold and the allocation it was meant to avoid happened anyway. One cycle of grace turns a cliff into a ramp.

What it costs when the GC is actually running

The eviction rule is only interesting if it shows up in throughput. Three benchmarks: allocating a 4 KiB buffer every iteration; the same work through a pool; and the same pool with a background goroutine calling runtime.GC() every millisecond underneath it.

The buffer has to genuinely outlive the call, or the compiler stack-allocates the unpooled case and the comparison measures nothing:

var keep *buf

//go:noinline
func use(x *buf) {
	x.b[0]++
	keep = x
}

               │   sec/op    │    B/op     │ allocs/op │    %miss    │
NoPool-12        367.1n ± 1%   4.000Ki ± 0%  1.000 ± 0%           —
Pool-12          7.202n ± 0%     0.000 ± 0%  0.000 ± 0%   600.0n ± 0%
PoolUnderGC-12   8.236n ± 1%     0.000 ± 0%  0.000 ± 0%   376.9µ ± 6%

The pool is 51 times faster than allocating. A collection every millisecond raises the miss rate by a factor of 628 — from one miss in 167 million operations to one in 265,000 — and costs 14% of throughput. Still 44 times faster than not pooling.

The arithmetic is the mechanism

That miss rate is worth a second look, because it is not a vague “GC hurts a bit” number. It is the eviction rule.

At 8.2ns per operation the loop runs roughly 122,000 iterations per millisecond, so one GC interval is about 122,000 operations. The measured miss rate is one per 265,000 — which is one miss per two GC cycles.

That is exactly what the victim cache predicts: the single buffer this loop cycles survives the first collection in the victim, and is lost on the second. The benchmark and the runtime source agree to within rounding, which is the strongest form of evidence a measurement like this can offer.

When this actually hurts

The cost of an eviction is one New call per object the pool was holding. Three things make that expensive:

An expensive New. A 4 KiB make costs 367ns. A New that compiles a template, dials a connection or builds a 1 MB scratch buffer costs orders of magnitude more, and it now runs every two GC cycles per P.

Low throughput. The benchmark above hides eviction because it performs 122,000 operations between collections. A service handling 200 requests per second with a GC every 100ms performs about 20 operations per cycle — the pool is cold as often as it is warm, and it is close to pure overhead.

Many Ps. Pools are per-P. Twelve Ps means twelve pools to refill after every second collection, not one.

The corollary is the practical rule: sync.Pool pays when objects are cheap to create and created constantly. When they are expensive to create and used rarely, you want a free list you own — and then bounding it is your problem, which is the responsibility sync.Pool was taking off you.

What this does not measure

runtime.GC() every millisecond is a stand-in for GC pressure, not a model of it. A real collector runs concurrently, is paced by GOGC and GOMEMLIMIT, and its cycles are not evenly spaced. What this experiment reproduces faithfully is the eviction schedule; what it does not reproduce is the write barrier, the assist cost, or the effect of a pool full of pointer-bearing objects on mark time.

The single-P eviction test is a deliberate simplification too. On a real multi-P workload, Get can steal from another P’s shared queue before falling through to New, so the miss rate is lower than the per-P arithmetic suggests and depends on how work is distributed.

And none of this says whether pooling is worth it in your service. It says what the pool does and what it costs when it fails. The allocation you avoid may have been free anyway — see escape analysis for how often that turns out to be true.