# Reducing allocation churn in an agent platform

Author: Chris Chabot
Canonical URL: https://chabot.dev/blog/reducing-allocation-churn-agent-platform/
Published: 2026-11-03

Why an agent gateway with broad retained state allocated gigabytes a minute, how repeated sizing and serialization drove it, and what bounded buffers and encode-once paths do and don't fix.

Our agent gateway is a single Bun process that routes event streams between clients and agents, commits project state, supervises idle work and keeps a local database of recent activity. All of that shares one event loop, so slow work in any of those jobs delays every stream. On 31 October 2026, on the development gateway that carried our own agent work, the loop's median lag read 920 ms and its p99 2.7 seconds, resident memory swung between 2.3 and 4.4 GB within half an hour, and the process was allocating about 3.9 GB of large objects a minute while holding about 1.2 to 1.5 GB of heap.

Numbers like those read like a leak. They turned out to be something more tedious, which was the gateway doing the same work over and over on data that hadn't changed: walking it, serializing it, copying it and writing it out again, once per event.

That day I set down a direction for the gateway and a charter for every change to the platform. The gateway should be a router that decodes and encodes only what routing needs, keeps in-memory history only in bounded ring buffers and stores little or nothing that another system already holds, with targets of under 200 ms p99 loop lag and under 1.5 GB resident an hour after a reload. The charter opens by saying that events must be able to flow through this system forever while it stays fast, low-latency and calm, and it comes down to writing rarely and in batches, allocating once, encoding once, giving everything a bound or a retention rule, and measuring before and after. Agents working under my direction took the measurements and made the cuts, one small change at a time, each behind a switch that defaulted to on, with a test that counted calls, walks or retained entries with the switch in either position, and each released on its own at least 15 minutes after the previous reload.

## What it held and what passed through

A process's memory answers two questions that are easy to run together: what is it holding, and what has it pushed through in the last minute? Measuring either on a live gateway can't be allowed to hurt it, so nothing signaled the process and nobody took a full heap snapshot. A continuous profiler wrote a CPU profile per window, touching a trigger file produced a 60-second sample of large allocations by site or a summary of estimated retained size per registered holder, and a health endpoint reported resident memory and loop lag.

What it held was broad rather than large in any one place. Twenty-five minutes after a start, the heap was about 1.2 GB across 11.2 million objects, and the top registered holders were 82 MB of resident turns, 19 MB of projection items and 16 MB of source references. A later pass over the process's memory, printing sizes and key names and never values, found no single culprit: about 1 GB of a 1.5 GB heap was external string text, runs over 64 KB came to 546 MB, and the rest was widely spread IDs, words and prose. That floor is real and worth reducing, but it is project and turn state, not a leak.

What passed through was another matter. In the first 60-second sample the largest site, a wake-up check run from the idle-work supervision pass, took 498 MB, about an eighth of the total, and behind it came a `JSON.parse` in the agent-grants code at 435 MB and an `Array.join` in the continuation code at 212 MB, with no single site in charge. Snapshot polling came to 28 MB a minute, which ruled it out as the main source. Meanwhile mimalloc reported 19 GB committed against 2.5 GB resident. Our reading was that resident memory was following the allocation rate and not the retained set, and that's as far as those figures go, because committed bytes, JavaScript heap, external strings and resident pages are different quantities and the gap between them doesn't by itself prove fragmentation or a leak.

## Measuring what can't have changed

The CPU profile made the pattern concrete. The largest item, at 26.7% of samples, was the function that sizes a record as JSON, and its callers ran through the whole commit path: the content-addressed record store after each commit, the installation of a committed record as the live one, the pressure check that sizes every section of a record above 80% of its limit, the step that moves work evidence out of a record, and the engine's trim. Records in the gateway are immutable, and each new version shares its unchanged, deep-frozen sections with the version before it, which is the economy the whole design depends on. The sizing code didn't take advantage of it. Only the root's size was remembered, so each measurement walked again through every frozen section the record shared with its predecessor. A section that could not have changed was being measured again on every commit, by several callers.

Once frozen sections stopped being walked again, the walk went from 18.8% of samples to one sample in about 31,700, and the sizing function as a whole from 20.4% to 6.2%. Those figures come from windows about three minutes after a reload in two different processes, so they describe that hotspot and not the gateway's latency.

The largest allocation site had a similar shape. That wake-up check found a worker's latest completed attempt by running `findLast` over the coordinator's entries, through a draft proxy, once per completed attempt, and two supervision passes called it. It dropped out of the profile with the cuts in, though I put less weight on that: five minutes after another reload it hadn't reached the top 25 either, because the completed attempts were still loading.

Those percentages need a caution, because the first aggregation got them wrong. It weighted each sample by its time delta and treated the main thread as busy for the whole window, but this profiler records samples only while JavaScript runs and has no idle node, so the gaps between samples include idle time. Counted by sample, the shares rank where the busy time goes. They don't say how busy the thread was, and they aren't percentages of elapsed time.

The new top of the list showed the same pattern one level down. The record store, having let go of a pack's bytes, read records from the pack again and verified each one's content digest. A storage meter ran regex replacements once for each record read it counted. The allocation sample from that process listed about 10 GB in 64 seconds across its largest sites, well above the 3.9 GB a minute we began with, led by the text built for those digest checks. The old leaders were gone or reduced, and some of the new ones were boot work that falls away within minutes, so that sample doesn't show allocation getting worse. It doesn't show it getting better either. "After" was a different process doing a different job.

## One redraw, serialized three times

In early November a sample under different load read 3.55 GB a minute, and it contained the cleanest example I have of the problem. A worker's message held 44 terminal blocks of 1.6 to 2 MB each, and it was still growing. Terminal frames were published from a timer, which put them outside any callback from the session service, and under the rule at the time that meant a synchronous commit of the whole item for every frame, 3.3 times a second. In that sample this one item took 378 MB of `JSON.stringify` for its commit, another 377 MB for the journal and most of 333 MB of SQLite binds. The same content was being serialized for each destination in turn, for a redraw of a screen that would be replaced a third of a second later, and the bill rose as the message grew.

The fix recognizes that a redraw of a block the item already holds isn't a change of state worth committing. It now travels the path that streaming tokens take. The first frame and the block's completion still commit, and a commit hands the journal the text it has just written, so the journal no longer serializes it again, and a switch restores the old rule if that ever proves wrong. Ninety related tests passed and each of nine deliberate mutations of the fixes turned one red, which matters in a change whose whole purpose is to stop doing something. A similar case followed: one 7.6 MB part was being serialized 19 times a minute, because a commit freezes a section it changed as a new object and something needed to find the tokens inside it. Those tokens are now found by walking the frozen values with no text built at all, a change held against the JSON behavior on adversarial strings and 2,000 generated trees.

## What a bound does and doesn't buy

Ring buffers were in my direction for the gateway and in the charter, so it's worth being exact about what one gives you. The simplest version in the platform is a few dozen lines: one backing array allocated at construction, a head index and a count, and a `push` that writes a slot and advances the head, overwriting the oldest entry once the ring is full. Its own comment states a modest and useful claim, that recording an event or a log line during memory pressure never grows the heap beyond the item itself. The last four words matter. A slot holds a reference, so the ring fixes the number of items and not the bytes behind them, and reading it allocates, because `toArray` and `newest` build new arrays. My charter says rings give "fixed memory, no growth, no fragmentation, kind to kernel paging". That's what I want from them, and a JavaScript array of references to variable payloads doesn't establish it. All of this is userspace work, changes to what a TypeScript program asks its runtime to do, and none of it replaces the allocator or reaches into the kernel.

The ring that lets a peer catch up on a stream shows what else comes with a bound. It adds an age limit to the capacity, evicts the oldest frame when either is reached and remembers the highest sequence number it has evicted. A reader asks for everything after a sequence number and gets back what remains, along with an explicit gap if some of what it asked for has gone. That is the bargain with any bounded history: growth is replaced by loss, so the loss has to be reported. The same ring clones each frame on the way in and again on the way out, which bounds what it retains and does nothing about what it copies.

A bound can also be set too low. A transcript cache capped at 1 GiB was refilled 464 times in 58 minutes, about 683,000 items with 399 evictions, and SQLite inserts and decoding came to about 15% of the CPU profile. The cache is a disposable file with its page cache held at 32 MB, so raising the cap to 4 GiB cost only disk, and the first reading afterward showed no evictions. It didn't answer why agents were reading so many old turns, which is the better question.

So rings are one implementation choice, and a narrow one. They limit what stays in memory. The larger part of the charter is about the flow: encode an event once and share the bytes with every consumer, never parse and re-serialize unchanged data, combine and deduplicate before writing, and write at most every few seconds instead of once per event or per turn. A few seconds of delay at the storage boundary is a deliberate give, and so is a little more steady memory. Local-state SQLite shows why that's the rule to aim for and also how far there was to go, since it grew from 3% to 15% of samples over the first ten minutes after a reload, and under the stream fan-out one path still committed synchronously while an event was being broadcast.

## Reading the result

The first reading after the terminal release showed large allocations at 1.2 GB a minute against 3.55, no evictions against about seven a minute, resident memory between 2.3 and 3.0 GB, and a boot peak of 6,025 MB against 9,051. I like those numbers, and they aren't a controlled comparison, because the workers running at the time were doing different things and neither a whole hour nor a terminal-heavy worker had been read. Later that morning, with the coordinator working and not idle, the gateway's memory objective flipped on and off around its 3 GB line, a long way above the 1.5 GB target.

The sampler can mislead on its own account as well. It charged 1.4 GB a minute to upload sites, all of it "gap" charges that the sampler itself calls a lead and not a measurement. The two `Uint8Array.from` copies per upload that look guilty in a code review took 2.2 ms for 5.3 MB, at 5 to 11 MB of uploads a minute, so they were left alone and the true source of the charges stayed unread.

That leaves collection itself. By then the SQLite stalls had stopped appearing, and what remained was the forced collections, 2 of 246 in one day taking 9.5 and 14.7 seconds against a median of 0.23. One fell during a release gate on the same host and the other while the gateway's own memory was being scanned, which would be an embarrassing answer if it turned out to be the answer. Any collection over a second now logs its CPU and major page faults, which will separate swap from starvation from real work. For now I can tell you what those two pauses coincided with, and not what caused them.