Keeping a shared event loop fast across sixty agents

Why a shared event loop is only as fast as its worst synchronous stretch, and how encode-once fan-out, closing back-pressured sockets and group commit kept one gateway responsive for sixty agents.

October 20264 min readPerformanceRuntimeAgent systems

Dense small dabs of dusty lilac and graphite at left grow larger and sparser toward nearly clear warm chalk at right, with pale yellow.
On this page
  1. Measuring stalls before changing code
  2. Coalescing and encode-once fan-out
  3. Back pressure and byte-bounded replay
  4. Durability off the loop and reads at commit time
  5. Peak load and the performance charter

Running a coding-agent gateway in a single process is a sound choice. One copy of state avoids the consistency overhead you invite the moment you add a second node. That holds up until sixty agents stream tokens, tool output, and durable writes through the same event loop, where throughput and tail latency are set by the worst synchronous stretch. A 14 MB JSON.stringify, a 63 ms fsync, or a history scan inside a broadcast stalls every active agent at once.

Adding gateway nodes or layering on caches either abandons the single-writer model or inflates heap in a process that already has plenty of it. Keeping one gateway responsive means treating it strictly as a router whose per-event work stays small and fixed: move anything proportional to history off the loop or to commit time, bound retained state by bytes, and honor back pressure instead of absorbing it.

I applied this to the gateway for our coding-agent platform, a single-process TypeScript service running on Bun, a JavaScript runtime, sharing a server with the rest of the stack. Agents run in separate runtime processes; the gateway relays model streams, terminal traffic, and browser tab updates, orchestrates each agent loop, and persists durable state to SQLite.

Measuring stalls before changing code

Optimizing a shared event loop by intuition mostly polishes paths that are not the bottleneck. Dozens of synchronous stalls were spread across subsystems, invisible without a profile. Before changing hot paths, I built a measurement harness and required every change to land with before-and-after numbers: 5 ms CPU profile windows so stopping the profiler did not distort the loop, an event-loop watchdog naming whatever ran during lag spikes, host tracing with perf and bpftrace, and a relay benchmark driving 40 agent streams across 80 WebSocket tabs while checking that every character arrived once and in order.

Coalescing and encode-once fan-out

Model output arrives as tiny token deltas, and relaying each individually multiplies fixed work across state updates, replay entries, persistence marks, and frames for every subscribed tab. The gateway coalesces consecutive text deltas per stream into 50 ms windows without reordering; any non-delta event flushes a held delta first. Downstream, subscriptions are indexed by stream so broadcasts skip unrelated sockets, and each outbound event is serialized once, with per-tab frames spliced from those bytes rather than re-encoded per client. On the 40-stream, 80-tab benchmark, coalescing cut traffic from roughly 120,000 frames and 58 MB to 34,000 frames and 19.6 MB.

Back pressure and byte-bounded replay

Bun’s WebSocket send returns 0 when a socket exceeds its back-pressure limit and drops the frame. Ignoring that return value previously left slow tabs silently missing events mid-stream, and absorbing the overflow in a gateway buffer is worse, since one stalled tab then grows memory for the entire process.

Instead, when send returns 0, the gateway closes the socket with code 1013 so the client reconnects and replays from its last sequence number. Relying on replay exposed a second flaw: histories capped by entry count alone still grew unchecked on large terminal output or tool payloads. Bounding them at 2 MiB of serialized bytes per stream gives each stream’s history a hard memory ceiling regardless of payload size.

Durability off the loop and reads at commit time

Those 63 ms fsync stalls came from safety reservations written to SQLite before dispatching a paid model request, so spend and execution state survive a crash. Moving fsync off the event loop and grouping flushes on the SQLite WAL preserves that guarantee without blocking other agents: a reservation commits without its own fsync, and the request dispatches only after the background group fsync completes, so nothing goes out before the record hits disk. On the benchmark, twenty concurrent reservations cost two fsync calls and p99 loop lag fell from 13.1 ms to 1.2 ms.

Read paths failed the same way whenever they reconstructed state on demand. Waking a project’s coordinator agent previously paged backward through thousands of events and fetched its memory file from remote storage, hitting a 4.6-second live median on the hot path. Because every message already passes through the gateway on commit, maintaining a per-project message ring at commit time alongside a version-keyed in-memory copy of that file removes both the history scan and the remote read. Wake preparation dropped to 1 to 3 ms on the benchmark.

Peak load and the performance charter

By October 2026, the gateway carried 62 concurrent agents at peak, roughly the traffic of that many Claude Code sessions, moving 40 million context tokens a minute on the shared server. When project load grew further, event-loop lag and resident memory climbed again under allocation churn and larger project state.

To govern later changes, I wrote a performance charter that opens: “Events must be able to flow through this system forever while it stays fast, low-latency and calm.” In practice, that reduces to four rules: keep recent state in bounded ring buffers, allocate and encode once, write rarely in batches, and measure before and after.

Chris Chabot, October 2026