Keeping long agent turns steerable and within their context budget

An agent’s turn is a loop: the model asks for something, the host does it, and the result goes into the next request. The boundary between one step and the next is the only place where the host can act on what has changed. That is where it reads a message from the person, reports a job that has finished, checks how large the next request will be and decides whether older context needs folding into a summary. Anything that stops a step from ending delays all of those decisions, and a boundary that misreads the state makes them badly.

In late October we reviewed about 36 hours of our coding agent’s messages and platform logs, some 43,000 messages across 24 threads and their sub-agents. The model itself was in good shape. Prompt caching hit about 99% of input tokens, and a steer, which is a message a person sends to an agent that is already working, was followed within seconds once it reached the model. What failed was the machinery around the model, in two ways that looked unrelated.

The first was steering. A steer could not reach a turn blocked in a foreground shell command, build or test run for up to 40 minutes. One that arrived while the model was streaming had the opposite problem: it took effect within 250 ms by throwing that step away, which happened 57 times and discarded about seven minutes of model time. Agents also had no general way to wait, so they improvised, with pgrep loops that matched their own command lines, waits on jobs that had already died and long parks behind slow reviewers. Long commands came back marked “unconfirmed”, with advice not to check.

The second was context. Only 10 of 19,666 model calls failed hard, and all ten were the same defect: an overflow at the 1M-token window, sent five times as if it might go away and followed by an emergency fold with no cache to read from. Threads were carrying 400,000 tokens to 1M for hours, and at a median of 357,000 tokens per call, cache reads were 71% of spend despite that excellent hit rate. Part of the cause was a rule written to be considerate. Threshold compaction gave way for the whole of a person-initiated turn, so that nobody would wait behind a summary, and each steer re-armed the wait. Steered threads ran from 800,000 tokens to past the limit, which left the person the rule was protecting waiting behind five doomed requests and a cold fold.

Those five requests had a wonderfully small cause of their own. The provider’s error reads “prompt is too long: N tokens > 1000000 maximum”, our inference service is written in Go, and Go’s JSON encoder escapes > as \u003e. The guard looking for that message never matched it, so the error went down the generic path, which halved max_tokens and tried again.

These look like separate problems, but in both cases the next boundary could not react to the actual state, either because it never arrived or because it was judging something other than the request about to go out. The redesign makes each agent’s step boundary a choice among events: finished operations, steers, messages, timers and context-budget events. That only works if boundaries keep arriving and each one can see what is true.

A yield is not a result

Keeping boundaries coming means that every tool has to give control back within a bound, which we set at about 45 seconds. Past it, a shell command becomes a managed job that returns its handle and its output so far, other tools become in-process jobs, and the step ends. The model can then read a steer and answer it while the command keeps running. With a boundary never far away, a steer no longer needs to abort a streaming response, so the 250 ms preemption could go.

Agents built this under my direction, one package at a time, and a separate Codex review then went through each package, with a fixer resolving what it found. The review of this one is worth dwelling on, because deciding what a yield means is harder than making it happen. As first delivered, a yielded call was booked as finished at the moment it yielded, and the calls issued after it were released. If a step issued a build and then a test of that build, the test would run against a build still in progress. A yield is a state transition, and what it hands back should say only that the work has moved. In the corrected version, later dependent calls do not run, and the yielded call settles exactly once, with its real result, when that result exists.

The review took apart several other confident claims. The bound applied per call rather than per step, so a step with several slow calls could hold a turn for several bounds, and shell calls had 30 seconds of grace on top of it. The promise that a run never ends while a detached call is running was kept by a park that had no bound, ignored notices, held the worker slot and could leave an outcome unread behind the final answer. The new wait primitive reported unknown outcomes as finished and let any agent read or consume another agent’s detached call. Those defects were fixed. The review also checked the builder’s wider claim that 22 issues were resolved: nine were fully resolved and the other thirteen only partly.

What came out the other side is a set of explicit waiting states in place of the improvisation. One await covers jobs, sub-agents, CI checks, replies, file lease releases and timers, and it returns early when a message arrives, at a periodic check-in, or when a job looks dead because its process has gone or it has produced no output for 15 minutes. A command whose outcome is not yet known carries an obligation naming its handle, how it will settle, how the result will be delivered and how to check that it is alive, which replaced the old advice not to check.

Judging the request that will actually be sent

Staying inside the window is the same kind of problem one layer up. Before every model request, the host now projects its size, using the last size the provider reported plus whatever has been added since, and judges that number against two lines. Above the trigger, a fold is due but optional, and a person waiting for an answer may postpone it for up to two step boundaries, so a question is not held behind a summary. At the hard ceiling the fold is mandatory: nobody can postpone it, and a steer that arrives while it runs waits for the next step instead of aborting it. The old rule had neither a limit on postponement nor a line it could not cross. The default trigger stayed at 80% of the window, and I rejected a lower ceiling, because workers keep large contexts on purpose.

Where the ceiling sits took more care than you would expect. The provider rejects any request whose input and max_tokens together exceed its window, so the ceiling has to leave room for the output the request actually asks for. The first version reserved the model’s nominal 64,000 output tokens and arrived at a ceiling of 900,000, while the request was sending 128,000, and 900,000 plus 128,000 does not fit in a million. The corrected ceiling on that window is 855,616: the lower of 90% of the window and the window less the real output budget and a reserve.

The review’s broader finding was that the budget had not been judged on the request actually sent. Fresh sessions, the new prompt, queued steers and the first step after a fold all escaped it. Those are also the cases where the estimate is weakest, since there is no recent provider count to anchor it, and across all 24 folds of those days the estimates had run 39 to 63% low. An unanchored estimate is now multiplied by the ratio observed for that model, limited to between 1 and 2.5 and learned from fold receipts that survive a restart. As first delivered, that calibration was never applied, lived only in memory and was shared across models. Overflow itself is now read from the structured status first, with Go’s escapes decoded, and it is never treated as transient.

With contexts staying large, what could change was what they held, and tool results the model had already acted on were never being removed. A result now has a lifecycle: fresh when it arrives, consumed once the model has taken a later step, and then replaced by a receipt giving the call, its size, its SHA-256 and its first line, with the full text kept outside the context. These folds need no model call and happen in declared epochs, once context has grown by a fixed step and the fold would free at least 16,000 tokens, so the prompt’s prefix stays byte-stable between them. Questions to the person, loaded skills, todo lists, notes, images and the newest step are never folded.

What it looked like running

We checked the release with short replay scenarios, which is how I prefer to work on this kind of problem: start close to where the failure happens, run it, fix it and run it again. In the first, an agent had to run a long foreground command and answer a steer during it. Its initial attempt, sleep 75, was refused by the guard against sleep-only commands, so it ran an 80-second busy loop instead. The command moved to the background inside the bound, a steer was answered four seconds after it was sent with the command still running, and the command’s output then arrived through await, with no failed tool calls in the turn.

The second was a long steered turn crossing compaction, with the trigger set low at 52,000 tokens to force the issue. The steer was followed, and two mandatory folds ran inside the turn, each before the next request, taking it from 86,949 tokens to 32,812 and from 88,222 to 44,682. In an earlier round the fold had run only after the turn ended.

The peak request in that run was 82,028 tokens, 58% over the trigger, which looks exactly like a missed fold. The log says otherwise. After the first fold settled, the budget projected 65,419 tokens and postponed for the waiting person, projected 82,008 and postponed again, using up the allowance, and then projected 88,222 and declared the fold mandatory, which ran before the next request. Each step was adding 17,000 to 21,000 tokens of file pages, so the overshoot is two postponed steps and nothing else. A trigger 16 times below the ceiling makes that look dramatic. At the default trigger, the ceiling caps the same growth at about 7%, and an operator who wants no postponement can set the allowance to zero.

Those were single scenario runs, so they show the mechanisms working and are not a latency figure for production. Reading back through the review, many of the defects it found came from the engine believing something more convenient than the actual state: that a yielded call had finished, that an unknown outcome was known, that the next request would look like the last one, or that it would ask for the output the model’s profile advertised. A turn stays steerable when its boundaries keep arriving, and it stays within budget when each boundary looks at the request that is actually about to leave.

Chris Chabot, November 2026