Replaying injected memory without breaking extended thinking

Why memory injected into an extended-thinking request must be recorded and replayed at its original position, and how to keep history, canonical memory and admitted input apart.

October 20264 min readMemoryAgent systemsMemory as a service

Wash, scumble, knife marks, a scratched line and a thick dab move across the canvas in Prussian green and ochre, ending in dull vermilion.
On this page
  1. Capture and projection off the reply path
  2. Replaying admitted memory inside the prefix
  3. Accounting for signed reasoning blocks
  4. Testing beyond single-turn recall

A memory layer can pass its retrieval checks and single-turn tests, yet have its second turn refused because the context injected on turn one is gone. Most implementations treat retrieved memory as disposable, re-ranking and splicing it into each request and dropping it once the turn finishes. That keeps transcripts lean and works well enough in ordinary request-response loops.

With extended thinking, everything ahead of a signed reasoning block forms an immutable prefix, so once retrieved memory reaches the model, its lifecycle is governed by the provider’s prefix contract rather than the memory store. Injected context therefore belongs in append-only history, recorded verbatim at admission and replayed at its original position on every continuation.

I hit this building project memory for our hosted coding-agent platform, where each project has a long-running coordinator agent managing coding workers for its owner. A coordinator active for weeks has to retain standing preferences, such as keeping updates short because the owner reads them on a phone, alongside project decisions; otherwise the owner repeats the same guidance and the coordinator escalates questions it could answer. Project memory lives in one versioned Markdown file per project, one entry per line with its source, readable and editable by people and agents under the rule that memory is neither permission nor proof.

Capture and projection off the reply path

Project memory is only worth having if maintaining it neither turns the owner into a curator nor slows down replies. Background extraction reads existing sessions, labels each entry as something a person said, an agent inferred, or a tool observed, and publishes directly to the project’s Markdown file without an approval queue, so no fact waits for a human to release it. When a correction arrives, consolidation supersedes the replaced entry instead of leaving conflicting versions behind.

Reads stay off the reply path as well: foreground requests skip live search and model calls, drawing instead from a small local projection built in the background. Deeper lookups happen through explicit tools, the same split between always-present context and just-in-time retrieval used by Claude Code and by Letta, an open-source agent framework with tiered memory. Local selection and request checks took at most about 22 ms, and paired browser runs measured median first-reply times around two seconds whether memory was on or off, within provider jitter.

Replaying admitted memory inside the prefix

Getting capture, storage, and local projection right still leaves the provider boundary unsolved. Under Anthropic’s preserved-thinking documentation, everything ahead of a signed reasoning block must remain unchanged across continuations, and inserting a reminder on one turn only to remove it on the next counts as a prefix edit. Rebuilding turn two from stored messages alone drops or rewrites the memory block that preceded the signature; under strict continuity the request is refused, and looser recovery can discard the reasoning state.

Moving memory into the system message is the obvious shortcut, and it fails on both authority and replay. Placing extracted notes and agent inferences alongside system instructions promotes retrieved text to instruction authority, breaking that permission-and-proof boundary, and updating canonical memory mid-session still mutates the prefix.

Preserving replay requires keeping three representations apart. Committed conversation history stays untouched as the record of what people and agents exchanged. Canonical project memory holds provenance and supersession lineage in the versioned file. Beside both sits an immutable record of each admitted provider input, storing the exact segment sent, its insertion point, and its authority.

On continuation, admitted segments replay verbatim at their original positions before new turns append. When canonical memory changes mid-thread, the updated view appends after the preserved prefix instead of rewriting an older segment, and a new optional view cannot evict admitted input, truncate history, or reduce the output token reserve. Capture and compaction skip these admitted segments as source material so memory cannot learn from its own echo.

Accounting for signed reasoning blocks

Replaying full provider requests verbatim exposed an accounting fault at the same boundary. Our pre-flight context guard rejected a legitimate continuation with roughly half the context window free, because signed reasoning blocks were charged twice: once from the originating response’s output usage and again as serialized wire bytes during replay estimation. Each unchanged, authenticated reasoning block is now charged once, and that bound is accepted only from a completed response, since a stream can abort after emitting a signed block and before final usage arrives.

Testing beyond single-turn recall

Single-turn recall suites miss prefix violations, echo loops, and double-counted reasoning blocks because every case stops before turn two is assembled. Validating project memory requires running live capture, consolidation, and multi-turn continuation together.

In a fresh project, automatic background capture extracted two standing owner conventions in 40 seconds and a correction in 80 seconds, ignored transient remarks, and consolidated the result into two canonical entries with the replacement citing both sources. When a separate worker then read that memory and its report ran back through capture, zero duplicate entries were retained.

Chris Chabot, October 2026