Putting scripts in charge of agent-run releases

How fingerprinting inputs makes release test results reusable, why a post-mortem's lesson has to become a script that refuses, and how to keep acceptance evidence through a rolling upgrade.

October 20264 min readRelease engineeringAgent systemsVerificationRelease qualification for a cloud platform

Broad horizontal sweeps of raw ochre and oxide rust fill the lower canvas, scraped to show green-black, beneath thin warm chalk and one skipping dark stroke.
On this page
  1. Fingerprinting release checks
  2. Turning post-mortems into refusals
  3. Two commands for rolling upgrades
  4. Acceptance evidence that survives the roll

Write the fix for a failed release into a coding agent’s runbook after a careful post-mortem, and the next release will repeat the failure verbatim if nothing enforces it. Agents are good at the diagnostic side of release work, reading failure logs and patching broken test drivers. They also optimize to keep moving and close out the turn, which is useful right up until you make one the scheduler. In that role, an agent treats prose instructions as advice, re-proving unchanged components and retrying past conditions that should halt the run.

The principle that makes agent-run releases repeatable is “scripts decide, agents diagnose”: what runs, in what order, and whether to proceed belong in deterministic code that can refuse, while agents read failures and propose fixes. I built this pipeline for Ferrite, my cloud application platform, which combines a core developed and certified by another team with my services, dashboard, CLI, MCP server, and about fifty demo apps, running in production as Ferrite Cloud at ferrite.cloud. Coding agents run each release under my direction, which meant moving every control decision out of their instructions and into scripts.

Fingerprinting release checks

Without a cache boundary, an agent asked to qualify a release verifies everything it can reach. A full verification run of Ferrite took ten hours, mostly re-deploying and re-browsing unchanged demo apps. Of 180 catalogued issues in that run, only 11 were real defects; the rest came from drifted tests, missing runner setup, stale artifacts, and local capacity limits.

Build systems like Bazel cache by content hash; the same principle applies to agent and browser verification of a release. Every testable unit gets a SHA-256 fingerprint over the pinned platform composition and the Git object IDs of everything it depends on, including the scripts the check runs through, so changing a verifier invalidates its own passes.

A passing receipt stands only for that exact fingerprint, turning release preparation into mostly a lookup: the pipeline executes only units lacking a valid receipt, cutting preparation from ten hours to under three. Each run writes its own event ledger with classified causes instead of leaving an agent to reconstruct failures from logs afterwards.

Turning post-mortems into refusals

Before rolling upgrades, releasing Ferrite Cloud meant building a replacement cluster and cutting traffic over. Even with fingerprinted verification, one replacement install took about 20 hours of active work, so I had eight reviewer agents audit the session transcripts.

The mechanical steps took about an hour. Roughly four hours came from how the operating agent drove the work, from idle waits to reporting the cluster ready before running browser checks. The reviewers’ central finding was that the most important fix from a prior failed install had been written into an agent skill rather than a script; on the next attempt, an operating agent skipped it and overrode the tooling’s refusal to start.

A live cluster runs the upstream core plus my patches and the installer’s configuration rewrites, and the upstream team’s certification suite of about 11,000 tests must pass on that exact tree. Certifying only after provisioning a fresh cluster put hours of avoidable failure inside the release window. Every remedy moved into code that fails closed: the cluster tree is composed and certified locally the day before the window, and a readiness gate refuses the first mutating step until every required receipt is valid for that frozen release.

Two commands for rolling upgrades

Rebuilding a cluster on every release meant replacing serving infrastructure even though the upstream core already supported in-place upgrades. Two commands, prepare and go, now replace a reinstall. Both print their plan unless told to execute; when executed, they refuse before writing anything if a receipt is stale or names another tree, and resume from the last valid phase after an interruption.

For the rollout itself, go runs the upstream team’s roll script unchanged as a detached job and counts success only on a clean exit with the script’s success marker. Any other exit is classified explicitly so an agent diagnoses a named failure instead of guessing whether to retry.

Acceptance evidence that survives the roll

The pipeline carried out the first in-place rolling upgrade of ferrite.cloud in November 2026, showing why acceptance evidence has to survive the operation it judges. Before and after a roll, acceptance snapshots every account, deployment, domain, and live credential to confirm nothing disappeared. During the upgrade, live probes exercise the cluster, including a writer app that admits mutations by idempotency key, refuses reused keys with mismatched payloads, and records each admission before answering so any double admission is visible immediately.

Metering continuity could not be judged after that first roll. The platform’s metering statement API keeps only the newest 200 ledger lines, while the probes wrote about 25 lines a minute, so the probes evicted the roll’s entries from the window before the post-roll snapshot ran. A sampler now reads the statement continuously throughout the roll, and acceptance stitches every read by sequence number, failing on any gap.

Chris Chabot, October 2026