Verification
Checks that could have failed: formal models, replay, acceptance tests and what a pass establishes.
2026
A catalogue import is not finished when the upload ends
A large book import passes through a source fixture, uploaded objects, an activated generation and a reader's filtered view, and each has its own count. Why a check has to name which count it means, and how to verify one without uploading again.
DataImportsVerificationFerrite
A desktop can look correct and still lose your work
A Firefox Undo that could take a whole note sat beside 5,550 passing screenshot comparisons. What pixel checks, behavioural journeys, store readback and honest reports each catch in a React rebuild of macOS Catalina.
VerificationBrowsersReact
A failed save should leave the person’s work where they can recover it
What a form owes you when the server refuses your text: where the only copy lives while a request is in flight, how a task board and a shared document editor were repaired, and how to test a refusal without hiding it.
EditingFormsConflictsVerificationBrowsers
A faster check has to fail the same way
Two optimization campaigns and a queue change on agent tooling, where golden outputs, differential checks and A/B gates test whether a speedup keeps its behaviour.
Agent systemsPerformanceVerification
A running app still needs to publish a usable page
Why a passing health check says nothing about the page: how the Macalicious desktop publishes its frontend into a blob at boot, why startup writes need their own budget, and what asset-before-index ordering still doesn't guarantee.
DeploymentVerificationReliabilityMacaliciousFerrite
Authentication is only the first part of deciding who may use an app
What an application still has to decide after Google or GitHub has verified someone: who is admitted, how fast a removal takes effect, what an invite may do, and what happens when an app changes hands.
SecurityVerification
Compacting agent conversations without losing the source
How a compaction ledger keeps instructions verbatim, ties each fact to the journal entries behind it, and recovers original evidence within a fixed scope.
Agent systemsMemoryVerification
Comparing an agent’s context without copying the expected answer
What it takes to show a port builds the same model request: rebuilding each part from independently captured inputs, labelling copied parts as supplied, and reading zero differences beside zero complete requests.
SelectedAgent systemsPortingContextVerification
Copying code on purpose when applications can’t share a crate
When an application may link only one crate, shared code has to be copied. A canonical source tree, declared mounts, file hashes and a drift check keep the deployed copies aligned.
PlatformsVerificationSecurityPerformance
Correcting the grader without rewriting the run
A coding-agent evaluation whose first grader was wrong in both directions, and how original records, corrected verdicts and repairs stayed separate results.
SelectedEvaluationPlatformsVerification
Delivering background job results to the turn that owns them
Binding background jobs to their owning turn and sandbox so that completions survive restarts, arrive at least once and report a result someone can vouch for.
Agent systemsPlatformsVerification
Did your agent experiment test what you think it tested?
Wrong models, arms that never switched on, shifting workloads and features that ran without helping: checks that show an agent comparison contains the change.
SelectedAgent systemsEvaluationVerification
Evidence should belong to the run that used it
How per-run source rows, revision checks and publication-time binding keep a research report tied to the evidence it used, and what that doesn't prove.
Agent systemsMemoryVerification
Following an agent failure down to the process
Tracing a completed-but-failed agent run through thread, deployment, tool call, adapter and terminal pane, and what to do when logs and status mislead.
Agent systemsPlatformsVerification
Grading agents without trusting the simulator
An evaluation lab where the oracle never calls simulator code, grades in ten gates, tests itself with mutations and can answer that it doesn't know.
SelectedAgent systemsEvaluationVerification
Keeping scheduled work moving after a crash
A morning pipeline built on durable, date-keyed obligations, leases and a five-minute sweep, so a crash between steps cannot silently drop the work that is owed.
Agent systemsPlatformsVerification
Making a search index prove itself at query time
A search engine that treats every posting as a lead, publishes updates by stage-then-flip, and checks each candidate against one served version before its ACL runs.
PlatformsVerificationSecurity
Monitoring should keep working when one of its own checks breaks
One alert policy the engine refuses could stop every policy sorted after it from running. Why validation can't prevent that, where the failure boundary belongs, and how the same question applies to notifications, overloaded ingestion and diagnostic bundles.
VerificationObservabilityReliabilityRust
One application definition, several ways for an agent to use it
Generating discovery files, OpenAPI and MCP tools from one declared surface keeps the formats agreeing. It can't make a flag true, write a recovery procedure or count as extra test coverage. A skills pilot and a live byte comparison show where the line falls.
SelectedAgent systemsMcpApi designVerification
“Ported” is several different claims
Why a parity percentage can't describe a port of an agent toolkit: a ledger that keeps gaps as rows, and the separate proofs a behaviour needs natively, in a constrained application, when hosted and against a real provider.
PortingVerificationAgent systemsArchitecture
Porting an agent by making omissions visible
How a checklist generated from the syntax tree, clean-context reviewers and a ledger that keeps “translated” apart from “reviewed” exposed what a model-assisted port of a 340,000-line Python agent had left out.
SelectedAgent systemsPortingCode reviewFormal methodsVerification
Recovering a deploy that may already be serving
How a deployment client journals an attempt before submitting, retries under one operation ID, and after a crash accepts only status bound to that operation.
PlatformsVerification
Recovering a service deployment after its visible work has already happened
What a deployment controller has to do when a route has gone out and the catalog write fails: why a completed saga step can't be trusted on resume, how a retry recognises its own intent, and why cleanup is bound to the version it read.
RuntimeVerificationDeploymentReliability
Recovering an agent’s half-finished edits
How expected and intended digests let a restarted coding agent tell whether each file edit landed, is still owed or was overtaken, and when it must refuse to write.
Agent systemsVerification
Replaying the deterministic part of an agent
Recorded model replies and terminal output drove a port through whole sessions, so each difference could be located, classified and rechecked from a checkpoint. What the loop found, how its masks stayed honest and where live runs fit.
SelectedAgent systemsPortingReplayVerification
Reviewing code that won’t hold still
Binding review approvals and test results to the exact source revision they examined, and stopping when the latest change has gone unread.
Agent systemsVerification
Shipping an SDK that does not depend on the developer’s laptop
What has to be true before a private source tree can become a usable public SDK: fresh snapshots, a private manifest, a checker that never runs exported code, clean-container consumers and generated notices.
SelectedSDK contractsVerification
Ten stories is a maximum: letting a morning briefing run short
How a briefing pipeline lets issues run short: a per-source cap, an evidence floor, smaller budgets for thin stories and a lexical audit of the final prose.
Agent systemsVerification
Testing a release without turning every release into a research project
How a release on an agent-operated platform stays finite: scope chosen once, evidence reused only where it still applies, limits on retries, and a plain record of what never got checked.
SelectedRelease engineeringVerificationAgent systemsOperations
Testing the promises an API actually makes
How to test what a multi-service JSON-RPC API promises: coverage from its advertised methods, hostile payloads, concurrent writes and literal replay checks.
SelectedInterfacesSecurityVerification
The test passed, but the person still could not do the thing
What a browser test has to establish beyond a 200 or a successful click, followed along a new user’s path from documentation to clipboard to the app and its dashboard.
SelectedVerificationBrowsersRelease engineering
What a double-entry ledger can promise without a transaction
A double-entry ledger on a store with only conditional writes: a journal entry lands whole, account totals catch up, and retries recognise work already applied.
PlatformsVerification
What a plugin upgrade is allowed to destroy
Plugin packages, plugin data and sandbox caches have different lifetimes. How an installer applies digests, atomic renames and restore rules to each one.
PlatformsSecurityVerification
What an evidence vault can prove without a notary
What content hashes, separated review and a chained audit log establish inside one application, and where blob deletion and the signing key's holder limit them.
SecurityVerification
What “clean locally” is allowed to mean
How a local platform emulator reproduces leases, retries and denied egress, and says where its verdict stops at the moment a check comes back clean.
PlatformsVerificationInterfaces
What “completed” means in an AI gateway
Why an AI gateway held back the terminal event until stored responses had landed, and what that release path did and did not promise about usage accounting.
PlatformsInterfacesVerification
What did the agent actually change?
How an agent platform computes a pull request from a registered base and a scoped view of the workspace, and keeps absent, unverifiable and unknown outcomes distinct.
Agent systemsPlatformsVerification
What happens when two windows edit the same note?
Two windows save the same note and the later save is refused. What happens next decides whether the person keeps their text: a revision check, a three-way merge applications opt into, a Keep Mine choice, and a test that holds a save back on purpose.
ConcurrencyCollaborationFrontendVerification
What should be identical in a five-language agent SDK
Which parts of a multi-language SDK must match exactly, such as errors, identity and event order, how shared fixtures enforce them, and what each language keeps.
SelectedAgent systemsInterfacesVerification
When a usage dashboard is entitled to show zero
Treating usage replies as evidence: a browser parser that refuses inconsistent numbers, and display states that separate an observed zero from unavailable, stale or partial.
InterfacesPlatformsVerification
When the evaluator is the broken part
Sixteen generated apps passed their functional checks while the benchmark recorded two passes. Separating what was built, how it was built and what the observer could prove explains the gap.
EvaluationVerification
When the older worker finishes last
How a change-tracking pipeline fences cursor moves, repairs a stale publish across D1 and R2, and separates an empty source from an unreachable one.
PlatformsVerification
Whose mistake did the coding-agent benchmark catch?
Reference and empty-patch controls, test-file collisions and environment faults: how to adapt a coding-agent benchmark and tell an agent's failure from the harness's.
SelectedAgent systemsEvaluationVerification