Two independent editing passes, two separate documents, on the same night. Both passes staged suggestions built on quoted lines that were not in the actual text. Both died at apply-time.
i.How the failure presented
The editing workflow goes like this. An AI editor reads a document, identifies candidates for revision, and stages each suggestion as a structured edit: the current line, the proposed replacement, and the reason. Before any suggestion applies, the workflow byte-verifies the current-line string against the source document.
On this night, two separate editing passes worked two separate documents. When the verification step ran on the first pass, the quoted current-line did not appear in the source. Not a paraphrase match. Not a near-match. The string was not there.
The second pass, on the second document, failed the same check. One of the suggestions had built a repetition argument on a quoted phrase the editor described as appearing “twice in this section.” The phrase appeared zero times. The editor had invented the repetition in order to justify the suggestion.
Neither pass applied any changes to either document. The verification gate had done its job.
ii.Why this happens
An AI model reading a long document holds a compressed representation of what it read, not a verbatim copy. When it surfaces a quote, it is reconstructing from that compressed representation. The reconstruction is usually close. Sometimes it drifts. The model does not experience the drift as an error. It experiences it as a quote.
This failure mode is not a sign of a broken model or a bad editing pass. It is a structural property of how language models handle long-context retrieval. It shows up on creative work because creative editing requires precise reference to the source. The argument “this line is weak because of its relationship to that line” only holds if both lines are accurately identified.
The failure is quiet. An editor that stages suggestions with plausible-sounding quoted rationale, in a workflow without byte verification, will apply changes to source material that may or may not be the changes that were argued for. The edit lands; the argument it was based on is invisible.
On AI editing workflows Deterministic gates earn their keep on creative work, not just code. Byte-verify the full passage, not the memorable line.
iii.The fix that held
The byte-verification gate was already in place before this night. What the night confirmed was that the gate is not paranoid. It is necessary, at a rate higher than expected, on documents long enough that the model cannot hold the full text verbatim.
The lesson that became a standing rule: in any AI editing workflow, the target string of every staged edit must be byte-verified against the source before the edit applies. “Verbatim” does not mean the model’s reconstruction of the text. It means the characters in the file, in order.
This is not about distrust of the model’s judgment on what should change. The suggestions may be correct. The argument for why they should change may be sound even when the quoted evidence is wrong. The byte-verification gate is not a quality judgment on the edit. It is a safety gate that prevents the workflow from applying changes to the wrong location in the document, or from building on quoted evidence that does not exist.
iv.Deterministic gates on creative work
The reflex in engineering is to reach for deterministic verification on code and to treat creative work as a judgment domain where mechanical checks do not apply. This is wrong.
Creative documents have structure. They have specific sentences in specific positions. They have quoted passages. They have character names, place names, technical vocabulary that appears at exact locations. The same structural properties that make code verifiable make long-form documents verifiable at the reference level.
A gate that checks whether a quoted line exists in the source before applying an edit built on that line is trivially implementable. It catches a failure class that is otherwise invisible. It is worth building before the first editing pass, not after the first failure.
The two editing passes on this night cost no document integrity. The failures surfaced cleanly, were logged as rule-confirmation, and became a specification addition for any future editing skill: byte-verify the edit target, always, before application.
Drafted with Bishop, my AI partner.
Words picked, edited, and approved by me.
Model provenance: Claude Code (Claude Opus and Sonnet)