Show the change as a diff and make the user press apply

The letter composer in the claims console had a control called Improve this paragraph. A handler selected a paragraph, pressed it, and the paragraph was replaced with a cleaner version about 1.4 seconds later. It was the most-used AI feature in the product, running on 38% of outbound letters, and it was replacing the editor buffer directly, which the composer then autosaved.

Six weeks after launch a policyholder disputed a deduction, quoting a letter that said an excess may apply to this settlement. The claims file said the excess was £250 and the handler's original paragraph had said so. The rewrite had made the sentence smoother and, in doing so, had dropped a number that the model had no reason to treat as load-bearing. Searching the outbound archive for that phrasing found 1,140 letters over the six weeks with the same substitution or one like it. There was no way to recover what the handlers had originally written, because the buffer had been overwritten and the autosave had committed the result; the only trace of the original text existed in drafts that had never been sent.

The engineering fix was not a better prompt. It was that a generated change to something a user owns is a proposal, and a proposal has to exist beside the original rather than in place of it, with a human action between the two.

The diff is only readable if the edit is small#

The obvious version of this — render the old paragraph and the new one side by side — was tried first and did not work, for a reason that is specific to generated text. Asked to improve a paragraph, a model rewrites the paragraph. Across 400 sampled rewrites the character-level diff touched a mean of 71% of the paragraph, so the side-by-side view showed two blocks of text almost entirely highlighted, and a handler comparing them was doing the same work as proofreading a fresh paragraph from scratch. The excess sentence would have been just as invisible in that view as it was in the overwrite.

What made the diff usable was changing what the model was asked to produce. Instead of a rewritten paragraph, it emits a list of targeted edits, each anchored to text that must match exactly.

{
  "edits": [
    {
      "find": "We have concluded our assessment of the matter and are writing to advise that",
      "replace": "We have completed our assessment and can confirm that",
      "reason": "shorter, plainer"
    },
    {
      "find": "the excess of £250 will be deducted from the settlement figure",
      "replace": "the £250 excess will be deducted from the settlement",
      "reason": "shorter; amount preserved"
    }
  ]
}

Applying edits by exact string match rather than by offset makes the operation verifiable, and it makes a whole class of failure loud instead of silent.

class AmbiguousAnchor(Exception): ...
class MissingAnchor(Exception): ...

def apply_edits(original: str, edits: list[dict]) -> tuple[str, list[Hunk]]:
    text, hunks = original, []
    for e in edits:
        n = text.count(e["find"])
        if n == 0:
            raise MissingAnchor(e["find"][:40])
        if n > 1:
            raise AmbiguousAnchor(e["find"][:40])
        start = text.index(e["find"])
        hunks.append(Hunk(start, e["find"], e["replace"], e["reason"]))
        text = text.replace(e["find"], e["replace"], 1)
    return text, hunks

Each of those edits renders as a hunk, which is the word a diff tool uses for one contiguous stretch of changed text shown with a little of the unchanged text on either side, so that the reader can see the change in place rather than as a bare pair of strings. Mean characters touched fell from 71% to 12%, the median rewrite produced three hunks rather than one, and the numeric substitution that caused the incident became a hunk with £250 on one side and nothing on the other, which is a difference a person notices in about a second. Anchoring also gave the system a free integrity check: a model that hallucinates the text it is editing produces a find string that is not in the document, and that raises rather than quietly appending.

The failure this does not catch is an edit the model declines to propose. Constraining output to a list of edits makes the model conservative, and on a small evaluation the constrained mode missed 14% of the improvements the free rewrite made. That is the trade, and for a letter leaving the building it is the right side of it.

What the apply step costs and buys#

Adding an explicit apply meant a handler now pressed Improve, read three hunks, and pressed Apply or accepted them individually. The numbers over the following quarter:

The eight-point drop in acceptance is a real cost and the sort of number that gets a feature killed in a review that only looks at engagement. It is also partly the feature working: 22% of accepted rewrites had a hunk removed, which means in the old design those changes were shipping unexamined. The row that pays for all of it is the third, where the rate of material unintended change in sent correspondence fell by a factor of seven, and the last row is what turned a six-week silent incident into something a query could have found on day one.

Recoverability has a storage meaning that is easy to get wrong. An undo stack inside the editor component is not recoverability — it dies on refresh, it does not survive a session, and it cannot answer what did the handler write before the assistant touched it six weeks later. The original has to be a persisted revision with its own identifier, written before the proposal is generated, and the applied result has to reference it.

Choosing the pattern for the stakes#

Diff-and-apply is not the right interaction everywhere, and applying it universally produces a product that feels like filling in forms.

The dimension that actually decides the row is not how good the model is; it is what happens after a wrong output is committed and how long it takes anyone to notice. A wrong autocomplete is noticed instantly by the person who invoked it. A wrong clause in a letter is noticed by a customer six weeks later, if at all. The excess incident sat at the wrong row of that table for six weeks, and the model's accuracy was never the variable.

There is a second variable underneath the table that is easy to conflate with stakes, which is how many changes a user is being asked to evaluate per unit of attention. A handler reviewing one rewrite an hour can afford to read a diff carefully; a handler processing forty claims a day with a rewrite on each is being asked to perform 240 reviews a week, and review quality at that volume converges on rubber-stamping regardless of how the interface is designed. Where volume is high, the honest options are to narrow what the feature is allowed to change, so that fewer hunks appear and each one is smaller, or to move the check off the human entirely by validating the specific property that matters — in the letter case, that every currency amount, date and policy number present in the original survives into the result. That validator was eleven lines of code, it ran in under a millisecond, and it would have blocked all 1,140 letters. A deterministic check on a narrow invariant is worth more than a human review that people have stopped performing, and the two are complementary rather than alternative.

For changes that are not prose, the same discipline needs a different rendering. A structured record update is best previewed as a field-level before-and-after, because a text diff of serialised JSON puts punctuation changes on the same visual footing as a change of settlement amount. An outbound action — send this letter, raise this payment request, close this claim — has no "before" at all, and the preview is a statement of what will happen and to whom, which is the subject of the next chapter.

A diff is not a review#

The most common overestimate of this pattern is that showing the change means the change was examined. On eight-hunk diffs in the composer, handlers pressed Apply all a median of 2.3 seconds after the diff appeared, which is faster than the text can be read. Acceptance was not review; it was a click that the interface had asked for and the handler had learned to produce.

Three things moved that. Keeping hunks few, by asking for at most four edits per invocation, brought the median deliberation time to 6.1 seconds on a three-hunk diff. Ordering hunks by risk rather than by position, so that any hunk touching a number, a date, a name or a negation sorts first, put the dangerous change where attention is highest. And treating those risk-bearing hunks differently — requiring them to be accepted individually rather than swept up by Apply all — cost handlers about 1.5 seconds on the 9% of rewrites that contained one. That last measure is a deliberate friction spend on a narrow slice, and it is the pattern that chapter 9 generalises.

Even with all of that, the diff protects only against changes the user can evaluate by looking. A rewrite that replaces a correct legal phrase with a different, plausible, incorrect legal phrase is visible in the diff and invisible to a handler who does not know which is right. Diffing localises the change; it does not supply the knowledge to judge it, and a product whose users cannot judge the domain needs a check that is not a human reading a diff.

The pattern is also the wrong tool where the generated output is the whole artifact rather than a change to an existing one. A first draft of a letter has no original to sit beside, so there is nothing to diff, and the protections have to come from the disclosure and marking of the previous chapters plus whatever sits between the draft and the outside world.

That gap is where the remaining exposure lives. The diff and its apply step govern the moment before a change commits, and they say nothing about the letter that has been sent, the payment request that has been raised, or the claim that has been closed. Those actions have left the building, and the affordance that is supposed to cover them is usually a five-second toast in the corner of the screen.