Choose the format from the cap, the change size and the verifier

A settlement service written in Java in 2014, still processing about 40 million rows a month, and now the target of an agent that will pick up small tickets against it. SettlementBatchProcessor.java is 2,340 lines. LedgerReconciler.java is 3,180. Files under a thousand lines exist but are the minority. The tickets are things like "treat a null external_ref as unmatched rather than throwing" — five to twenty lines, inside one method, no signature changes. The model available has an 8k output cap. The test suite has 3,140 tests, 84% branch coverage on the packages the agent will touch — branch coverage being the fraction of the individual arms of every if, switch and loop condition that some test actually executes, a stricter measure than the fraction of lines run — and it takes six minutes on CI.

Three of those facts decide the format before any measurement, and they decide it in a specific order.

The cap eliminates whole-file rewrite outright. Java runs dense — long identifiers, package-qualified types, annotations — so twelve to fifteen tokens per line is the right planning figure, which puts SettlementBatchProcessor.java at 28k to 35k output tokens against 8k available. That is not a reliability judgement about rewrites; it is that the format cannot be emitted for the majority of files in this repository, and every attempt would leave a truncated file on disk where a working one used to be.

The change size decides between the two survivors. A twenty-line edit expressed as a search-and-replace block costs roughly the size of the change twice — perhaps 600 output tokens including the surrounding anchor context — which is 7% of the cap and leaves the model plenty of budget to reason before it writes. Output scaling with the size of the change rather than the size of the file is the property that makes this repository tractable at all, and unified diff has it too, at slightly lower token cost.

The suite breaks the tie, and it does so by covering exactly the weakness search-and-replace has left. Its residual failure is an edit that lands in the wrong place: a unique-looking anchor that turns out to be in Order rather than Cart, or a body shared by three classes. An edit that lands in the wrong method changes behaviour that some test exercises, and at 84% branch coverage on these packages a wrong-place edit fails a test with high probability, at a cost of six minutes and a rerun. Unified diff's residual failure in this repository is the multi-hunk arithmetic from chapter one, and while the suite catches those too when they misapply, most of them fail at the applier and cost a retry that the search-and-replace path largely avoids.

That interlock is the part worth generalising. Every format leaves a residual failure mode, and the format to pick is the one whose residual your verifier can see — the verifier being whatever machinery in the repository can decide, without a human reading the diff, that an edit is wrong: the compiler, the type checker, the linter, the test suite, or all four in sequence. A repository with 12% coverage and no type checking has no verifier to catch a wrong-place edit, which changes the answer for the same files and the same model — there, the uniqueness and symbol-span checks from chapter four stop being a nice guard and become the only thing standing between the agent and a silent defect, and you would weight far more heavily toward small, tightly scoped anchors and human review of every diff.

The inputs, and what each one moves#

Working the same six inputs across four situations gives four different answers, which is the clearest evidence that there is no universal format. Two of the rows below use shorthand worth spelling out: greenfield means a codebase written recently and from scratch, with none of the accumulated size and convention drift of a decade-old service, and tsc is the TypeScript compiler, which in a typed codebase rejects a large fraction of wrong edits in a second or two without any test having to run.

The third row is the one that surprises people who read the first three chapters and concluded that rewrites are always wrong. A migration touching 60% of a 900-line module produces a search-and-replace patch with thirty separate anchors, thirty chances of a byte mismatch, and a compounding probability that at least one fails; expressed as a rewrite it is a single 11k-token output that fits a 32k cap with room to spare, and the file's own structure is regenerated coherently rather than patched thirty times. The format follows the shape of the change.

Standardising on the strongest model's format puts the failures in the wrong place#

A pipeline that must run on three models — a frontier model, a mid-size one, and a small local model for cheap high-volume tasks — invites the proposal to pick one format for all of them, on the grounds that the frontier model handles unified diffs well enough that the question is settled.

Format is not a property of the model alone, and the cases that break each single-attribute claim are easy to construct from this chapter's own examples. Holding the model fixed at the frontier one, the greenfield TypeScript row and the legacy Java row get different answers, so it is not the model alone. Holding the repository fixed at the legacy Java service, the small local model cannot reliably reproduce a forty-line anchor byte for byte and does far better with tightly scoped six-line anchors and a re-read before every quote, so it is not the repository alone. Holding both fixed and varying only the task, a twenty-line fix and a package-wide rename want different formats. The decision is a function of all three, which is why it belongs to a routing layer rather than to a constant.

The specific damage from standardising on the frontier model's format is that it sets the interface by the model that needs the least help. Hunk-header arithmetic and verbatim context reproduction are exactly what a small model is worst at, so a single diff format concentrates the failures on the model with the tightest retry budget, working on the tasks whose value is lowest, where a six-minute suite run per retry costs more than the task is worth. The strongest model would have been fine with anything.

The simplicity argument is worth taking seriously, because a pipeline with three parsers, three appliers and three failure taxonomies genuinely is worse. The design that keeps the simplicity where it matters puts one internal representation in the middle and makes the format a serialisation detail at the model boundary:

type EditOp =
  | { kind: "replace"; path: string; anchor: string; text: string; target?: string }
  | { kind: "write";   path: string; text: string };

interface FormatAdapter {
  instructions: string;                    // goes in the system prompt
  parse(modelOutput: string): EditOp[];    // the only format-specific code
}

const ROUTES: Array<{ when: (t: Task) => boolean; adapter: FormatAdapter }> = [
  { when: t => t.fileLines * t.tokensPerLine > 0.5 * t.model.outputCap, adapter: searchReplace },
  { when: t => t.changedLines / t.fileLines > 0.5,                      adapter: wholeFile },
  { when: t => t.model.tier === "small",                                adapter: searchReplace },
  { when: () => true,                                                   adapter: wholeFile },
];

Everything downstream of parse — the uniqueness count, the symbol-span check, the snapshot, the failure taxonomy, the retry policy, the metrics — is written once and shared. What varies per model is a prompt fragment and a parser, which is a few hundred lines in total. The pipeline that the product manager wants to keep simple is still simple; the thing that varies is the smallest surface that can vary.

The part of that design least worth trusting is the routing predicate itself. tokensPerLine is an estimate that differs by language and by file, changedLines is a guess made before the model has decided what to change, and the 0.5 thresholds are round numbers that came from one fixture run on one model version. It is a heuristic dressed as a rule, and the honest mitigation is to log the route taken alongside the four outcomes from the previous chapter so that the thresholds can be moved on evidence rather than on argument.

Reading a production dashboard without over-reacting to it#

An agent on unified diffs, 400 production edits, three numbers: 8% of patches fail to apply and are retried, 1.5% apply with fuzz into the wrong location and are caught in review, and multi-hunk patches fail four times as often as single-hunk ones.

The 1.5% is the one to worry about, and not because it is larger — it is five times smaller. It is a detection-limited estimate rather than a count. The 8% is measured by the applier, which sees every failure it produces, so 8% is the true rate. The 1.5% counts only the wrong-place edits a reviewer happened to notice, and reviewers notice a misplaced hunk in a small diff far more often than in a large one, so the real rate is 1.5% divided by an unknown catch rate. If review catches half of them, the true figure is 3%, and the uncaught half are in production. The 8% costs a retry, which is thirteen seconds and a fraction of a cent, and it is fully observable. The 1.5% costs a defect and is not.

The third number is the actionable one, because it names a lever. Multi-hunk patches failing four times as often is chapter one's compounding drift, and it is a property of how the agent batches its edits rather than an intrinsic property of the work. Splitting a multi-hunk change into separate single-anchor edits removes the compounding entirely.

Switching everything to whole-file rewrite moves all three numbers, and not uniformly in the right direction. The 8% goes to approximately zero, which is the cheapest and loudest failure category being eliminated. The 1.5% wrong-location number goes to zero for that mechanism and is replaced by a collateral-change category — dropped methods, reordered imports, reverted lines — which the fixture run in the previous chapter put at 11% of applied attempts, silent and reaching review looking clean. The multi-hunk ratio disappears as a category and is replaced by sensitivity to file size, which on this repository means the 2,300-line files cannot be emitted at all. The proposal trades a visible 8% for an invisible number that is larger.

The change worth making instead is search-and-replace with the uniqueness and symbol-span checks, one anchor per edit, and re-reading the file between edits in the same run. What would change the answer is a fixture run — not production traffic — showing the collateral-change rate for rewrites under 3% on files that fit the cap, together with a cap large enough to hold the repository's median file. Those are both measurable, and if a future model with a 64k output cap and a low elision rate clears them, the arithmetic that ruled rewrites out here stops applying. Provider-supplied editing tools are moving in the same direction, standardising a str_replace operation as a first-class tool call with the parsing done outside the model, which removes the fence-parsing failures without touching the anchor problem underneath.

Whichever format ships, some fraction of edits will leave a file that does not parse. The next chapter is about what the loop does in the ninety seconds after that happens, which turns out to have exactly one correct first move and several tempting wrong ones.