Skip to content

8 min read

Deleting an unverifiable rule beats hedging it

On this page

Two days into the inherited 4,000-token file, nine entries had been retired on the cheap checks and one entry would not resolve either way:

markdown
Do not run `reconcile --repair` against the primary. Use the replica
snapshot job (`ops/reconcile-from-snapshot.sh`) instead.

Both things named still exist. The --repair flag is in the CLI, the snapshot script runs nightly, and neither has been touched in two years. There is no incident number, no date, and no owner; git log attributes the line to a single commit in 2024 titled "notes for the agent", authored by someone who left the company that autumn. The obvious reading is that --repair once did something dangerous against a live primary — took a long lock, wrote without a transaction, raced with the ingest worker — and that it may have been fixed in any of the eleven releases since, or may not have been. Reading the source of reconcile shows a repair path that takes a row-level lock and commits in batches of 1,000, which looks safe now and says nothing about what it did in 2024 or about failure modes under load.

The teammate reviewing the cleanup made the argument that this chapter is about. Keeping the line costs perhaps 30 tokens a session; removing it, if the danger is real, costs a production incident on the primary ledger database. Those are not comparable magnitudes, so keep it. He then proposed what feels like the reasonable compromise: annotate it — may be out of date, verify before relying on this — so nothing is lost and the agent can judge.

The asymmetry is real, and it holds for a minority of entries

He is right about the direction. The cost of keeping a stale entry and the cost of removing a live one are drawn from very different distributions, and anyone arguing that a config file should be pruned aggressively has to take that seriously rather than dismiss it as timidity. Where the argument is weakest is in assuming the asymmetry applies uniformly, when it varies by class of entry in a way that is fairly easy to reason about.

Entry classCost if kept and staleCost if removed and still neededWhich way it leans
Style preferenceTokens; occasional pointless reformattingSlightly less consistent codeEither; barely matters
Fact with an active correction channelTokens; one failed call that self-correctsOne failed call that self-correctsKeeping is genuinely cheap
Instruction duplicated by a lint or hookTokens onlyNothing — the check still firesRemoving is nearly free
Procedure naming a step that existsTokens; occasional wasted stepA skipped step someone noticesKeeping is defensible
Prohibition with no enforcement behind itUnbounded and invisibleThe prohibited action happens once, visiblyRemoving is usually better

The first four rows are where the intuition comes from, and in those rows it is sound. Most entries in most files are those rows, which is why "leaving it in is safer" feels true to people who have spent their attention on ordinary config files rather than on incidents.

The last row is the one under discussion, and chapter 1 gave the reason it behaves differently: a prohibition that no longer applies has no discovery channel, because the action it prevents produces no error, no latency measurement, and no log line. That property also breaks the cost estimate. What the retained prohibition costs is the value of the option it forecloses, in situations you cannot enumerate in advance, which means "just some tokens" is not a small number so much as a number computed only over the visible costs of an entry whose costs are invisible by construction. Concretely for this line: during a reconciliation failure at 02:00, the repair path takes four minutes and the snapshot path takes fifty, and an agent holding the prohibition will route around the fast path without mentioning that it did so. That is one incident-length delay per occurrence, unbudgeted, in exactly the situation where the delay is most expensive.

There is a second mechanism that does not depend on any single line. A file with unowned, unverified entries teaches every future reviewer what the file is for, and entries get added to match the standard already visible in it. Chapter 6's budget only produces good decisions when the reviewer will read the whole file and rank against it; a file people have stopped reading as a whole is one where the genuinely load-bearing invariant about checkout and ledger sits in the same undifferentiated block as somebody's 2024 note, and gets the same amount of attention.

What the hedge actually does

The annotation proposal — leave the entry, mark it doubtful — is the option that sounds like it dominates both alternatives, and it is worse than either. Running the three variants through chapter 4's harness on a task that touches reconciliation makes the mechanism visible.

ArmFollowed the prohibitionIgnored itRaised it with the user
Line present, unqualified8 / 800
Line present, marked "may be out of date"4 / 83 / 81 / 8
Line removed—8 / 80

The hedged arm is the only one whose behaviour you cannot predict. The false constraint is still in the context doing its work, and the qualifier hands the model licence to decide per session whether it applies, resolved by whatever else is in that particular window. Wrong-but-predictable behaviour is a thing you can debug: it reproduces, the trace cites the rule, an ablation isolates it, and chapters 3 and 4 both work. Inconsistent behaviour defeats those instruments directly, since the chain of custody shows one session out of several and the removal arm returns a muddy difference that a reviewer can attribute to variance. Hedging an entry removes your ability to detect the problem you hedged it for.

The organisational effect is the more lasting damage. Writing "may be out of date" converts an open question into a documented state, and documented states do not get worked on. Nobody is on the hook to run the experiment, because the uncertainty has been acknowledged, and acknowledgement reads as diligence in a way that leaving the bare line does not. A year later the annotation is still there and the file has three more of them.

When you cannot verify and cannot afford to be wrong

For the reconcile --repair line specifically, there is a move that dissolves the dilemma rather than resolving it, and it is chapter 6's table applied under uncertainty. The prohibition is advisory, which is why its truth matters so much; make it enforced, and its truth stops mattering, because the cost of being wrong drops from an incident to a prompt.

bash
#!/usr/bin/env bash
# bin/reconcile — wrapper on the vendor CLI, first on PATH
if [[ " $* " == *" --repair "* ]] && [[ "$RECONCILE_DSN" == *"primary"* ]]; then
  echo "reconcile --repair against primary is gated." >&2
  echo "Provenance unknown (AGENTS.md, 2024, author departed);" >&2
  echo "it may be safe now. To proceed: --repair-primary-approved-by <name>" >&2
  [[ " $* " == *" --repair-primary-approved-by "* ]] || exit 1
fi
exec /usr/local/libexec/reconcile "$@"

The gate costs nothing per session, it applies to humans as well as agents, and it converts an unverifiable claim into a speed bump that carries its own provenance. The config line then comes out of the file, because the behaviour no longer depends on the agent having read it. This is the general shape for entries in the bottom row of the table that you genuinely cannot resolve: move the constraint to where it executes, then delete the sentence.

Not every unverifiable entry deserves a wrapper — most are not about anything dangerous enough to justify building one — which is why the default still has to be a deletion rule that a person can apply without judgement.

A rule that survives literal application

The teammate asked for something he could apply mechanically, and the honest constraint on such a rule is that it will be applied to entries you have not imagined, by people with less context than you, on days when they are in a hurry. What makes literal application safe is reversibility rather than caution, so the rule is built around making removal cheap to undo and around putting the burden of evidence where the knowledge is.

An entry that asserts a fact or forbids an action, and for which nobody can produce evidence from the current system in thirty minutes, is removed in a pull request that records what was tried. The removal is logged in AGENTS.removed.md. The owning team is notified and has two weeks to restore it with evidence attached, in which case it comes back with an owner and a trigger. Never annotate an entry with doubt: verify it, delete it, or convert it into a check.

The thirty minutes is what keeps it mechanical. It is not a judgement about how important the entry seems, which is the judgement that always resolves in favour of keeping; it is a bounded search, and if the search fails, the entry has failed a test rather than lost an argument. The two-week window is what makes deletion safe to apply literally, because the failure mode of removing something needed is bounded by how quickly it comes back, and a notification with a diff attached is far more likely to produce evidence than any survey ever was. People who could not recall a rule in the abstract can usually say "yes, that one, here is why" when shown a deletion.

The log is what stops this being lossy:

markdown
## reconcile --repair on primary
- Removed: 2026-08-09 in #4602 · previously in `AGENTS.md` since 2024-03-11
- Author departed; no incident reference; no owner
- Checked: reconcile source (row locks, 1,000-row batches), 2 yrs of
  ops channel search, on-call runbook, 11 release notes — nothing found
- Replaced by: `bin/reconcile` wrapper gating `--repair` against primary
- Restore with: evidence that the repair path is unsafe under load

AGENTS.removed.md is never loaded into the agent's context, so it costs nothing per session, and it answers the question that otherwise re-litigates every deletion eighteen months later when somebody wonders whether there used to be a rule about this. Together with the entry index from chapter 5, it means the file's history is legible without anyone having to remember it.

What you hand back at the end of the two days is therefore more than a shorter file. There is the file itself, down from 4,000 tokens to about 1,400. There is AGENTS.entries.yml giving every surviving entry an owner, a verification command and a trigger, which is what makes the next audit a matter of running commands. There is the removal log with the evidence trail for each deletion. There are the two or three constraints that moved into wrappers, hooks or lints, which are now enforced rather than hoped for. And there is a CI check on the budget, so the next person who wants to add a line has to make the comparison rather than the addition.

That is the working state, and it is maintained the way any other production artefact is: when a session goes strangely, check the assembled config before blaming the model; when a migration lands, answer the trigger with a query rather than a shrug; when a retro proposes a new line, price it against the weakest line already there.

Config hygiene — read the current version at https://daviddominguez.dev/learn/en/config-hygiene/deleting-an-unverifiable-rule-beats-hedging-it

Back to top