Stale guidance overrides the correct evidence sitting in the code

The task was to add a webhook call to the fulfilment provider. The agent's first attempt produced this, and then produced it again:

attempt 1/4
+ from platform.http.v1 import Client
  ModuleNotFoundError: No module named 'platform.http.v1'

attempt 2/4
+ from platform.http.v1.client import Client
  ModuleNotFoundError: No module named 'platform.http.v1'

attempt 3/4
+ import platform.http as http
+ client = http.v1.Client(timeout=30)
  AttributeError: module 'platform.http' has no attribute 'v1'

attempt 4/4
+ class _V1ClientShim:      # local shim, platform.http.v1 unavailable

Line 340 of the root config reads: All outbound HTTP goes through platform.http.v1.Client. Do not use requests or httpx directly. It was added fourteen months earlier for a good reason, which is that the wrapper carries the retry policy, the outbound allowlist and the trace headers. Six months earlier the platform team shipped platform.http.v2 with a different constructor and deleted v1, migrating 312 call sites in one change. Nobody edited line 340, because nobody knew it was there.

Each of those failed runs burned about 41,000 tokens and six minutes before ending in a shim that bypassed the allowlist entirely. The pattern appeared 19 times in a month across the repository, which is roughly $95 of wasted inference and, more importantly, one class of patch that silently opts out of a security control.

Wrong guidance beats correct code#

Delete line 340 and run the same task. The agent searches for HTTP usage, finds 312 call sites of platform.http.v2.Client, copies the nearest one, and gets it right; measured over 20 runs on the same task with the rule removed, 19 produced a correct v2 client and one used httpx directly. The codebase was already carrying the answer, in the form chapter 2 described — hundreds of demonstrations, which is the largest instruction source in any run.

The rule did not merely fail to help. It reversed a correct outcome, because a stated rule arrives before any evidence is gathered, comes from the position of authority the harness gives config files, and is phrased as a requirement rather than a pattern. The agent read line 340 as a constraint and treated the absence of platform.http.v1 as a problem to be worked around, which is why attempts two and three tried different import paths for a module that does not exist and attempt four built a shim. An agent given no guidance is a system that self-corrects against the code; an agent given wrong guidance is a system that argues with it.

That is the general lesson and it is worth stating without hedging: stale guidance is worse than no guidance. Everything else in this chapter follows from taking it seriously, because it means the maintenance burden of a conventions file is not optional tidiness but a correctness requirement, and a rule with no owner has an expected lifetime after which it becomes a liability.

Ownership and triggers are part of the rule#

The frontmatter from chapter 3 already carries owner. Two more fields turn that into something a machine can act on:

---
id: http-002
scope: "services/**/*.py"
owner: platform-runtime
anchors: ["platform.http.v2.Client", "packages/platform/http/"]
reviewed: 2026-02-14
---
All outbound HTTP goes through `platform.http.v2.Client`, which applies the
retry policy, the egress allowlist and trace-header propagation. Do not use
`requests` or `httpx` directly.

anchors is the field that would have caught this one. Every rule that names a symbol, module or path is making a claim about the codebase, and that claim is checkable:

import importlib, pathlib, sys, datetime

def check(rule: dict) -> list[str]:
    problems = []
    for anchor in rule.get("anchors", []):
        if "/" in anchor:
            if not pathlib.Path(anchor).exists():
                problems.append(f"{rule['id']}: path {anchor} no longer exists")
        else:
            module, _, symbol = anchor.rpartition(".")
            try:
                mod = importlib.import_module(module)
            except ModuleNotFoundError:
                problems.append(f"{rule['id']}: module {module} no longer exists")
                continue
            if not hasattr(mod, symbol):
                problems.append(f"{rule['id']}: {module} has no {symbol}")
    age = (datetime.date.today() - rule["reviewed"]).days
    if age > 180:
        problems.append(f"{rule['id']}: unreviewed for {age} days")
    return problems

Run that on every pull request and the v1 rule fails the day the platform team deletes the module, in the same change that deletes it, in front of the people who know what the replacement is. That is the whole point of a trigger: the review has to be forced by the event that invalidates the rule, because it will never happen on a schedule.

Version the rules as well as date them. A rule keeps its id for life and gains a revision each time its text changes, so http-002@4 is a specific sentence rather than a slot that has held four different sentences. That is what makes the instruction-set hash from chapter 3 useful across time: a run record naming http-002@3 can be joined to the exact wording that was in force, and a compliance number that drops the week after a rule was edited becomes attributable rather than mysterious. Without revisions, the log says a rule was active and the file says what the rule says today, and those are different claims separated by however many edits happened in between.

The scope glob gives a second trigger. If a pull request modifies files under packages/platform/http/, every rule whose scope or anchors intersect that path is surfaced for review. This is CODEOWNERS logic applied to prose — the convention, named after the file that carries it, of mapping path patterns to the teams whose review a change to those paths requires — and it has the same failure mode, which the last section of this chapter deals with.

The Active --> Retired edge on the right is the one teams forget to build. A rule that has been converted into a lint check, as chapter 5 recommends, should leave the file rather than sit alongside its own enforcement, because a paragraph restating a check that already fails the build is pure token cost.

Auditing 900 lines nobody owns#

Now the harder case: you inherit a repository whose agent config has grown to 900 lines over a year, written by eleven people, with no owner. A new engineer proposes deleting it and starting again, on the grounds that most of it is stale, nobody can say which rules do anything, and it costs tokens on every request.

Three of those claims are almost certainly correct. In the file this section is drawn from, 900 lines held 214 distinct rules; 38 of them referenced symbols or paths that no longer existed, 47 duplicated or directly contradicted another rule, and the file added about 11,000 tokens to every session. The diagnosis is right and the anger behind it is earned.

The proposal is still wrong, for two reasons. Some fraction of those 214 rules encodes something learned expensively — the egress allowlist rule exists because of an incident, the never write to the analytics schema from billing rule exists because of a data-integrity bug that took a week to find — and deletion discards that evidence without knowing which lines carried it. Nobody remembers which rule prevented which incident until the incident recurs. The second reason is that deletion treats the symptom: the cause is that no rule had an owner and no rule had a review trigger, and a fresh file written under the same conditions accretes the same way, only now it also lacks the good rules.

The alternative is an audit, and it is a process with an output rather than an afternoon of reading. Four passes, each mechanical enough that two people can do 214 rules in about three days.

The first pass groups rules by the code they touch. Assign every rule a scope glob, which forces the question of what it is about and immediately reveals the rules that cannot be scoped because they are not about anything — write clean code, think carefully about edge cases. Grouping also puts contradictions next to each other, which is how the 47 duplicates were found.

The second pass checks each rule against what the codebase does today. Anchor resolution catches the outright dead references. For the rest, count: the rule says queries go through the repository layer, so count the call sites that do and do not, exactly as chapter 2 counted 41 files of float arithmetic. A rule that the codebase violates in 80% of its own files is not a convention, it is an aspiration, and it should be labelled as one or dropped.

The third pass attributes each rule to an owner or to the incident that produced it. git log -L on the line range gives the commit, the commit gives the pull request, and the pull request usually gives an incident ticket or a review thread. On this file, attribution succeeded for 61% of rules; the remaining 39% went to the team that owns the code the rule's scope points at, with a note saying the attribution was inferred.

The fourth pass classifies by evidence. Does a machine check exist, or could one be written? Did the comparison from chapter 4 show a measurable effect? A rule with neither a check nor a measured effect nor an incident behind it is the retirement candidate.

The audit log is the part that answers the new engineer's real objection. Every deleted rule is recorded with its text, its final scope, and the reason for deletion, so when something recurs in eight months there is a searchable record of what was removed and why, which is precisely the evidence that deletion-and-restart destroys.

The part that will not hold#

The lifecycle is easy to design and the anchor check is easy to write. What decays is ownership, and it decays for organisational reasons that no amount of tooling fixes. The platform-runtime team that owns http-002 will be reorganised, its members will move, and the CODEOWNERS-style trigger will keep firing at a group that no longer exists. Ownership records need a periodic reconciliation against the actual org chart, and nobody wants that job.

There is a subtler failure that shows up sooner. If the stale-rule check blocks the pull request that broke the anchor, then an engineer deleting platform.http.v1 gets a red build about a config file they have never read, in a repository area they do not own, in the middle of a migration. They will do the fastest thing that turns the build green, which is to edit the rule to say whatever makes the check pass, and the rule will now be both current and meaningless. Route the trigger to the owner rather than the author — open an issue assigned to platform-runtime, annotate the pull request without blocking it, and let the migration proceed. The check that fires on the right event and lands on the wrong desk produces worse rules than no check.

Anchors also only catch the staleness that has a name. A rule saying use platform.http.v2.Client with a 30-second timeout stays green forever after the platform team changes the recommended timeout to 5 seconds, because the symbol still resolves. That class of drift is only visible through the measurement loop of chapters 4 and 6 — re-run the task set quarterly, watch for a rule whose compliance collapses or whose failure category shifts — and even that will miss rules whose bad advice does not break anything visible.

What this leaves you with is a working order of operations for a real repository. Take the inventory of what reaches the model and confirm it from the logs. Trace before rewriting, and count the counter-evidence in the code. Declare scope and precedence, log the resolved set, and hash it. Build a task set of forty convention-sensitive changes with verifiers, and run it both ways before claiming anything. Convert every rule that a checker can carry, and keep the prose that carries reasons. Price the tokens and the latency, and name the downstream metric the compliance number is standing in for. Then give every surviving rule an owner, an anchor and a date, and put the audit on the calendar for the quarter, because the file you just measured is already beginning to describe a repository that no longer exists.