Config, skill, tool, nowhere: sorting a diff into four destinations

Here is a real pull request against the payments team's CLAUDE.md, opened the Monday after a bad release week. It is four additions and the author describes it in the body as "things we learned".

--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -38,6 +38,11 @@
 ## Rules
 
+- Release freeze is in effect until 2026-03-14. Do not merge to main.
+- The `billing` service never writes to the `analytics` schema. Publish an
+  event; the `analytics` consumer owns those tables.
+- To cut a hotfix: [40 lines describing branch naming, the approval
+  chain, the smoke suite, the changelog entry, and the rollback drill]
+- The staging database host is `stg-db-07.internal`.

Every line is true on the day it is written and every line was added for a reason someone can name. The reach test from chapter 1 rejects two of them outright, which settles where they do not go and says nothing about where they do go. "Elsewhere" turns out to be three different places, and one of them is the absence of a place.

The four destinations differ in when they load and how they fail#

An entry can live in the always-loaded config file, in a skill that loads when its description matches the request, in a script or tool the agent runs, or nowhere at all. Those are not four tiers of importance. They are four mechanisms with different loading behaviour, different costs when the task does not need them, and — the part that decides most arguments — different failure modes when the content goes stale.

The column that does the work is the fourth one. A stale line of prose in a config file is indistinguishable, to the agent, from a correct one; the model has no way to check whether the staging database host is stg-db-07.internal is still true, so it acts on it and produces a confidently wrong connection string. A stale script fails on the first run with a resolution error and somebody fixes it that afternoon. Preferring the destination that fails loudly is worth several hundred tokens of inconvenience.

Sorting an entry is a short sequence of questions, and the order matters because the later questions assume the earlier ones came back negative.

Two of those questions deserve their reputation for being contentious. "Does it have an expiry nothing will update" is contentious because expiry is invisible on the day of writing. "Could the agent break it on almost any task" is contentious because chapter 1 spent its length on it. The other two are mechanical and are where most of the volume goes.

Working the four lines#

The freeze rule belongs nowhere, and it is the hardest of the four to say out loud in a review because it was written in response to a genuine incident. The problem is not the rule's importance and not its wording; it is the date. Until 2026-03-14 is a fact with an expiry, and nothing in the repository will update it. Nobody's job description includes revisiting line 41 of CLAUDE.md on the fifteenth of March. What happens on the fifteenth is that the agent reads an authoritative instruction not to merge, refuses a merge it should have made, or — more commonly and more expensively — hedges: it opens the PR, adds a comment about the freeze, and waits for an approval from a release captain who stopped being one three weeks ago. The rule was correct for eleven days and wrong indefinitely thereafter.

The version that is not nowhere is a freeze rule that reads its own state. If the team's release tooling knows whether a freeze is active — a flag in a config service, a file the release script writes, a label on the milestone — then the agent can be given a way to ask, and the answer is never stale because it is not a copy. That is a script, and it is the same reclassification the ECONNREFUSED entry gets below. If the team has no such state anywhere, the honest conclusion is that the freeze is a Slack-message-and-human-vigilance control, and writing it into a file the agent reads does not make it more than that.

The billing invariant stays. It cleared the reach test in chapter 1 at 61 of 200 sessions, and it has the second property that matters: it does not decay, because it is a statement about the intended shape of the system rather than about the world's current state. Schemas get renamed and the sentence needs an edit when they do, but the rename is a change someone makes deliberately, in a migration, with a grep, and the entry will be found by that grep.

The 40-line hotfix procedure is a skill. It fails the reach test — the agent can only violate it while cutting a hotfix — and it is a procedure rather than a constraint, which is what skills are for. Chapter 3 is about the bill that comes with this move, so the important thing here is the classification: procedures that apply inside a named activity are conditional content and get a conditional loading mechanism.

The staging hostname stays, and it is the entry that most reviewers get wrong in the ungenerous direction. It is twelve tokens. It is a fact the agent cannot derive from the repository, because the value lives in a secrets manager and the repository contains only DB_HOST=${STAGING_DB_HOST}. Without it, an agent asked to reproduce a staging-only bug will either ask the human — costing a round trip of two or three minutes — or guess staging-db.internal and spend a turn on a DNS failure. It has the shape of a good config entry: short, specific, executable, and wrong in a way that surfaces immediately. Hostnames do rot, and this one will rot the day the team moves to stg-db-08; the reason that is tolerable is that the failure is a connection refused on the first command, not a subtly wrong decision made silently.

Execution logic in prose is a script that nobody can run#

The fifth candidate, added a week later by a different engineer, is the one that best shows what misclassification costs:

If the integration suite fails with ECONNREFUSED, start the docker compose stack, wait for the health endpoint, then re-run the failed test file only.

Read that as an instruction to a person and it is fine. Read it as an artifact and it has three defects. It is untested, so nobody knows whether the health endpoint is at /health or /healthz, and the agent will find out by trial. It is unversioned against the thing it describes, so when the compose file gains a second dependency the sentence does not change. And it costs its tokens on every session, including the 190 out of 200 that never run the integration suite. The corresponding script has none of those defects:

#!/usr/bin/env bash
# scripts/retry-integration.sh — bring up deps, then re-run one test file.
set -euo pipefail

TEST_FILE="${1:?usage: retry-integration.sh <test-file>}"

docker compose -f docker-compose.test.yml up -d --wait
# --wait blocks until every service with a healthcheck reports healthy,
# so there is no polling loop to get wrong. Default timeout is 60s.

pnpm vitest run "$TEST_FILE"

The script is nine lines against the entry's forty-one words, it runs in CI — the continuous-integration checks a repository runs automatically on every pull request — so a drift in the compose file breaks it visibly, and the config file's obligation shrinks to a single line naming it. That line is executable in chapter 1's sense — the agent can act on it without interpretation — and it survives every future change to the recovery procedure, because the procedure now lives in a file that changes.

The general form: when an entry describes a sequence of commands, the config file's job is to name the thing that runs them, not to describe them. Description in prose is the worst of both worlds, since it costs unconditional tokens and provides conditional, unverified capability.

Sprint goals and the phrase that admits anything#

A different pull request adds three lines, and it is the one worth rehearsing because the argument for it is more sophisticated:

1. The payments service must never call the ledger service synchronously.
2. Our current sprint goal is to reduce p99 latency on checkout.
3. To regenerate the GraphQL types, run pnpm gen:types from the repo root after any schema change.

The first is a config entry. It is an architectural invariant with wide reach across payments work, it does not expire, and it is checkable — a grep for the ledger client in the payments package, or a module-boundary lint rule, confirms it. It is also a good candidate for enforcement in CI rather than in prose, which is chapter 7's argument, and until that lint exists the entry is doing real work.

The third is also a config entry, and it is the one people are least confident about because it looks like a procedure. It is one command with a stated trigger, it is executable verbatim, and a schema change is something the agent does on ordinary feature work rather than inside a named workflow. If it is absent, the agent edits the .graphql file, the generated types go stale, and the type error surfaces three files away with no obvious cause. Better still, this is a hook: a post-change trigger that runs the generator without anyone needing to remember. Where a hook is available, the entry moves to the hook and the config keeps nothing.

The sprint goal belongs nowhere, and the author's defence is that all three entries help the agent understand the team's context. Situational awareness is the phrase that admits anything, because there is no candidate entry it excludes — every fact about a team is context in some sense, and a criterion that rejects nothing is not a criterion. The sprint goal is a fact with a two-week shelf life and no owner for its removal, which is the freeze date wearing different clothes.

It is also the entry that will cause a problem soonest, and the failure is specific enough to predict. Two sprints later the team has moved to a reliability push, and an agent asked to add a feature flag to the checkout handler reads our current sprint goal is to reduce p99 latency on checkout and starts making choices with that in mind: it inlines a call to avoid a hop, skips a validation it judges to be on the hot path, or proposes caching a response that must not be cached. When the reviewer asks why, the agent gives an excellent answer citing the file. That is worse than an agent with no context, because the wrongness is justified, documented, and traceable to a line the team wrote on purpose.

The sorting produced three destinations for the release runbook's cousins and a clean rejection for the two dated facts. What it did not produce is any confidence that the hotfix procedure, now living in a skill, will actually be there the next time somebody cuts a hotfix at two in the morning. Moving something out of the always-loaded file changes it from content that is unconditionally present to content that is present when a description matches, and that substitution has a failure mode nobody in the review is watching for.