Hidden author intent is the defect a reviewer cannot see

The run under review produced a migration that added plan_tier to invoices as NOT NULL DEFAULT 'free', in one statement, on a table with 41 million rows. The db-migrations skill had loaded — the trace shows its body in context — and the two engineers looking at it could not agree on whether the agent had followed it. One said no, because the skill says columns go in NULL-able and this one did not. The other said the skill's exact words were "prefer conservative migrations and be careful with large tables", that Postgres 14 does not rewrite the table for a constant default so the statement took 40 milliseconds, and that being careful is precisely what the agent had done. Both readings survive contact with the document. The author, who was on holiday, meant the first one.

That disagreement is not about the migration. It is about a skill whose expected behaviour lives in the author's head, which means no run of it can be called correct or incorrect by anyone else, which means it cannot be reviewed, which means it will drift for months without anybody able to say when it started going wrong.

The test is what a stranger can predict before the run#

The property to write for is narrow and checkable: hand the skill to somebody on the team who did not write it, give them a request it fires on, and ask them to write down what the agent will do differently from an unaided run. If they can produce a list, and their list matches what the author would have written, the skill is reviewable. If they produce "it will be more careful about migrations", it is not, and the gap will surface later as an argument nobody can settle.

Reviewers of ordinary code have this property without thinking about it, because the function either returns the right value or it does not. A skill has no return value; its output is a change in behaviour, and behaviour is only observable against an expectation stated in advance. Acceptance criteria on a ticket are the closer analogy — "the export completes for accounts with more than 10,000 rows" is checkable, "the export should handle large accounts gracefully" generates a two-week argument in the review column.

Applied to the sentence that started this chapter, the exercise is quick and unkind. "Prefer conservative migrations" does not name an action, so nothing the agent produces can fail it. "Be careful with large tables" names neither the threshold at which a table is large nor what careful means when it is. "Consider whether a rollback is needed" leaves both the decision and the criterion with the model, which is the opposite of supplying local convention: the model already has an opinion about rollbacks and the skill has added nothing to it.

Name the action, the sequence, and the exception#

Three things have to be present before behaviour becomes predictable, and vague skills are usually missing all three at once.

The action is the thing the agent does that it would not otherwise do, stated as an instruction with an object: write the rollback file under db/rollback/ with the same timestamp prefix as the migration, rather than make sure migrations are reversible. The sequence is where the action sits relative to everything else, because "write a rollback" before generating the SQL and "write a rollback" after applying it are different procedures with different failure modes. The exception is the condition under which the action does not apply and what happens instead, which is the part that turns an instruction into something the agent can follow without judgement — and, more to the point, the part that lets a reviewer tell a legitimate deviation from a violation.

The right-hand column is longer, and the length is not decoration; each addition removes a place where two readers could diverge. The row about row counts is the one people skip, because it feels obvious that the agent will find the number somehow. It will — SELECT count(*) on a 41-million-row table, which takes about eight seconds and a full sequential scan, where reltuples is a catalogue lookup that is close enough for a threshold decision.

What a reviewable body looks like#

Here is the section of db-migrations that produced the argument, rewritten so that the expected behaviour can be stated by someone who has never spoken to the author.

## Adding a column

1. Add the column NULL-able and without a default, in its own migration file.
2. If the column needs a value for existing rows, write a second migration
   containing only the backfill. Do not combine the two.
3. Get the row count from `pg_class.reltuples` for the table. If it is above
   1,000,000, the backfill batches at 10,000 rows with a 200 ms pause; write
   the batching loop into the migration, do not rely on a manual run.
4. Write the rollback file under `db/rollback/` with the same timestamp prefix
   before you propose the migration.
5. Only after all four exist, propose the change and list the files you wrote.

Exception: if the column is on a table under 10,000 rows and takes no default,
steps 2 and 3 do not apply — say in your summary that you skipped batching and
why.

Stop and ask instead of proceeding if: the change drops a column, alters a
column's type, or renames anything. These need a data-team review and this
skill does not cover them.

The rewritten section is roughly the same length as the vague one it replaced, which is worth noticing because the usual objection to precision is that it bloats the document. What made the original short was that it left decisions out, and every decision it left out was one a reviewer would later have to litigate.

Against this version, the run at the top of the chapter is unambiguously a violation: step 1 says NULL-able and without a default, the agent shipped NOT NULL DEFAULT 'free', and there is no exception that covers a 41-million-row table. The engineer who defended the run had a real technical point about Postgres 14, and the place to make it is a pull request against the skill, not a review of a run.

Expected behaviour is a thing you write down, once, next to the skill#

The other half of reviewability is that somebody records the prediction. A short block in the repository next to SKILL.md costs ten minutes and turns every future review into a comparison rather than a reconstruction.

## Expected behaviour — db-migrations

On a request to add a column, a run with this skill loaded differs from an
unaided run in five observable ways:

1. Two migration files instead of one, when a backfill is needed.
2. The column is NULL-able in the first file, always.
3. A rollback file exists under db/rollback/ before the change is proposed.
4. Batching appears in the backfill iff reltuples > 1,000,000.
5. The summary names every file written.

A run that drops or retypes a column ends in a refusal naming the statement.

Five lines, each one a thing you can grep a transcript for. This is also what makes a change to the skill legible: if someone edits step 3 to allow a rollback written after the fact, the expected-behaviour block changes in the same commit, and the review is about whether the team wants that rather than about whether anyone noticed.

The review that would have caught this on the day it shipped#

The original db-migrations body passed review in about four minutes, because it read well and everything in it was true, and reading well is what a reviewer of prose is trained to check. The questions that would have caught the defect are not about the writing at all.

Ask, for each numbered step, what a transcript would look like if the agent had skipped it. If the answer is "the same, more or less", the step is not doing anything. Ask which sentence the run at the top of this chapter violates; if two people give different answers, the document has a fork in it. Ask what the agent does with a request the document does not cover, and if there is no answer in the file, the answer is whatever the model would have done unaided, which is worth knowing before it happens rather than after. Ask, finally, who wrote the expected-behaviour block and whether they were the author — a block written by the author records what they meant, and a block written by a second reader records what the document actually says, and the interesting information is in the difference between them.

That last exercise is cheap and unusually effective. On the incident-response skill from the previous chapter, the author's block said the agent would page the secondary on-call if the primary did not acknowledge within five minutes; the reader's block said nothing about the secondary, because the document mentioned the escalation path in a bulleted list of definitions and never said to walk it. Neither person was wrong about what they had read. The five-minute rule existed only in the author's head, and it stayed there through two incidents before anybody noticed the agent had never escalated anything.

Precision has a real cost, and the escape hatch is part of the design#

A body specified this tightly fails in a way the vague one did not. The vague version applied, badly, to everything; the precise version applies exactly to the cases it enumerated, and the first request that falls outside them puts the agent in a bind. Someone asks for a partial unique index on a table that is 900,000 rows today and will cross the threshold next month. Step 3 says batching applies above a million and this is below, the spirit of the rule says batch anyway, and a model following instructions literally will not batch.

The repair is not to loosen the language back toward "be careful", because that returns the whole document to unreviewable. It is to say what to do when the procedure does not cover the case:

If the request does not match any procedure above, do not improvise a policy.
Follow the general rules (NULL-able first, backfill separate, rollback file)
and say explicitly in your summary which part of this skill you were extending
and why, so it can be added.

That paragraph does two jobs. It keeps behaviour predictable at the edges, and it makes the skill's own gaps show up in transcripts, where they can be closed. Gaps discovered this way are worth more than gaps imagined in a planning meeting, because each one arrived attached to a request somebody actually made.

There is a limit to how far this goes. A skill specified to the point where every branch is enumerated has become a program written in English, and at that point the honest move is to write it as a program — which is the subject of the next chapter, and also a reason not to push precision past the point where a human can read the document. The working threshold is behavioural rather than syntactic: specify until a stranger's prediction matches the author's, and stop.

Verifiable is not the same as enforced, and not the same as deterministic#

Two adjacent ideas get attached to this and neither is what the chapter is about. The first is enforcement. A precisely written skill has exactly as much power over the agent as a vague one — the model reads it and complies or does not, which 12.1 established about every text-based mechanism and the first chapter of this topic re-drew around skills specifically. What precision buys is that non-compliance becomes visible. Before the rewrite, the agent ignoring the skill and the agent interpreting the skill generously were indistinguishable in a transcript. After it, they are different events, and only one of them is worth a bug report.

The second is determinism of output. Two runs of the rewritten skill on the same request will not produce identical SQL; the column ordering will differ, the comment wording will differ, one run will name the rollback file with an underscore and the next with a hyphen. That is fine. The property being asked for is that the decisions are predictable — NULL-able or not, one file or two, batched or not, refused or proceeded — while the prose around them stays free. A skill that tries to pin the output text as well is fighting the model to no purpose, and it is the surest way to produce a document nobody wants to maintain.

Which leaves the case where the check itself is the problem. Step 3 asks the agent to read reltuples, compare it to a threshold, and write a batching loop with the right numbers — three operations that a six-line Python script does identically every time and that a model does approximately, especially at the end of a long session. The team already has that script sitting in scripts/check_backfill.py, bundled with the skill since the day it shipped, and the traces show the agent has never once run it.