A prompt is resident every turn; a skill loads when it matches

The system prompt for the internal engineering agent was 11,400 tokens when someone finally printed it. It had started, eighteen months earlier, at about 400: repository layout, the test command, a note about not touching generated files. Then the release-notes format went in, because that was the cheapest fix available on the day. Then the incident-report template. Then a paragraph from the security team about not writing credentials into config files, and a longer one from the data team about which columns in the events table are PII, and a description of the four deployment environments, and a list of the twelve services with their owners. Every addition was individually justified and none of them was ever removed, which is the normal life cycle of a file that anyone can append to and nobody is on the hook for.

Here is what 11,400 resident tokens cost in a single session. The agent ran a median of 34 model calls per session across the previous month. Prompt caching covered most of it — the provider keeps an unchanged leading block of context on its side between calls, so re-sending it bills at a reduced cache-read rate rather than at full input price — but the block is read on every one of those 34 turns and it occupies its space in the context window on every one of them:

resident system prompt      11,400 tokens
turns per session (median)     34
tokens read per session   387,600
sessions per weekday          ~90
tokens read per weekday  34,884,000

At a cache-read price of $0.30 per million input tokens that is about $10.50 a day, or roughly $2,700 a year, to carry a block of text through every conversation. The money is survivable. What is not survivable is the second number: of those 11,400 tokens, the fraction relevant to any given session was, by inspection of 40 sampled transcripts, between 3% and 22%. A session fixing a flaky test read the PII column list, the release-notes format, the incident template and the environment descriptions on all 34 of its turns, and used none of them.

Residency is the property, not size#

The distinction between a system prompt and a skill is not that one is small and the other is large. It is when the text is present. A system prompt is resident: it is in the context on turn 1 and on turn 34, whether the session is about release notes or about a segfault. A skill's body is not present until something decides the request matches it, at which point it is read into context and stays for the rest of the session. What is resident for a skill is only its description — typically 20 to 40 tokens — sitting in a list of available skills alongside every other one.

That difference changes what each mechanism is for, in a way that survives every argument about token prices. Text belongs in the system prompt when every session must obey it regardless of what the session is about. The security team's line about credentials qualifies: there is no task where writing a secret into a config file becomes acceptable, so the rule has to be present when the model does not know it needs it. The release-notes format does not qualify, because a session that is not writing release notes cannot violate it.

The inverse test is the useful one. For each block of resident text, ask what fraction of sessions could plausibly get the wrong answer without it. If the answer is above roughly half, residency is buying you something. If it is under a tenth, you are paying for the text in every session to be correct in a tenth of them, and you are also paying a cost that does not appear in any token count, which is that the model has more instructions to weigh on every turn. Instructions compete. A model reading eleven paragraphs of standing rules attends to each one less reliably than a model reading three, and that degradation shows up long before the context window is anywhere near full.

The last row is the one people skip. Neither mechanism has any way of knowing that what it says stopped being accurate, and neither produces an error when it is wrong. Both are read as true and acted on. That property is shared, which is precisely why it cannot be the axis you choose between them on, and it needs its own treatment against mechanisms that do error — which is chapter 5.

The description is the router, and it is code you have to test#

Moving a block out of the prompt and into a skill converts a residency cost into a routing decision, and the routing decision is made by matching the request against the skill's description. That description is the entire interface. The body can be perfect and it will never be read if the description does not match how people actually phrase the request.

We measured this directly on the release-notes format from chapter 1. The first version of the skill described itself the way its author thought about it:

---
name: release-notes-format
description: Company release-notes formatting standard.
---

Against the same 200 recorded requests used to test the tool, it loaded on 118 of them, or 59%. The misses were not exotic. Requests phrased as "write the changelog for 4.2", "draft the version announcement", "summarise what shipped this sprint for the release email" did not match a description that only says "release notes". Rewriting the description to name the situations rather than the artefact changed the number substantially:

---
name: release-notes-format
description: The company's release-notes format — section order, ticket-ID
  column, no marketing adjectives. Use when writing, editing or reviewing
  release notes, changelogs, version announcements or "what shipped" summaries,
  including drafts for email or Slack.
---

That version loaded on 191 of 200, or 95.5%. It also fired on 4 requests where it was not wanted — someone asking for a customer-facing launch blog post got the internal format's adjective ban applied to marketing copy, which is a genuinely wrong outcome rather than a harmless one. Broadening a description trades misses for false fires, on the same curve as any other classifier, and the right operating point depends on which error is worse for the particular procedure. For a formatting convention, over-firing is cheap and easy to override in conversation. For a skill that says "always deploy through the canary pipeline first", over-firing into a local development task is annoying and under-firing is an incident.

The comparison to the tool from chapter 1 is worth making numerically, since both mechanisms were proposed for the identical requirement:

The prompt and the revised skill are within noise of each other on correctness, which is the expected result, because they are the same text and the model is doing the same thing with it. What separates them is that one of them is also present in the 74% of sessions that never touch release notes. That is the whole argument, and it is an argument about the portfolio rather than about this one capability: any single block is affordable to make resident, and the eleventh one is what turns an 400-token prompt into an 11,400-token one.

What this replaced, and what the evidence actually shows#

Skills exist because the alternative that came before them stopped scaling. The pattern in 2023 and 2024 was a single instruction file that every project accumulated — the agent's whole standing knowledge in one place, read on every turn. It worked at 500 tokens and degraded steadily past a few thousand, in the way described above. Loading bodies on demand behind a short description is a direct response to that, and it is the same idea as lazy loading in any other system: keep an index resident, keep the payload out until something asks.

The evidence on whether this actually wins is more interesting than the story suggests. Vercel published an evaluation comparing a plain instruction file against a skill-based setup on their own agent workloads and found the flat configuration file outperformed skills on their tasks — reported in "AGENTS.md outperforms skills in our agent evals". The mechanism behind that result is the routing risk made concrete: on a workload where nearly every task needs nearly all of the standing knowledge, on-demand loading can only lose, because it introduces a decision that can be wrong in exchange for saving tokens you needed anyway. Their agent's tasks were homogeneous. The engineering agent above is heterogeneous, with sessions ranging over test failures, deploys, data questions and release notes, and heterogeneity is what makes residency wasteful.

So the axis is not "prompts are old and skills are new". It is the shape of your session mix. A narrow agent doing one kind of work should carry its knowledge resident and skip the routing problem entirely. A broad agent doing eight kinds of work pays eight times over for residency and should be routing, accepting that some percentage of the time it will route wrong.

Two columns of the matrix, and three still missing#

What the last two chapters have produced is the beginning of a decision matrix, and it is worth being explicit about which columns exist so far. Chapter 1 gave the capability boundary column: which of the three gaps this is, and therefore which mechanisms are even eligible. This chapter has given the cost column, and shown that it is not a single number — a resident cost paid per turn is a different kind of thing from a per-invocation cost, and the mechanisms cannot be ranked on cost without saying which one you mean.

Three columns remain. Latency is the easy one to measure and the easy one to over-weight. Owner is the column that separates a tool from an MCP server, and it is the one that does not appear in any benchmark, which is where the next chapter goes: two implementations of the same Jira integration, one of which fails inside your deployment while the other is a dependency that someone else has to keep running, patch, version and answer the pager for at 02:40.