The tool-definition block

The tool block is the serialised set of tool definitions — names, descriptions and JSON Schemas — that an agent sends on every call so the model knows what it may invoke. It is a fixed cost in the strict sense: the same tokens, in the same order, whether the request needs a tool or none, and it is paid before the user's question, the retrieved evidence and the conversation have been considered at all.

Being identical on every request is exactly what hides it. When an answer is bad, the artefacts an engineer opens are the ones that differ between the good case and the bad case: the query, the retrieved chunks, the session history. A component that is the same on the request that worked and the request that failed cannot explain the difference between them, so it never becomes a suspect — while quietly setting how much room every other slice has to share.

The block nobody looks at#

Breaking a real registry down per tool makes the shape of the tax visible:

Two tools that had never once been called were costing 1,330 tokens on every request for a month — export_audit_bundle at 690 plus reconcile_billing_rows at 640 — which across 128,000 requests is 170 million input tokens paid for capability that was exercised zero times. The multiplication is the whole argument for reading this table at all: a per-request figure of 1,330 tokens reads as nothing, and it is the same number as 170 million once it is priced against how often the request happens. A fixed cost is small per unit and large per month by definition, which is why fixed costs are the ones that survive review. They were registered because the agent "might need to" export an audit bundle, a judgement made in a planning document and never revisited against usage.

Those per-tool numbers have to come from the counting endpoint rather than from tokenising the schema files, because what the model sees is the serialised form the SDK produces rather than the JSON on disk. The measurement is two calls and a subtraction:

BASE = [{"role": "user", "content": "."}]

def tool_block_tokens(tools: list[dict]) -> int:
    with_tools = client.messages.count_tokens(
        model=MODEL, system=SYSTEM, tools=tools, messages=BASE).input_tokens
    without_tools = client.messages.count_tokens(
        model=MODEL, system=SYSTEM, tools=[], messages=BASE).input_tokens
    return with_tools - without_tools

def per_tool(tools: list[dict]) -> dict[str, int]:
    return {t["name"]: tool_block_tokens([t]) for t in tools}

Both functions work by difference, and per_tool calls tool_block_tokens once per tool with a list containing exactly one entry, so each figure answers "how much larger does the request get when this one schema is present". Sum those twenty-three figures and the total is 6,640 tokens, against 6,800 for the block measured whole — so the parts come to less than the whole, and the shortfall is 160 tokens. Nothing there is measurement error. The 160 tokens are the envelope the provider wraps around the tool list, structure that belongs to the list rather than to any tool in it, so no measurement of a single tool can contain it and no amount of care with the individual schemas will produce it.

The direction of that gap is the part to remember, because it is the dangerous one. A per-tool table quietly undercounts the assembled block, which means every argument built on it — this tool only costs 505 tokens, we can afford one more — is short by a fixed amount that never appears in any row. The rule that applies to message framing applies here too: the remainder belongs to a named line rather than to a rounding error. The same function belongs in CI, where it costs one API call per build:

def test_tool_block_within_ceiling():
    measured = tool_block_tokens(TOOLS)
    assert measured <= 3_200, f"tool block {measured} exceeds its 3,200-token slice"

The individual schemas are the other half. create_incident at 612 tokens looked like this:

{
  "name": "create_incident",
  "description": "Creates a new incident record in the incident management system. Use this tool when the user reports a production problem that needs to be tracked formally, or when you have determined through investigation that an issue warrants an incident record. Do not use this tool for questions, for informational requests, or when the user is merely speculating about a possible issue. Before calling this tool you should confirm with the user that they want an incident created, unless they have explicitly asked for one. The incident will be assigned to the on-call rotation for the specified service.",
  "input_schema": {
    "type": "object",
    "properties": {
      "title": { "type": "string", "description": "A short title for the incident, ideally under 80 characters, describing the observed symptom rather than the suspected cause." },
      "severity": { "type": "string", "enum": ["sev1", "sev2", "sev3", "sev4"], "description": "The severity of the incident. sev1 means total outage affecting all customers. sev2 means major degradation affecting many customers. sev3 means partial degradation affecting some customers. sev4 means minor issue with no customer impact." },
      "service": { "type": "string", "description": "The name of the affected service, which must match a service registered in the service catalogue." }
    },
    "required": ["title", "severity", "service"]
  }
}

Rewritten to 148 tokens, with the usage guidance moved into the system prompt where it is stated once rather than once per tool:

{
  "name": "create_incident",
  "description": "Create a tracked incident record. Confirm with the user first unless they asked.",
  "input_schema": {
    "type": "object",
    "properties": {
      "title": { "type": "string", "description": "Observed symptom, under 80 chars" },
      "severity": { "type": "string", "enum": ["sev1", "sev2", "sev3", "sev4"] },
      "service": { "type": "string", "description": "Name from the service catalogue" }
    },
    "required": ["title", "severity", "service"]
  }
}

The severity descriptions are the interesting deletion, and the saving is worth counting because the obvious count is wrong. Four other tools took a severity parameter and each carried the same four sentences defining sev1 through sev4, which with create_incident makes five copies of 90 tokens travelling in every request. Moving the definitions into the system block does not delete all five. It leaves one, in the system prompt, where the guidance is stated exactly as fully as before — so the saving is the four copies that went, which is 360 tokens, and not the whole of what the five copies weighed between them. De-duplication saves n minus one copies, never n, and a plan that budgets for n is a plan that will come in short.

The general shape underneath is that a per-tool description is charged per tool while a system-prompt statement is charged once, so anything true of several tools is in the wrong place if it lives in their schemas. Enum values often carry their own meaning well enough that prose restating them is pure overhead; where it is not — where sev2 genuinely needs a definition — one statement in the system prompt serves every tool that uses it.

The options, and what each one costs#

The first three are the ones to do first, because they carry no architectural change and the risk is measurable on the eval set you already have. Trimming descriptions is the only one with a genuine quality trade: descriptions exist to help the model choose the right tool, so cutting them can raise the rate at which the model picks search_runbooks when it should have picked query_metrics. Running a 240-case tool-selection eval before and after the rewrite above gave 94.2 percent correct selection before and 94.6 percent after, which is inside the noise band, meaning the range a measurement of this size can move by chance alone — and the direction is not an accident, since a smaller, sharper set of descriptions is easier to discriminate between than nineteen paragraphs that all read alike.

Per-request subsets are where the sizeable recovery lives and where the second-order costs start. Routing tools by intent — metrics and deployment tools for a diagnosis query, incident and workload tools for a remediation query — cut the block from 6,800 to about 2,800 tokens on a typical request. The cost is that the tool block is the stable prefix of the request, so varying it per request invalidates the prompt cache for everything that follows it, and on this workload the cache had been carrying 8,000 tokens of tools and system instructions at a tenth of the input price.

That last figure turns into a per-request cost in one step. Those 8,000 tokens were being charged at a tenth of the base input rate and are now charged in full, so the increase is nine tenths of what 8,000 tokens cost at that rate, which on this model's pricing is the roughly $0.021 above. Set it against what the routing recovered, which is 4,000 tokens of tool block: the tokens saved are half as many as the tokens whose discount was lost, so the saving is worth about half of what the loss costs and the change is a net increase in spend. That is why per-request subsets can be a net loss on a workload with a warm cache and a clear win on one without, and why the token column alone never settles it. Losing the cache also raised time-to-first-token by about 180 ms, because a cached prefix is a prefix the model does not have to read. Clustering the routing into three fixed tool sets rather than a bespoke set per request kept the hit rate at 78 percent, because sessions tend to stay within one intent, and recovered most of the tokens.

A long registry is also a selection problem#

Space is the cost that shows up in the budget, and it is not the only thing a long registry does to a request. Running the same 240-case eval against progressively smaller registries separates the two effects, because the routed subsets and the full registry contain the same schemas written the same way:

The token column and the accuracy column move together and for unrelated reasons. Choosing among five options is a shorter menu than choosing among twenty-three, and the twenty-three contained search_runbooks, search_incidents and search_knowledge_base, three entries whose descriptions differed by a clause and whose correct usage differed by more than that. A model that picks search_incidents for a runbook question spends a round trip, returns nothing useful, and either recovers with a second call or answers from what it has, which is where the "needing a second attempt" column comes from and why a routing change pays twice: the block shrinks and the choice gets easier.

The differences worth acting on need a size, and the size comes from the eval rather than from the table. On a 240-case eval a proportion near 0.95 carries a standard error of about 1.4 points — the amount the figure would be expected to wobble if you graded a fresh sample of the same size — which follows from the proportion and the sample size and from nothing about tools. The gap between 94.6 and 95.9 is a little over a point, which is smaller than one such wobble and is not evidence of anything. The gap between 91.4 and 94.6 is more than three points, which is over two wobbles, and a difference of that size arising by chance is unlikely enough to act on. The defensible reading is that the audit was worth doing and that routing to five tools buys tokens rather than accuracy — 2,840 down to 1,180 is 1,660 tokens recovered — which is still a good trade when the accuracy column is telling you nothing either way.

Routing needs an escape hatch, because a subset chosen from the user's first message is a prediction and predictions are wrong. The version that shipped exposes the intent subset plus a single request_full_toolset tool costing 40 tokens, which the model calls when nothing in front of it fits; the next turn carries the whole registry. That fired on 3.1 percent of requests, added about 700 ms to those, and turned the failure mode of a bad route from a missing capability into a slower answer.

The registry grows without touching your repository#

A tool block audited to 2,840 tokens in February measured 7,100 tokens in April, and the diff that caused it was a configuration change adding a Model Context Protocol (MCP) server for an incident management vendor, MCP being the standard interface through which an outside service advertises its tools to an agent. That server advertised fourteen tools, all of them registered automatically, several of them overlapping tools the agent already had, and none of them reviewed by anyone who had read the budget. No pull request in the agent's own repository touched a schema.

This is why the CI assertion above is written against the assembled registry rather than against a list of files. A tool registry that accepts entries from external servers is an input to the budget in the same way retrieval configuration is, and the assertion is what makes the day it changes a build failure with a name attached rather than a quiet reduction in how many chunks reach the model. Where the external tools are genuinely wanted, the honest response is to re-run the audit with them included and decide which slice pays for them, which is the conversation the assertion forces.

Caching lowers the price, not the space#

The tax is not something prompt caching removes, and the reason is what a cache entry contains. What is stored is the model's computed state for those tokens — the work of reading them, done once and reused — rather than a compressing of the context. The tokens themselves are still in the request, still at the same positions, still counted against the window. Caching skips the reading. It does not shorten the thing being read. That is why a cached tool block costs about a tenth as much and takes up exactly as much room, and confusing the two is how a team concludes that a large fixed block is free: the invoice improves, the window does not, and only one of those two was the constraint that was hurting. The space cost is invariant, which is why the question "does this tool justify its schema on every request" is worth asking even when the money is negligible.

Cutting the tool block also does not improve retrieval. The chunks that arrive are still ranked by the same retriever, and freeing space raises how many of them reach the model without changing which ones the ranker chose. A coverage gain from an audit is a gain from removing a constraint, not evidence that the retrieval configuration was ever right.

And a per-tool table is a description of what tools cost, not a judgement about which are worth it. export_audit_bundle at 690 tokens with zero calls in thirty days is an easy deletion; restart_workload at 505 tokens with 96 calls is not obviously a good or a bad trade, and nothing in the measurement decides it. What the measurement does is put the question in front of someone, priced, at a moment when the answer can still be changed.

Every slice now has a measured size and a defensible way to shrink it. What none of that supplies is the answer a design review actually asks for, which is why retrieval gets 14,000 tokens rather than 10,000, and what happens to the product if it gets 10,000 instead.