Tool definitions and request framing

A request that a careful count scored at 31,520 tokens can be rejected by the API at 33,410. Nothing in that gap is a tokenizer disagreement or a rounding error: it is material that genuinely forms part of the request and that a concatenation over message content has no way to see. A tool definition — the declaration of one function the model is allowed to ask for, carrying its name, a description of when to use it, and a schema for its arguments — is usually the bulk of it, and the rest is framing: a few tokens wrapped around each message, plus the separators the provider puts between the conversation and the system block, which is the standing instruction text that sits ahead of every request as its own segment.

This chapter enumerates what a content-only count misses and prices each part, starting from the serialized payload, because seeing the structure once makes the omissions obvious.

The payload is a structure, and a concatenation reads one field of it#

What leaves your process is a JSON document. The strings you assembled sit inside it, and so does everything else the model is going to be shown:

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1200,
  "system": [
    { "type": "text", "text": "You are a billing support agent for Acme..." }
  ],
  "tools": [
    {
      "name": "search_invoices",
      "description": "Search the customer's invoice history. Use this when the customer asks about a charge, a refund, or a payment that they cannot find. Returns at most 25 invoices, newest first. Do not use this for subscription changes.",
      "input_schema": {
        "type": "object",
        "properties": {
          "customer_id": { "type": "string", "description": "The Acme customer identifier, e.g. cus_8812f. Take this from the session, never from the customer's message." },
          "from_date":   { "type": "string", "format": "date", "description": "Inclusive lower bound, ISO-8601. Omit to search the last 12 months." },
          "status":      { "type": "string", "enum": ["paid", "open", "void", "uncollectible"], "description": "Filter by invoice status. Omit for all statuses." }
        },
        "required": ["customer_id"]
      }
    }
  ],
  "tool_choice": { "type": "auto" },
  "messages": [
    { "role": "user", "content": "my card was charged twice on the 3rd" },
    { "role": "assistant", "content": [
        { "type": "tool_use", "id": "toolu_01A", "name": "search_invoices",
          "input": { "customer_id": "cus_8812f", "from_date": "2026-05-01" } } ] },
    { "role": "user", "content": [
        { "type": "tool_result", "tool_use_id": "toolu_01A",
          "content": "[{\"id\":\"in_9921\",\"amount\":4200,\"status\":\"paid\"}, ...]" } ] }
  ]
}

A concatenation over messages[*].content picks up the user text, the tool result string and nothing else. It misses the whole tools array, the system array, the tool_use block's name and arguments, the role labels, and the delimiters the provider inserts when it renders this structure into the flat sequence the model actually reads. That rendering is not something you can see, and its size is stable enough to measure and account for.

That single search_invoices definition costs 187 tokens: 4 for the name, 51 for the description, and 132 for the schema, which is written in JSON Schema — a vocabulary for describing the shape of a JSON value, so that input_schema above states in machine-readable form that customer_id is a required string and status may only take one of four values. The property descriptions inside it are 78 of the 132, and the enum values, the type keywords and the JSON punctuation make up the rest. Six tools of that shape is 1,100 tokens, and six tools with richer descriptions — which is what happens after two rounds of prompt tuning where somebody adds "do not use this for X" to fix a misfire — is comfortably 1,600 to 2,800.

Splitting the schema's 132 in two is what says which parts are negotiable. Descriptions take 78 of it, which leaves fifty-four for the type keywords, the enum values and the JSON punctuation. That fifty-four is a floor rather than a cost: it is what expressing the same constraints in the same vocabulary weighs, and no rewording gets underneath it. Everything above the floor is prose, which is why the description lines are the only place an edit has anything to work with.

Tool definitions are the largest thing a content count never saw#

The reason tool definitions dominate the invisible portion of a request is that they are prose written in a JSON container. Every clause of guidance about when to call a tool, every example identifier in a property description, and every enum value is text that gets serialized and counted, and unlike your messages it is present on every call in the session whether or not the model uses any of it. An agent that makes 40 model calls in one task pays for its tool definitions 40 times.

Why it has to be present on every call is worth deriving rather than accepting, because the rest of this chapter follows from the answer. A model call carries no memory of any call before it: the model is handed one sequence of text and returns a continuation of it, and nothing it was shown last time is available to it now. So a capability the model may reach for on this call has to be described inside this call. That makes the tool block content rather than configuration: it is resent in full every time.

Configuration is the word to resist there, and it is tempting for three good reasons. The tools array is built once at start-up, it usually lives in a different module from the one that assembles messages, and it is passed to the SDK as a named parameter rather than appended to a list. All three are true of things that get registered once and referred to afterwards. None of them changes what happens at the boundary, where the array is serialized into the same flat sequence as your messages and counted by the same tokenizer.

The composition is worth breaking down once, because it tells you which edits actually reduce the number:

Cutting a property description from 40 tokens to 15 saves 25 tokens on every request for the life of the deployment; renaming customer_identifier to cid saves 2 and costs the model its ability to guess what the field means. The productive edits are removing tools the agent does not need for this task, collapsing three near-identical tools into one with a mode parameter, and rewriting descriptions that grew by accretion. A support agent that ships with 14 tools because "they might come up" is spending 3,500 tokens per call on capability it uses in 4 percent of sessions.

The rule underneath those two examples is visible in the table itself. Two of its rows are written in English — the tool description and the property descriptions — and on a tool with several properties those two outweigh every other row put together. So the edits that move the number are the ones that remove or shorten prose, and the edits that feel like optimisation are the ones that shorten identifiers, because identifiers are what an engineer instinctively economises on. Shortening one saves two or three tokens once and deletes the only account of what the field means that the model will ever be given, which is a bad trade at any price.

The fourteen-tool agent prices out the same way once it is divided through. Fourteen tools costing 3,500 tokens per call is about 250 tokens apiece, and capability reached for in 4 percent of sessions is capability the other ninety-six percent of sessions carry for nothing. The saving there is not in making those definitions terser. It is in not sending the ones this task cannot use.

Attaching a different set of tools per phase is the structural version of the same saving, and it is available more often than teams assume. An agent that plans, then edits, then verifies does not need the editing tools during planning or the search tools during verification, and swapping the tools array between phases takes the per-call block from 3,500 tokens to about 900. What it costs is the cache: changing the tool block changes the prefix, so the first call of each phase is a cache miss on everything that follows it, which is why the swap pays off when phases are long and hurts when the agent alternates between two tool sets every other call.

That cache interaction is worth stating as a separate ledger, because it points the two costs in opposite directions. Tool definitions sit at the very front of the rendered prompt, ahead of the system block and the conversation, which makes them the most cacheable region of the entire request — expensive in window space on every call, and cheap in money after the first. Trimming schemas to save money is therefore often the wrong target. Trimming them to buy window space is a real gain, because no cache gives you tokens back.

Underneath that ledger is a single mechanism, and it also settles the phase-swapping trade above. A prefix cache can reuse only a prefix that is byte-identical to one it has already seen, so material at the front of the rendered prompt is reusable on every call that leaves it alone, and everything behind a change is not reusable at all. Swapping the tool array between phases changes that front, invalidates the cached prefix, and forces the first call of the new phase to rebuild everything after it. One rebuild spread over many calls is cheap; a rebuild on nearly every call is not.

Everything else the provider counts#

Beyond the tools, the parts of a request that a content-only count misses are each small, and there are enough of them to matter at a boundary. Message framing costs roughly 3 to 5 tokens per message on most providers, covering the role marker and the delimiters between turns; on a 74-message transcript that is 220 to 370 tokens, which is invisible in a 200,000-token window and decisive in an 8,000-token one. The system block is its own segment with its own framing and, on some APIs, its own structural preamble. A tool_use block re-serializes the tool name and the arguments the model produced, so an assistant turn that called three tools with substantial arguments can be 400 tokens even though it contains no visible prose.

The per-message figure is worth measuring rather than taking on faith, because it differs between providers and between model generations, and it is the one component you can pin down exactly in about a minute. Count a request with one short message, count the same request with that message duplicated, and the difference minus the message's own content is the framing:

def framing_cost(client, model, unit={"role": "user", "content": "ok"}):
    one = client.messages.count_tokens(model=model, messages=[unit]).input_tokens
    two = client.messages.count_tokens(model=model, messages=[unit, {"role": "assistant", "content": "ok"}]).input_tokens
    body = client.messages.count_tokens(model=model, messages=[{"role": "user", "content": "ok ok"}]).input_tokens
    return one, two - one, body        # (11, 8, 12) on one 2026 model

The first number includes a fixed per-request preamble that exists whether you send one message or two hundred, and the second is the marginal cost of a turn. Knowing both means the framing term in a budget is fixed + per_turn × len(messages) with real constants in it rather than a guessed round number, and it means you can predict what a compaction pass will recover before you run it: dropping twelve short turns from a transcript recovers their content plus roughly a hundred tokens of framing, which is sometimes the whole of the shortfall you were trying to close.

Structured output — the arrangement where you hand the provider a schema and require the model's reply to conform to it — has the same shape as tool definitions and gets overlooked more often, because the schema lives in a response_format field that reads like configuration rather than content. A response schema with 12 fields and descriptions is 250 to 500 tokens on every call. Images are the largest per-item cost in the list: a 1024×1024 screenshot is on the order of 1,100 to 1,600 tokens depending on how the model tiles it, and an agent that takes a screenshot after each of ten actions has spent 14,000 tokens on evidence. Collected, the components a content concatenation misses look like this:

The discipline that follows from the table is to count from the request object rather than from your strings: build the exact dictionary you are about to pass to the SDK, then measure that. The provider's token-counting endpoints accept the same shape as the create call for precisely this reason — model, system, tools and messages all go in, and what comes back is the number the create call will compute.

counted = client.messages.count_tokens(
    model=req["model"],
    system=req["system"],
    tools=req["tools"],
    messages=req["messages"],
)
print(counted.input_tokens)   # 33,410

Sampling parameters and retries are not part of the input count#

Two things that feel like they belong in the table above are not counted at all. Sampling parameters — temperature, top_p, stop sequences, the max_tokens value itself — are request metadata and cost nothing, so there is no point economizing on them. Stop sequences in particular look like text and are not counted as input.

Retries and continuations are the other thing this list does not cover, because they multiply the count rather than add to it. A request that is rejected and resent unchanged is counted twice by your rate limiter and once against nothing at all, and an agent that recovers from a malformed tool argument by asking the model again is paying for the whole prompt a second time to fix forty tokens of output. Those are budget problems in the accounting sense, and they do not change what any single request weighs.

Nor does this chapter explain a mismatch in the other direction. If a careful count of the full payload comes out above what the provider reports, the difference is usually the tokenizer rather than the structure: a local library implementing a slightly different vocabulary, or the same vocabulary at a different revision. That is a separate error with a separate remedy, and it is the reason two reputable libraries can hand you different numbers for the same 30,000-token prompt.

A request is a structure, and its weight is the weight of the structure. Tool definitions are the part of that structure most likely to be both large and unaccounted for, because they are written like documentation, live in a different module from the message assembly, and are resent on every call in a session regardless of use; framing, system blocks, output schemas and images make up the rest, each small enough to ignore individually and collectively worth a few hundred to a few thousand tokens. All of it becomes visible the moment the thing being counted is the request object rather than a string.

Counting the full payload tells you that you are over, by how much, and — if you have counted the components rather than the total — which part of the request is responsible, which is what makes the eviction decisions later in this topic tractable rather than arbitrary. What it leaves open is accuracy: whether the number you just computed locally is close enough to the provider's that a packer can decide a boundary against it, and what a fill target of 90 percent is actually asserting when somebody writes it down.