Classifying a tool-schema edit

Teams normally ask a merged version of those two questions — "is this change safe?" — and the merged question resolves to whichever half is more salient at the time. That is why a library upgrade reads as safe and a description reword reads as risky, when the cache consequences run the other way. Chapter 2 gave you the offset map, which prices an edit once you have decided to make it. This chapter is about the classification that comes first, because roughly half the edits a team queues against a tool registry turn out to sit in a cell nobody had identified.

The two questions have a structural relationship worth stating, because it makes one quadrant impossible. The model has no access to your service, your registry or your intent; the only thing it receives is the serialised prefix. If those bytes are identical, the model's view of the contract is identical, which means a change cannot be model-visible without also being byte-visible.

Four cells, one of which cannot exist#

The top-left cell is the only genuinely free change available to you, and it is larger than it looks. Fixing a bug in what a search tool actually queries, adding a tenant scope to the SQL behind it, tightening a numeric bound the handler enforces, swapping the downstream service it calls, changing its timeout from 8 seconds to 3 — none of those touch the serialised schema, so the prefix is byte-identical and the cache is untouched, and none of them change what the model believes it can call, so no re-planning happens. Everything else in a tool definition exists precisely to be read by the model, which is why that cell is the only one of its kind.

Membership in that cell has a mechanical test, and it is worth spelling the chain out because it is the same fact answering both questions at once. The model receives the serialised request and nothing else. A handler is not in the serialised request, so no change to a handler can be model-visible. The cache compares the serialised request and nothing else, so no change to a handler can be byte-visible either. One property — that the request is the entire channel — produces both halves of the cell, which is why a change lands in it or does not, and never lands in it for one question and outside it for the other. The test to run is therefore not "is this a big change" but "does this alter the document I serialise", and that is a question a diff answers rather than a reviewer.

A worked instance makes its size concrete. A logistics support agent shipped a change to search_shipments that scoped its query to the caller's tenant, dropped the downstream timeout from 8 seconds to 3, and capped the result set at 50 rows so that a warehouse with 4,000 open shipments could not blow the context window. Three changes, one of them a security fix, all of them things a reviewer would flag as significant, and the serialised tool definition came out byte-identical because none of them are expressible in a JSON Schema that already said {"type": "string"} for a shipment query. Hit rate across that deploy stayed at 96% with no visible step, and the offset map in the build artefact reported no movement, which is the correct reading rather than a missed detection.

What the deploy did change was the content of tool results, since a capped query returns fewer rows and a tenant-scoped one returns different rows. That content is not part of the shared prefix — tool results live in the message history, which is per conversation and was never reused across users — so the fleet-wide cache is genuinely untouched. The cost of the change showed up instead as a small drop in output tokens and a measurable improvement in answer quality on multi-tenant accounts, which is the kind of result that gets attributed to nothing in particular because the deploy carried no cache signal to hang it on. There is a caveat that is easy to miss in a multi-turn agent: those results land in the message history, which becomes part of the prefix on every subsequent turn of the same conversation, so the change is not free with respect to a single long conversation's own accumulated prefix if you deploy mid-conversation and something replays an earlier turn. That is a narrow case and a real one.

The top-right cell is where accidental invalidation lives. The changes in it feel like nothing because they are nothing to every tool a backend engineer normally reasons with — a formatter, a linter, a library upgrade, a rewording — and they are total to a prefix matcher. Two tool definitions can describe the same tool, accept the same arguments under any validator, and read identically to a model, while diverging as byte sequences a few dozen characters in:

// before: hand-rolled
{"name":"get_order","description":"Fetch one order by id.",
 "input_schema":{"type":"object","required":["order_id"],
  "properties":{"order_id":{"type":"string"}}}}

// after: pydantic v2, same callable interface
{"name":"get_order","description":"Fetch one order by id.",
 "input_schema":{"type":"object",
  "properties":{"order_id":{"title":"Order Id","type":"string"}},
  "required":["order_id"]}}

The reason a reordering counts at all is that nothing between your registry and the provider's matcher ever builds an object. A JSON document has unordered keys, which is a statement about what a parser is required to do with it, and every consumer in the chain honours that — your tests, any validator, the model's reading of the schema. The matcher is not a consumer of the document. It walks a token sequence produced by serialising it, and a serialiser has to emit the keys in some order, so it commits an arbitrary choice to bytes on the way past. "Key order does not matter" is true of everything that reads the schema and false of the one thing that prices it, and the same goes for indentation, separator spacing and a field a library adds for its own convenience.

Both documents describe the same tool. Any validator would accept the same arguments against either, and a model reading either one would form the same understanding of what get_order takes. They diverge 62 characters in, which is all the cache is being asked about. The bottom-right cell is the honest work: a real interface change, which costs a cold window and should, and which the rest of this topic is about paying for deliberately.

The empty cell is worth a sentence of its own, because knowing it is empty removes a question you would otherwise have to keep asking. There is no such thing as a change the model notices and the bytes do not, so a deploy that leaves the rendered prefix identical cannot have changed tool selection, argument shape, or anything else the model decides — whatever else it broke, it did not break that. That makes the byte comparison the cheaper of the two tests and the one to run first: render the tool block before and after, diff it, and if it is identical you are in the left column and only the second question remains.

The top-right cell is also the one you can land in without writing a line, which is the property that makes it worth engineering against rather than watching for. If the bytes you send are produced by a library, then your prefix is a function of your dependency tree, and a dependency tree changes on somebody else's release schedule. A patch bump to a schema-generation library is, in cache terms, an edit to the front of every request in your fleet, made by a person who has never seen your service and merged by a bot at three in the morning. Nothing in the pull request will say so, because from the library's point of view nothing happened: it emits a document that means what it always meant.

Decoupling your bytes from your libraries#

The fix for the top-right cell goes beyond "be careful", and it is to stop letting a third-party library decide the byte-level shape of something that participates in a cache key. The tool schema you send is an artefact you control; generating it from your models is a convenience, and canonicalising the result before it goes on the wire — rewriting it into one fixed key order and spacing, so that two logically identical schemas always serialise to identical bytes — costs a dozen lines and removes an entire class of overnight invalidation.

import json
from collections import OrderedDict

SCHEMA_KEY_ORDER = ("type", "properties", "required", "enum", "description")

def canonicalise(node):
    """Stable key order, no library-invented fields, no incidental whitespace."""
    if isinstance(node, dict):
        node = {k: v for k, v in node.items() if k != "title"}
        ordered = OrderedDict()
        for key in SCHEMA_KEY_ORDER:
            if key in node:
                ordered[key] = canonicalise(node[key])
        for key in sorted(k for k in node if k not in SCHEMA_KEY_ORDER):
            ordered[key] = canonicalise(node[key])
        return ordered
    if isinstance(node, list):
        return [canonicalise(x) for x in node]
    return node

def to_wire(tool) -> str:
    return json.dumps(canonicalise(tool.schema()), separators=(",", ":"))

Under this, a schema-generation library swap produces byte-identical output to the hand-rolled dictionaries it replaces, the deploy costs nothing, and the pinned fingerprint from topic 2.15, on prompt caching mechanics, stays green because there was genuinely no change to the thing the provider matches on. Stripping title is a judgement — it is a field some teams want the model to see — and the point is that it becomes a decision made once, in your code, rather than a decision made for you by a minor version bump. Sorting the keys you did not anticipate, rather than dropping them, means a library that starts emitting something new lands in a deterministic position instead of wherever its iteration order puts it.

Canonicalisation is not free of risk in one direction: introducing it is itself a byte change, so the deploy that adds this function invalidates everything once. That is the correct trade, since you pay one cold window to stop paying an unpredictable one every time a dependency moves, and chapter 7 covers scheduling that window.

Cache cost against correctness cost#

A failure mode of the two-question framework is using it as a cache-first filter, where an edit that changes bytes gets rejected on cache grounds without anyone asking whether the model needed the change. That inverts the priorities. A tool description that causes the model to pass a shipment id where an order id belongs is producing wrong answers on every call, and the cost of leaving it wrong is unbounded while the cost of fixing it is one cache window, quantified and short. The framework exists to tell you which cell you are in so you can price the change, and a change in the bottom-right cell that fixes a correctness problem is a change you make.

The reverse failure is treating a top-right change as automatically worth making because it is only cosmetic. Rewording a description without changing its meaning still moves tokens, and tokens are what the model conditions on, so "semantically identical" is a claim about your reading rather than about the model's. Topic 5.11, on tool descriptions functioning as few-shot examples, covered how much selection turns on phrasing; a reword that a human calls equivalent can shift the boundary between two similar tools. A change in that cell therefore carries a definite cache cost and an uncertain behavioural benefit, which is the worst risk profile of the four cells, and it is the one that people ship on Friday afternoons because it feels safe.

The classification is per edit and the cost is per deploy, which is the other thing it is easy to get backwards. Five queued edits that all land in the top-right and bottom-right cells cost one cold window between them if they ship together and five if they do not, so the question the framework answers is not "may this edit ship" but "does this edit have to travel alone". A correctness fix in the bottom-right cell travels alone because waiting costs wrong answers; a reword in the top-right cell has no such claim and rides with whatever else is queued, which is the whole basis of the batching policy chapter 7 arrives at.

Both questions are asked about a request you are about to build, and they classify an edit at the moment of deploy. Neither of them says anything about the requests already in flight. A conversation that started an hour ago carries a history full of tool calls written against the schema you are replacing, and those calls come back to you on every subsequent turn regardless of which cell your edit fell into.

That is a different mechanism from prefix matching, with a different time constant. Prefix matching has a five-minute memory and forgets your edit as soon as the caches rewarm; a stored conversation remembers the old schema for as long as the conversation lives, which is what makes an interface change stay expensive long after the cost of the cold window has been paid.