Validate the call before the dispatch table sees it

A provider's reply to an agent loop arrives as a list of content blocks, and the interesting ones are the tool_use blocks: each carries an id the provider assigned, the name of a tool the model wants run, and an input object holding the arguments it wants that tool run with. The loop's job is to turn each of those into a call. Written the shortest way, that is a dictionary lookup and an argument unpacking — DISPATCH[call.name](**call.input) — and both halves of that expression are trusting something they have no reason to trust. The name came from a model that generates text and can generate a name that is nearly right. The arguments came from the same place, and "nearly right" for an object means a missing field, a value outside an enum, or a key nobody declared.

Three checks sit between the provider's response and your tool code, and none of them are in the loop as written so far. The name might not exist. The arguments might not parse. Arguments that do parse might still violate the schema you published — the input_schema on each tool declaration, written in JSON Schema, the vocabulary that states which fields a JSON object may carry, what type each one is, and which of them are required. Each of those is a fact the model can act on, which means each belongs in a result block against that call id rather than in a traceback.

What a missing gate looks like from the outside#

At 02:14 the worker pool started restarting. The traceback was two lines long and pointed at a dictionary lookup:

  File "agent/loop.py", line 61, in run
    output = DISPATCH[call.name](**call.input)
KeyError: 'serch_orders'

Read the line that raised rather than the exception's name, because the shape of that expression is the whole explanation. DISPATCH[call.name] is a subscript on a plain dictionary, and a plain dictionary's contract is that a missing key raises. The value on the model's side of that lookup is a string it produced token by token, and nothing between the two enforces that the string is one of the keys. So the failure is not an unusual event that happened to a correct program; it is the program's specified behaviour meeting an input its author assumed could not occur. The same is true one expression to the right: **call.input binds whatever keys arrived to whatever parameters the function declares, and a key the function does not declare raises TypeError from Python rather than from anything you wrote.

The reason that failure is expensive is that it happens in the wrong place. An exception raised inside DISPATCH[...] propagates up through the loop, and the loop is usually being run from a request handler that has no branch for it, so the process that dies takes the message list with it — and the message list is the only record of the run. Contrast the alternative for a moment: the same misspelling handled as a result block costs one step, and the model reads the correct spelling and calls again. The difference between those two outcomes is not the model's behaviour, which is identical in both. It is whether the loop had somewhere to put the answer.

Three gates, one shape of answer#

Every rejection path ends at the same place as a successful call: a result block carrying the call's id, so the pairing rule holds no matter which branch was taken. That is the structural reason to put the gates here rather than inside each tool. A gate that raises has to be caught somewhere, and the somewhere is a place where the call id is still in scope, which is precisely the function the gates live in.

That paragraph states a conclusion, and the chain under it is short enough to walk. The pairing rule set out two chapters ago — an assistant turn's tool calls oblige exactly one result block per call id in the message immediately following — makes no exception for calls that were never made. A rejected call produces no tool output, but it still occupies an id in the assistant turn the provider already sent you, and that turn is already in your message list. So the rejection has to be expressible as a result block, or the next request is malformed. A result block cannot be built without the call id. Therefore the check has to run somewhere the call id is still in hand, and the last such place before your tool code takes over is between the response and the dispatch table. The gates are where they are because of a rule established for an entirely different reason, which is the first of two places in this chapter where one property does two jobs.

from jsonschema import Draft202012Validator, ValidationError

VALIDATORS = {t["name"]: Draft202012Validator(t["input_schema"]) for t in TOOLS}


def check_call(call, tools: dict) -> str | None:
    """Return rejection text, or None if the call may proceed to dispatch."""
    if call.name not in tools:
        names = ", ".join(sorted(tools))
        return f"No tool named {call.name!r}. Available tools: {names}."

    if not isinstance(call.input, dict):
        return (
            f"Arguments for {call.name} were not a JSON object. "
            "Send an object whose keys match the declared parameters."
        )

    errors = sorted(VALIDATORS[call.name].iter_errors(call.input), key=str)
    if errors:
        e = errors[0]
        field = ".".join(str(p) for p in e.absolute_path) or "(root)"
        return (
            f"{call.name} rejected: {field} {e.message}. "
            f"Received {json.dumps(e.instance)[:120]}."
        )
    return None

The function returns text or nothing, and that signature is the design. check_call does not raise, does not dispatch and does not decide what to do about a bad call; it answers one question — is there something to tell the model — and leaves the caller holding the id it needs to say it. VALIDATORS is built once at module level from the same TOOLS list that goes into the request, so each entry is a compiled checker for one tool's declared schema. iter_errors walks the object against that schema and yields one object per violation; e.absolute_path is the sequence of keys leading to the offending field, joined here with dots so a nested field reads as filters.status, and e.instance is the value that was actually there. The (root) fallback covers a violation of the object as a whole, such as a missing required key, where there is no path to point at.

Reporting one error rather than all of them is a deliberate choice. A validator that walks a nested schema will happily produce eleven messages for a single malformed object, and the model corrects the first thing it reads; eleven messages cost tokens on every remaining step and improve the correction rate by nothing measurable. Truncating the echoed value at 120 characters is the same instinct applied to the other side — a rejection that quotes a 6 KB argument blob back at the model has turned a diagnostic into a payload.

The wording matters in the same way it did for tool failures. "No tool named 'serch_orders'. Available tools: get_shipment, issue_refund, lookup_policy, search_orders." gets a corrected call on the next step essentially always, because the correct spelling is now sitting in the context four tokens away from the mistake. A bare "unknown tool" gets a second guess, sometimes at a name that also does not exist. Keep the list bounded: it is fine at eight tools and a mistake at eighty, where naming the three closest matches by edit distance — the count of single-character insertions, deletions and substitutions that turn one name into another — is both cheaper and more useful.

That the correction rate is essentially total has a mechanical reason worth stating, since it is the same reason the previous chapter's error text had to name an alternative. The model's next choice is made from the message list, and the message list now contains the misspelling and the correct spelling within a few tokens of each other. Nothing has to be inferred and nothing has to be remembered. A bare "unknown tool" changes the message list too, but it adds no candidate, so the model's best available move is another guess drawn from the same place the first one came from — which is why the two rejection texts differ in outcome rather than merely in politeness.

What the numbers look like, and where the parse gate applies#

Across roughly 2,000 tool calls from a week of a production support agent with six tools declared, the gates fired like this: 28 unknown names (1.4%), 16 schema violations (0.8%), and no parse failures at all. That last figure is not luck, and it is the one that varies most by provider.

Those two rates share a denominator, which makes the gap between them a real quantity rather than an impression. Twelve more calls in the week named a tool that did not exist than filled a declared tool's schema wrongly — name errors ran to roughly two-thirds again as many as shape errors. That is worth knowing before you decide which gate to write first, and it points at something the raw rates do not: getting a name right is a free-recall problem over your whole inventory, whereas getting a shape right is a fill-in problem with the schema in front of the model. The harder task fails more often, and the harder task is the one you can help with by shortening or disambiguating your tool names.

Read that table down its third column rather than across its rows, and one variable explains every entry: whether the arguments crossed a boundary as text. Where the provider hands you an object it built itself, there is no text to be malformed and the parse gate has nothing to do. Where the arguments arrive as a string — an OpenAI-style function.arguments, or anything your own transport serialised and deserialised on the way — the string can be truncated or corrupted, and the parse gate is load-bearing. The fourth column has no such variation, and that is the point of putting the two columns side by side: parsing is a property of the transport, and conformance is a property of the model, so only one of them goes away when you change providers.

The parse gate on the Messages API is defensive rather than load-bearing, and that is worth saying plainly instead of implying every gate carries equal weight. Where it does bite is on providers that hand you a string: a tool call whose arguments run past the output token limit arrives as valid-looking JSON with the closing brace missing, and the correct rejection text says so — "arguments were truncated; call the tool with fewer items" — because the model's fix is to make a smaller request, not to re-emit the same one.

The schema gate is where the state of the art is moving. Strict tool use constrains generation against the declared schema rather than checking afterwards, which takes the 0.8% violation rate to approximately zero at the cost of a one-time schema compilation on the first request using it and a requirement that the schema stay within the supported subset — additionalProperties: false and an explicit required list, no recursive definitions, no numeric bounds. That last constraint is why the gate does not disappear: amount_cents with a minimum of 1 is exactly the kind of rule strict mode will not enforce for you, so a validation pass on the values you care about survives even when the shape is guaranteed.

That residual deserves to be named rather than tucked into a subordinate clause, because a reader who has just been told the violation rate goes to approximately zero will otherwise conclude that strict mode deletes the gate. What strict mode guarantees is that the object has the declared keys, with the declared types, and no others. What it does not guarantee is anything about the values inside those types, and every bound you care about lives there: a minimum, a maximum, a string pattern, a range on a date. So the class of violation strict mode removes is exactly the class a well-formed refund request would never have committed, and the class it leaves is the one that moves the wrong amount of money. The rate falls a great deal and the remaining rate is where the damage is.

One registry, so the schema and the function cannot drift#

The KeyError at 02:14 had a second cause behind the first one, and it is structural. TOOLS was a module-level list of schemas, DISPATCH was a module-level dictionary of functions, and nothing enforced that they described the same set. Somebody had added a tool to the dispatch table and forgotten the schema, and somebody else had renamed a schema without renaming the function; neither mistake produced an error until a specific conversation went a specific way.

A registry that owns both removes the class of bug rather than the instance:

REGISTRY: dict[str, Tool] = {}


def tool(schema: dict):
    def register(fn):
        REGISTRY[schema["name"]] = Tool(
            schema=schema, fn=fn, validator=Draft202012Validator(schema["input_schema"])
        )
        return fn
    return register


@tool({"name": "search_orders", "description": "...", "input_schema": {...}})
def search_orders(customer_id: str, status: str | None = None) -> str:
    ...


TOOL_SCHEMAS = [t.schema for t in REGISTRY.values()]   # what goes in the request

The request's tools array is now derived from the same object the dispatch reads, so a tool that exists in one and not the other is not expressible. Here a hand-built loop starts converging on what a framework's decorator gives you, which is worth noticing: the decorator is not magic, it is this, and having written it you know exactly which invariant it is protecting.

The registry also gives the gates somewhere sensible to live. check_call takes the registry rather than a bare dictionary, the validator is built once at import rather than per call — compiling a schema per call costs a few hundred microseconds and does it thousands of times a day for no reason — and the "available tools" list in a rejection is generated from the same source as the request, so it can never name a tool the model was not actually offered.

One caveat worth stating: deriving argument binding from the function signature via **call.input means a schema change that the function signature does not match raises TypeError inside the call rather than at the gate. The schema gate catches most of that, since a schema with additionalProperties: false and a matching required list will reject the unexpected key first, and the two staying in sync is exactly what the registry is for.

That caveat is the stumble in this chapter, and it is worth stating as a belief rather than as a footnote, because it is a belief a careful reader will arrive at on their own. Having written the three gates, the tempting conclusion is that **call.input is now safe. The quantity the gates hold constant is that every dispatched call is valid against the schema you published. The quantity varying underneath it, and unmeasured by anything in check_call, is whether that schema and the function's parameter list still describe the same set of parameters. A schema is data in one file and a signature is code in another, and nothing in Python compares them. So a gate that reports a hundred per cent conformance is telling you about the schema, not about the function, and the reason to build the registry is that it makes the two the same declaration rather than two declarations you have to keep equal by hand.

A rejection is a statement about the request#

The previous chapter's failures and this chapter's failures produce identical-looking result blocks, and conflating them makes both harder to fix. A tool that raised is a statement about the world: the billing service is down, the order does not exist, the policy forbids this refund. A rejected call is a statement about the request: the model asked for something that is not on the menu, or asked for it in a form the menu does not accept. The tool never executed, nothing changed anywhere, and there is no partial work to reason about.

They separate cleanly in the metrics, and they should. A rising tool-failure rate points at a dependency, and you page whoever owns it. A rising rejection rate points at your own surface: an ambiguous tool description, an enum whose values do not match the vocabulary the model naturally uses, two tools with overlapping names, a parameter that people call customer_id and the model calls customerId. Rejections are a design signal wearing the costume of a runtime error, and burying them in the same counter as timeouts is how the signal is lost.

Splitting the counter is a decision rather than a requirement, and the two are worth keeping apart. The requirement is the one derived above: every call gets exactly one result block, whatever happened to it, or the next request is malformed. Nothing in the protocol says the two kinds of failure have to be counted separately, and a loop that lumps them together is still correct. What it loses is the ability to route: one counter rising means find the owner of a dependency, the other means read your own tool declarations, and a single number that mixes them tells you to do neither in particular.

They also differ in what a good rejection contains. A tool failure describes a situation and suggests an alternative course of action. A rejection describes a contract and quotes the part of it that was broken. Writing "the billing service is unavailable, tell the user" style prose into a schema violation produces a model that apologises to the customer instead of fixing its arguments.

Rejections are one of the ways a run refuses to end#

Every rejection costs a full step: a model call, a decision, a result block, and another model call to try again. That makes the gates a place where a run can fail to terminate. A model that cannot satisfy a schema — because a required field needs a value it does not have, or because two constraints conflict, or because the enum simply lacks the value the task requires — will produce rejection, correction, rejection, correction until the iteration cap fires, and each cycle grows the message list and the cost.

The bound is cheap to add and worth adding at the same time as the gates:

consecutive_rejections: dict[str, int] = defaultdict(int)

text = check_call(call, tools)
if text is not None:
    consecutive_rejections[call.name] += 1
    if consecutive_rejections[call.name] >= 3:
        raise Unrecoverable(
            f"{call.name} was rejected 3 times in a row; last: {text}"
        )
else:
    consecutive_rejections[call.name] = 0

Three is a reasonable default because the useful corrections happen on the first retry: in the same week's data, 24 of the 28 unknown-name rejections were followed by a valid call on the very next step, three by a second rejection then a valid call, and one by a run that spent its remaining six steps alternating between two names that both failed. That last run is the one the counter exists for, and terminating it deliberately at step four with a diagnosable reason is strictly better than terminating it at step ten with none.

Those three counts partition the week's unknown-name rejections exactly, with nothing left over: twenty-four plus three plus one is the whole set. That is what turns a threshold from taste into a derived number. Twenty-seven of the twenty-eight rejections resolved themselves within two consecutive failures, so a counter that gives up on the third fires on the one run that never recovered and on none of the runs that did. Set it at two and it would have killed the three that needed a second attempt; set it at four or five and it buys nothing, since no run in the week recovered after a third failure. The number is not a guess about how patient to be, it is a reading of where the distribution ends, and it is the sort of number you can only get by counting rejections separately in the first place.

The counter is keyed on the tool name and reset on success, which is what makes "consecutive" mean what it should. A model that fails to name a tool, succeeds at something else, and fails again is exploring; the pathological run is the one that fails at the same tool with no intervening success. Resetting on success is therefore not a nicety, it is the difference between a detector for a stuck run and a quota on how many mistakes a long run is allowed to make in total.

Rejection loops are one of four distinct reasons a run does not terminate, and this is the only one of the four whose signature is visible at the gate itself. The others — repeated identical calls, transient failures, and a missing stop signal — show up in the trace rather than in a counter, and telling them apart is the subject of the last chapter of this topic.

The loop now has three ways to keep going: the model asked for tools that ran, the model asked for tools that failed, the model asked for tools that were rejected. All three append results and go round again, and by the argument in the previous chapter that is exactly why they can go round again — each of the three puts something new in front of the model. What has been treated so far as a single line — if not calls: return — is the only way the loop stops on its own, and that line has been carrying an assumption nobody has examined: that a turn without tool calls means the model is finished. It is worth pricing that assumption against the alternative before the following chapters start reading and diagnosing runs against it.

The rule to take from this chapter is that a call is not a call until it has been checked, and that checking it is cheap in the one place where the answer is still expressible. Look the name up, confirm the arguments are an object, validate them against the schema you already published, and turn any of those failures into a result block against the call's id, with a sentence naming the contract and the part of it that broke. Generate the schemas and the dispatch table from one declaration so the two cannot drift, count rejections apart from tool failures because they send you to different places, and bound the consecutive ones so a model that cannot satisfy a schema stops the run at a diagnosable point rather than at the cap.