Forced tool choice

The tool_choice field decides whether the model may pick a tool, must pick some tool, must pick one particular tool, or may not pick any. It is one line of configuration, it is exact about what it does, and it is reached for whenever a selection rate in the nineties is not enough — a compliance rule about grounding, a product requirement that a quoted delivery date always comes from a live lookup, a router turn where every branch ends in a call.

The useful way to read the field is by what it takes away from the model rather than by what it guarantees you. Under auto, a model that has no order number in front of it has three sensible moves: answer directly, ask which order the customer means, or call a different tool. Naming a tool deletes all three, and what fills the gap is the decoder doing what a constrained decoder does, which is produce a value for every field the schema declares required.

What each mode takes away#

{
  "model": "claude-sonnet-4-5",
  "tools": [ { "name": "lookup_order", "...": "..." } ],
  "tool_choice": { "type": "tool", "name": "lookup_order" },
  "messages": [ { "role": "user", "content": "hi again, thanks for sorting yesterday's mess out" } ]
}

The middle two rows are where the damage lives. The constrained decoder is now emitting a tool_use block for lookup_order whatever the conversation says, and the schema says order_id is a required string, so the sampling process fills that field the way it fills any required field with no source in the input. This is the same mechanism as the fabricated idempotency_key in chapter 4, where a required field's value lived in the dispatcher's session rather than in anything the model could read, arriving through a different door: there the field had no source because it lived in your infrastructure, and here it has no source because the customer said hello.

It is worth following that fill through to see why the invented value looks so convincing. The decoder is choosing tokens that continue the request plausibly, and the material it has for judging plausibility is the field name, the field's description, any pattern on it, and whatever order ids appear elsewhere in the conversation. None of that material includes your orders table, so "plausible" here means shaped like the ids I have seen and cannot mean present in the database. An id built from the right prefix and the right number of digits scores well on the only criterion available. That is why the failure does not announce itself: the value is not garbage, it is a competent imitation, and it will pass every check you have that tests form rather than existence.

The same reasoning explains the collisions further down this chapter. A fabricated id drawn from a narrow format — two letters, a dash, five digits — is drawn from a small space, and a small space with real orders scattered through it produces hits by chance alone. So fabrications that look like your data land on your data occasionally, and the rate at which that happens is set by how constrained the format is rather than by anything the model did.

What follows is worse than a wasted call. The lookup returns nothing, that empty result goes back into the conversation, and the model now has to say something about an order the customer never mentioned. Sometimes it apologises for being unable to find AB-00001, which is confusing. Sometimes, with a plausible-looking id in the history and a customer who is plainly talking about a previous order, it reconciles the two and quotes a delivery date.

How often a conversational turn has nothing to look up#

The rate at which this fires is a property of the traffic rather than of the model, and it is a query you can run before changing the setting. On a support assistant forcing lookup_order on every turn, a week of logs held 4,180 turns, of which 2,517 contained no order reference of any kind — greetings, thank-yous, complaints about tone, questions about the returns window, and follow-ups referring to something already on screen. Every one of those turns produced a lookup_order call. Of those calls, 1,904 carried ids matching nothing at all, 604 reused an id from earlier in the same conversation, and 9 collided with real orders belonging to other customers, which returned nothing only because the lookup is scoped by the session's customer.

Two sums are worth doing on those figures before reading any further. Dividing 2,517 by 4,180 gives a fraction just over three in five, which is the 60% used later in this chapter and is a fact about support traffic rather than about models: most of what people type at an assistant is not a query with an identifier in it. And the three call categories add back to 2,517 exactly, so they partition the affected turns rather than sampling them — every forced call on a turn with nothing to look up fell into one of the three, and none of them was a call that happened to be right. There is no fourth column of accidental successes, which is the column people expect to find.

The middle category is the one that quietly does damage, and it is easy to read as harmless. Reusing an order id from earlier in the conversation produces a lookup that succeeds, returns real data about a real order belonging to this customer, and puts that data in front of a model that then has to write a sentence about it. A failed lookup at least leaves the model with nothing to work from. A successful lookup of the wrong thing is indistinguishable, from inside the conversation, from a lookup the customer asked for.

That last figure is the one to sit with. Nine turns in a week produced a well-formed identifier for somebody else's order, and the only thing standing between the fabrication and a data leak was a scoping clause in a SQL query written for an unrelated reason. Forcing did not create that risk on its own; it raised the number of attempts from the fraction of turns where the model chose to look something up to every turn there is.

The belief worth dismantling here is that the scoping clause is the defence, and that with it in place the collisions are a curiosity. Scoping by the session's customer is a real control and it is doing real work — nine attempts hit and nine attempts were refused. What it defends is one query in one code path. Every other place a fabricated order id can travel is outside its reach: a second tool that takes an order id and was written by a different team, an analytics event, a support ticket the assistant opens quoting the id, a webhook payload. The control is narrow because it lives in a SQL predicate rather than in the schema, and a control that lives in one caller protects one caller. Reducing the number of fabricated identifiers in circulation is a different intervention from filtering them at one destination, and only the first of the two scales with the number of destinations.

The guarantee you wanted, without the invention#

Forcing is not the only way to stop a model answering from memory, and it is the only one on the list below that operates by removing the model's ability to say no. Three alternatives keep the guarantee and place it somewhere the model cannot route around.

The first is to force on the turn where the conditions are actually known. Nothing requires tool_choice to be constant across a conversation — it is a per-request field, so your code can set it from state your code owns. If the current user message contains something matching your order-id pattern, or a prior turn established one, force the lookup on that request and hand back to auto on the next.

def choose_tool(user_text: str, session: Session) -> dict:
    if ORDER_ID_RE.search(user_text) or session.active_order_id:
        return {"type": "tool", "name": "lookup_order"}
    return {"type": "auto"}

The second is to keep the force and give the model somewhere to put a refusal. If every turn must produce a call, add a tool whose entire job is to say that no lookup is warranted, and use any rather than naming one tool:

{
  "name": "no_lookup_needed",
  "description": "Call this when the customer's message does not concern a specific order — greetings, thanks, general policy questions, or complaints. Give the reason so the conversation can continue normally.",
  "input_schema": {
    "type": "object",
    "properties": { "reason": { "type": "string" } },
    "required": ["reason"]
  }
}

The dispatcher treats a call to no_lookup_needed as a no-op and returns an acknowledgement, and the model has recovered the ability to decline inside a regime where it must still emit something. This buys the audit property people usually want from forcing — every turn has an explicit, logged decision about whether a lookup happened — without requiring an order id to exist. It costs about 40 tokens of definition and one extra content block per turn.

Why that works when relaxing the schema does not is the same argument seen from the other side. Under any the constraint is emit some call, and that constraint is satisfiable by a call whose arguments all have sources: the reason string is generated from the conversation, which the model has in front of it. Under a named tool the constraint is emit this call, and satisfying it demands an order id, which the conversation may not contain. The decline tool does not weaken the guarantee that every turn produces a logged decision. It supplies a second way of satisfying the guarantee that does not pass through a field with no source. That is the general move whenever forcing is unavoidable: leave the constraint alone and widen the set of things that can satisfy it.

The third is to stop asking the model for the guarantee at all and put it in the dispatcher. If the requirement is that no response quoting a delivery date may be based on stale data, then the enforcement point is the code that assembles the answer: cache the last lookup with a timestamp, and if the model produces order-specific claims with no lookup inside the freshness window, re-run the turn with the fresh data attached to the context. This is more work than a one-line configuration change and it is the only option in the set that is actually a guarantee, because it does not depend on what the model chose.

Where forcing genuinely belongs#

None of this makes the field a mistake. Forcing is the correct instrument when the turn is not a conversation, and the clean cases share a shape: your code, not the model, has already decided that this operation must happen now, and the inputs are known to be present because your code assembled them.

Extraction is the standard example. You have a support email, you want it as an object with intent, order_id and sentiment, and you send one request with one tool and tool_choice naming it. There is no turn on which declining is meaningful, the input is in the message you just put there, and the forced call is doing the job of schema-constrained output with a tool-shaped API. A classification pre-pass over a fixed input is the same case. So is the first step of a pipeline where the model's contribution is to fill in parameters for an operation the user has already confirmed through your UI.

The test that separates these from a conversational assistant is whether the turn's inputs are guaranteed by construction. In the extraction case the email is right there in the request. In a support assistant the order id is guaranteed by nothing, because a conversation is a stream of arbitrary human messages and 60% of them were not about orders.

The same failure with a plausible trigger#

A banking assistant that forces get_balance on every turn is the same argument with different vocabulary. The case in its favour — that on a banking assistant every turn should check the balance anyway — describes a session in which the user asks about their balance repeatedly and nothing else. Real sessions contain "hello", "what were the fees on that", "can you repeat that in euros", "I want to close my account", and a complaint about the app crashing. On each of those the model must still emit a call, so an account number gets chosen when none was named, balances get fetched for accounts nobody asked about, and the model then has to write a sentence explaining a tool result the customer did not request. Conditional forcing on turns that mention a balance, plus a system-prompt rule that balances come from the tool, gets the same protection on the turns where it means anything.

The refund version has a narrower-sounding trigger and is not narrower in the way that matters. Forcing refund_order whenever the user's message mentions a refund sounds like a targeted rule; mentioning a refund supplies no order id. "Do I even qualify for a refund on digital purchases?" mentions a refund and identifies nothing, and under that rule it becomes a forced call with an invented order_id, against a tool that moves money. Compared with a rarely-selected refund tool, which is a conversion problem, a wrongly-triggered one is a financial incident, so the asymmetry runs the opposite way from the intuition that forcing is the cautious option.

That intuition deserves a fair hearing, because it is not silly. Everywhere else in engineering, removing a degree of freedom from a component makes the system easier to reason about: a constant is safer than a variable, a fixed route is safer than a dynamic one, and a step that always happens cannot be forgotten. Forcing looks like that move. What breaks the pattern is that the degree of freedom being removed is not whether to look something up. It is whether the inputs for looking something up are present, and that judgement was the only part of the turn where the absence of data was going to be noticed at all. Removing a component's ability to report that it cannot proceed does not make the system more predictable. It makes the failure mode change from a visible refusal to an invisible guess.

Two other limits are worth knowing before the field looks like a fix for anything. It cannot express a conditional, because it is evaluated by your client before the model has read anything, so any condition has to be computed by your code — which is what conditional forcing is. And on providers that support it, forcing tends to suppress the model's text content on that turn, so the friendly sentence that used to precede a lookup disappears and the user sees a longer silence.

A forced call, in the end, is not a correct call. The field constrains which tool appears in the response and constrains the arguments only to the shape of the schema, so everything in chapter 4, on why a tool is rarely chosen and badly argued, still applies: values with no source in the conversation are still invented, and a forced tool with an unsatisfiable required field fabricates on 100% of turns rather than on the fraction where it happened to be selected. Forcing raises selection to certainty and leaves argument quality exactly where it was.

The turns in this chapter all ended after one lookup, which made them easy to reason about. Once a result comes back that changes what the model wants to know — an order whose shipment is missing, a charge whose status is unknown — the model reads it and asks for something else, and the exchange stops being a round trip and becomes an iteration.