Tool calling

Read these first

• Schema-constrained generation

What you will know

• Declare a tool, have the model call it, and return the result

  The application executes the request and feeds a structured result back into the conversation; the model never runs your implementation directly.

• Handle parallel tool calls in one turn

  Independent calls should be dispatched concurrently, while preserving each call's identifier and isolating failures in the returned results.

• Force a specific tool when the task requires it

  A forced choice can prevent omission, but it also bypasses the model's ability to decline when inputs are incomplete or the operation is unsafe.

• Inspect the request payload to verify what the model actually sees of a tool

  Only the exposed name, description, parameter schema, and surrounding instructions influence selection; internal code, types, and comments do not.

• Debug a tool that is never selected

  Trace the emitted tool choice and compare it with the schema and prompt for ambiguous intent, overlapping tools, invalid required fields, or a mode that disables calling.

Chapters

1. 1

   Function calling

   Function calling is the protocol by which a model asks your program to run one of your functions: it emits a block naming the function and filling in its arguments, your code executes it, and the return value goes back into the conversation tagged with the call's identifier. The model's contribution is the request and nothing else, so a handler that branches only on text leaves the call unanswered — and a turn that used one tool bills about 4,090 input tokens against 1,940 for a tool-free one, because the whole conversation is resent.

   Read chapter
2. 2

   Tool definitions

   Each entry in the tools array is three fields — a name, a natural-language description of when to use the function and when not to, and a JSON Schema for its parameters — and that envelope is everything the model has to choose from and fill in. An unconstrained string field is a guess about your conventions, which is how 912 create_ticket calls arrived carrying eleven distinct priorities; a field in required is one the model must put some value in on every single call. Which parts you can edit once sessions are live is a separate question, and renaming is the one that surprises people.

   Read chapter
3. 3

   Reading the serialised tool payload

   Selection is decided by the tool name, description, parameter schema and surrounding instructions that actually appear in the request body, never by your function's types, internal logic or comments. An SDK, a schema derivation and a framework sit between the two, and none of them promises to be faithful — one derivation truncates a docstring at the first blank line, another puts a defaulted parameter in required. Printing the serialised payload from a production code path takes ninety seconds and turns 'the model ignores our tool' into a question with an answer.

   Read chapter
4. 4

   Diagnosing tool selection failures

   With the payload confirmed correct, a tool passed over on auto has a short list of causes. The commonest is a neighbouring tool whose description claims general user questions, leaving no stated boundary — and the edit that moves the line goes into the broad tool, not the skipped one. The second is a required argument the conversation cannot supply, which produces rare selection and plausible-looking invented values together; over 240 calls a required idempotency_key matched a real key not once. That one is a schema decision, and every boundary edit has to be scored on a labelled replay set covering every tool.

   Read chapter
5. 5

   Parallel tool calls

   One assistant turn can carry several tool-use blocks, and dispatching them one at a time turns four lookups into the sum of their latencies rather than the longest of them. Independent reads run concurrently; two writes with an order dependency do not, which is a property of your tools rather than of the turn. Every call the model made needs a result tagged with its identifier, including the one that returned 503, and a write that timed out is unknown rather than failed — saying so changes what the model does next but cannot stop a second charge, which is decided by an idempotency key recorded server-side.

   Read chapter
6. 6

   Forced tool choice

   The tool choice field is best read by what it takes away from the model rather than by what it guarantees you. Forcing `lookup_order` on every turn ends stale answers and removes the model's ability to say the inputs are not there: on a greeting or a complaint it must still emit arguments, so a required id gets an invented value. In one week 2,517 of 4,180 turns contained no order reference at all and every one produced a call. Conditional forcing, an explicit decline tool, and a freshness check in the dispatcher keep the guarantee without the invention.

   Read chapter
7. 7

   The tool-calling loop

   Returning a result does not end the exchange: it goes into a fresh request, the model reads it, and it may ask for more, which you execute and return the same way. The condition to test is whether this response contains a tool-use block, evaluated every time — and each pass resends everything, so a five-iteration answer cost 11,220 input tokens against the 1,640 a tool-free reply would have used. The turn ends when the model answers without emitting a call, and that is the only stopping condition the protocol provides.

   Read chapter
8. 8

   Bounding the tool-calling loop

   A system-prompt instruction lowers the probability of another call per iteration and bounds nothing, and the component being asked to obey it is the one that cannot count its own iterations. The ceiling belongs in the loop, which is the only place that can observe the count and act on it unconditionally — and the iteration that trips it still has to return a result for every outstanding call and then make one final request with tools switched off, so the user gets an answer rather than a 504. An iteration cap and a wall-clock deadline bind different resources and a loop that goes more than a couple of rounds wants both.

   Read chapter