Tool definitions

A tool definition is the object you put in the tools array of a chat request to tell a model that one of your functions exists. It is small — three fields — and it is the only thing the model ever sees of the function behind it. Your type annotations, your validation logic, the comment explaining why the second parameter exists and the wiki page describing your triage conventions are all on your side of the boundary; what travels is a name, a paragraph of English and a schema.

That makes the definition an unusually high-leverage piece of text. It decides whether the model reaches for this function or for a neighbouring one, what it puts in the arguments, and whether it asks the customer a question first, and every later chapter in this topic is an edit to it.

The envelope that travels beside the messages#

Each entry in the array is three fields, and there is nothing else in it. A name, which is the identifier the model will emit and the key your dispatcher will switch on. A description in natural language, which is the only place you can say anything about when this function is appropriate. And a JSON Schema for the parameters — JSON Schema being the standard vocabulary for describing the shape of a JSON object, its field names, their types and which of them must be present — which describes the object that comes back and marks each field required or optional.

{
  "name": "create_ticket",
  "description": "Open a support ticket for an issue that cannot be resolved in this conversation, such as a suspected billing error, a delivery that is lost rather than late, or anything requiring a human to contact a supplier. Do not use this to answer a question you can answer from lookup_order or search_docs, and do not use it to chase an existing ticket — use get_ticket_status for that. Ask the customer for a one-line summary before calling if they have not already described the problem.",
  "input_schema": {
    "type": "object",
    "properties": {
      "summary": {
        "type": "string",
        "description": "One sentence, in the customer's own words where possible, describing what went wrong."
      },
      "priority": {
        "type": "string",
        "enum": ["low", "normal", "high", "urgent"],
        "description": "urgent = money has left the customer's account with nothing to show for it, or a safety issue. high = the customer is blocked and has already contacted us once about this. Default to normal."
      },
      "order_id": {
        "type": "string",
        "description": "The order this concerns, in the form AB-40912. Omit if the issue is not about a specific order."
      }
    },
    "required": ["summary", "priority"]
  }
}

That block is the whole contract. The model is choosing between this function and every other one you attached on the strength of these three fields and the conversation, and it is filling in arguments whose meaning it knows only from what is written here.

The envelope is not free. Serialised, the definition above is about 230 tokens; a lean one with two parameters and a one-line description runs 60 to 90. Tool definitions sit in the prefix of every request in the conversation, before the messages, which means a toolset of a dozen tools at an average of 110 tokens is 1,320 tokens added to every single call for the life of the session. At the current Sonnet-class input price of $3 per million tokens that is about $0.004 per request before anyone has said anything, and on an assistant handling 200,000 requests a month it is roughly $800 of tool definitions. Prompt caching removes most of that, on the condition that the array is byte-identical between requests, which is a good reason to build it deterministically rather than assembling it from a dictionary whose iteration order you have not pinned.

That chain of figures is short enough to rebuild, and rebuilding it is how you get the number for your own toolset rather than this one. Twelve tools at a hundred and ten tokens each is twelve times a hundred and ten, which is the 1,320 figure. Price is quoted per million tokens, so the per-request cost is 1,320 divided by a million, multiplied by the $3 rate — a little under four thousandths of a dollar, which is where $0.004 comes from. Multiplication by the request count is the right last step because the definitions are sent once per request and not once per session, so 200,000 requests carry the prefix 200,000 times, and four thousandths of a dollar two hundred thousand times over is roughly $800. The unit that matters in that last line is the month: the request count was monthly, so the $800 is monthly too, and a figure lifted out of this paragraph into an annual budget is out by more than an order of magnitude.

Notice which quantity that sum is most sensitive to. The token figure and the price are both fixed by things you cannot easily change, but the tool count is multiplied straight through, so a toolset that grows from a dozen to two dozen doubles this line of the bill without a single extra call being made. Definitions are charged for existing, not for being used.

A description is written for a chooser, not for a reader#

The instinct most engineers bring to the description field is documentation: say what the function does, in the register of a docstring. Creates a support ticket. That sentence is true, it is what a wiki would say, and it gives the model almost nothing, because the model's problem is not understanding what a ticket is. Its problem is deciding, on this turn, whether this is the function to reach for rather than one of the other eleven, and then whether it has what it needs to call it.

Descriptions that work read like the note you would leave for someone covering your on-call shift. They state the trigger conditions in the vocabulary customers actually use, they state the boundaries against neighbouring tools by name, and they say what to do when a precondition is missing. The three fields below are the same function described three ways, with the selection behaviour each produced over a fixed replay set of 300 support conversations where a human had already labelled 46 as ticket-worthy.

The lift between the first and second rows comes from stating a condition rather than a definition. The lift between the second and third comes almost entirely from the sentences saying when not to use it, which is the half of the description most people never write. A tool with no stated boundary competes with every neighbouring tool on general plausibility, and the sibling with the broader-sounding claim tends to win.

The two columns move independently, and reading them together is what makes the table say something. Nineteen correct calls out of the 46 labelled cases is a little over two in five, so the docstring version missed more ticket-worthy conversations than it caught while still opening eight tickets nobody wanted. The second description doubles the correct calls and only slightly reduces the spurious ones, which is the signature of a condition that says when to fire and nothing about when to stop. The third adds six more correct calls — 44 of 46, so all but two — and takes the spurious count from five down to one. Those are different mechanisms and it is worth not blurring them: stating the trigger recovers cases the model was not reaching for, and stating the boundary suppresses cases it was reaching for wrongly. A description that does only the first will look excellent on a recall metric and keep filing duplicates.

The on-call note is a comparison rather than a claim about how models work, and it has one limit worth naming. A colleague covering your shift reads the note once, remembers what they read, and can ask you when it is ambiguous. The model reads the definition afresh on every request, carries nothing between sessions except what the conversation itself contains, and has no channel for asking you anything — the only question it can ask goes to the customer. That is why an ambiguity a human would resolve by checking with you resolves here as a guess, and why the description has to state the default rather than leave it to be worked out.

Parameter descriptions do the same job one level down. The priority field above does not describe the enum values, which would be redundant; it gives the operational meaning of the two that matter, in terms of facts the model can actually observe in a conversation, and it names a default for the ambiguous case. Writing The priority of the ticket. there costs nothing to type and buys nothing, since the field name already said that.

An unconstrained string is a guess about your conventions#

A field typed as {"type": "string"} with no further constraint invites whatever string best fits the conversation, and the model has no way to learn your accepted vocabulary from anything else in the request. What that produces is visible in a group-by over the priority argument of 912 create_ticket calls made by a support assistant in its first week live, against a schema that declared the field a plain string:

Eleven distinct values, of which the ticketing system accepted four — the four in the enum above — and silently coerced the rest to normal. The failure is not that the model was creative; it is that nothing in the request said which four were real. Adding six words of JSON — "enum": ["low", "normal", "high", "urgent"] — took the distinct-value count to four. The argument stopped being a free-text field whose conventions the model was guessing and became a closed choice the decoder is held to. The same move is available for every field whose acceptable values you already know: a pattern on an identifier, a format on a date, minimum and maximum on a quantity.

The size of the problem is in the table if the columns are added rather than scanned. All eleven counts sum to the 912 calls, so nothing is missing from the group-by. The four accepted values account for the first, second, third and fourth rows of the left-hand column, and adding those four gives eight hundred and twenty-four. Subtracting that from 912 leaves eighty-eight calls — just under one in ten — whose stated priority was discarded on arrival and replaced with normal. That is the figure worth carrying, because it is the one the ticketing system had no way of reporting: a rejection would have appeared in a log, and a coercion appears nowhere. Two of the seven unaccepted values are merely the wrong case, High and Urgent, which together are thirty-five of the eighty-eight; a case-insensitive lookup would have recovered those and left the rest exactly as wrong as before, which is why normalising on the receiving side is a partial fix and constraining on the sending side is not.

Why an enum works at all is worth one step down, because it is not a validation rule that runs after the model finishes. The provider constrains the decoder while it generates, as topic 5.2 on schema-constrained generation describes: at each position it computes which vocabulary tokens could still lead to an object the schema accepts, and it removes the rest from consideration before sampling. With a plain string, every token that can appear inside a quoted string is legal, and P1 is as reachable as high. With four values enumerated, the only tokens reachable after the opening quote are the ones that begin one of the four, and after u the word is already committed to urgent. The distinct-value count does not drop because the model became better behaved. It drops because the other values became unreachable.

Required means a value appears on every call#

The required array is the most consequential line in the schema, and it is routinely treated as a formality. Marking a field required does not mean this field is important or the caller ought to provide this. It means the model is constrained to emit a value for it in every call it makes to this tool, and the decoder will not let it produce an object without one. There is no mechanism by which the model can call create_ticket with priority left out, so on a turn where the customer's urgency is genuinely unclear, a value gets chosen anyway.

The mechanism behind that is the same masking described above, run in the other direction. The object being generated cannot be closed until every required key has appeared, so at the point where the model would emit the closing brace, that token is not among the ones the decoder will allow. Something has to come next, and the only things allowed are the keys still outstanding. There is no branch in that machinery for I do not know, and no path by which the absence of information in the conversation becomes an absence of a field in the output.

This is where a reasonable belief goes wrong. required reads like a precondition, and a precondition in ordinary code causes a failure when it is not met: the call raises, or the form refuses to submit, and somebody is told. Read that way, marking priority required looks like a safeguard that will make the model ask the customer how urgent this is before filing. It does the opposite. Requiring a field removes the option of leaving it out, and leaving it out was the only signal that would have told you the conversation did not contain the answer. A required field with no default stated in its description does not produce a question. It produces a confident value, indistinguishable in your logs from one the customer actually gave.

That constraint is exactly what you want for summary, since any conversation that warrants a ticket contains the material for a one-line summary, and a ticket without one is useless. It is exactly what you want for priority too, given a sensible default is stated. It is the wrong choice for order_id, which is why that field sits outside the array — plenty of ticket-worthy problems are not about a specific order, and a required order_id would force a value into every one of them.

The rule generalises: a field belongs in required when the information is reliably present in the conversation by the time the tool is appropriate, or when a default stated in the description covers the rest. A field whose value lives only in your infrastructure — an internal account key, a trace id, a tenant uuid — does not belong in the schema at all, required or optional. Your dispatcher knows those values and the model does not, so attaching them to the definition asks the model to produce something that does not exist in its input, and it will comply, because required means required.

Optional fields cost you something too. Every optional parameter is a decision the model makes on every call, and a tool with nine optional filters gets called with a different subset each time, which makes its behaviour hard to reason about in logs. Two required parameters and one optional one is a tool you can debug.

Which parts you can change once it is live#

Editing a definition is not uniformly safe, because the conversation history in a long session contains calls made against the old one. The distinction that matters is between changes the model reads fresh on the next request and changes that contradict what is already in the history.

Renaming is the one that surprises people, because the model treats the conversation as evidence about how these tools work. Four prior calls to fetch_order in the history are four demonstrations, and a request that declares only lookup_order while the transcript is full of fetch_order produces a model that hedges between them. Version the toolset rather than editing names in place, and let existing sessions finish on the definitions they started with.

The step under that is worth spelling out, because it explains the whole column of the table rather than just the rename row. A request is one flat sequence of tokens, and the model has no notion of which parts of it are authoritative declarations and which are transcript. The declared tools say a function is called lookup_order; four earlier turns in the same sequence show a function being called fetch_order and returning something useful each time. Both are evidence, both are in front of it, and demonstrated behaviour in context is exactly the signal these models are strongest at picking up. So the conflict does not resolve in favour of the declaration by rule. It resolves however the weight of the evidence falls, which on a long transcript is often towards the name that appears four times. The same reasoning is why making a field required mid-session is unsafe and adding an optional one is not: an added field contradicts nothing already in the history, while a newly required field makes every prior call in that history an example of a call that omits it.

Some of this arrangement is historical. Before providers exposed a tools field, the same effect was achieved by writing the function signatures into the system prompt and asking for a fixed textual form — the ReAct pattern, short for reason-and-act, in which the model was instructed to alternate a line of reasoning with an Action: search[refund policy] line that you scraped back out with a regular expression. That worked well enough to build the first agent frameworks on and failed in the obvious places: the model wrote Action: search("refund policy") one time in twenty, arguments containing a bracket destroyed the parse, and nothing constrained the argument types. Moving the declaration into a structured field let the provider constrain decoding against the schema, which is where the guarantees from topic 5.2, on schema-constrained generation, start applying to arguments, and it removed the parser from your codebase entirely.

Three fields, then, each doing a job nothing else in the request can do. The name is the key your dispatcher switches on, the description is the only place selection criteria can live, and the schema is the only place your conventions can be stated or a value compelled. Anything you believe about the function that is not in one of those three places is not in the request.

What none of this settles is whether the definition you wrote is the definition the model received. Between your source file and the wire sit an SDK, possibly a decorator that derives the schema from type hints, possibly a framework that rewrites descriptions or appends its own instructions, and the assumption that these agree is where the next chapter starts.