Tool design — granularity and surface

Read these first

• Tool calling

What you will know

• Choose between one broad tool and several narrow ones, with a reason

  Base the choice on task boundaries, argument ambiguity, permission scope, validation behavior, and the number of calls a typical workflow requires.

• Decide whether to expose an existing API or build a purpose-built surface

  A model-facing contract should remove transport concerns and internal identifiers while preserving the operations and constraints the task actually needs.

• Rewrite a tool that wraps an internal API too literally

  The resulting schema should absorb pagination, opaque IDs, leaky status codes, and other mechanics that otherwise consume model turns or trigger invalid calls.

• Instrument a tool-calling evaluation to isolate how surface changes affect task success, retries, and invalid calls

  A controlled task set can reveal whether improvements come from clearer arguments and boundaries rather than from prompt changes or easier test cases.

• Review someone's tool set and name the two tools to merge

  Merge candidates should have overlapping intent and confusing routing boundaries, while preserving separate authorization or failure semantics when those are materially different.

Chapters

1. 1

   Endpoint-per-tool surfaces

   One tool per route of an API you already have is a day's work, and it makes the model do your client library's job. On a billing workflow that shape needed seven calls where one was about invoices, completed 305 of 412 production runs, and sent 31,000 input tokens per run against 4,200 for the surface that replaced it. Where the turns go: cursors to carry forward, opaque ids to copy from one result into the next argument, status codes to classify as fatal or not.

   Read chapter
2. 2

   Instrumenting a tool surface change

   A pull request claiming task success moved from 61% to 88% also rewrote the system prompt and swapped the task set, so no arm of it isolates anything. The instrumentation every later recommendation is judged by: a task set held fixed across arms, one variable per arm, invalid calls and retries counted separately from end-to-end success, and per-tool and per-pair breakdowns — because a 64% success rate with 11% of calls failing validation is one badly broken tool hiding behind an average.

   Read chapter
3. 3

   The model-facing contract

   Wrapping an existing REST API one endpoint per tool is a day's work, and that day is the only cost paid once — every mechanic the wrapper exposes is paid again on every call, forever. Priced in the same units: 31,000 input tokens per run against 4,200, 74% completion against 96%, and roughly $35,000 a year plus three full-time people of handling time, against four engineer-days of build. What a purpose-built contract keeps, what it drops, and when wrapping is still right.

   Read chapter
4. 4

   Choosing tool granularity

   Two designs for one analytics agent: run_query(sql) against a read replica, or five typed tools with no free-form SQL. The broad one covers the 29% of questions the typed tools cannot, and loses anyway — it moves the warehouse schema into the model's head, so a hand audit of 120 answered questions found 9 wrong numbers and no errors at all. The five criteria that decide, the one condition under which the broad tool wins, and the middle ground that covered 88%.

   Read chapter
5. 5

   Merging overlapping tools

   Given search_docs, find_document, get_document_by_id, list_tickets, create_ticket and escalate_ticket, exactly one pair should be merged. The tell is complete overlap of intent with no wording that routes a request to one rather than the other, so calls split near a coin flip and neither tool has a definable behaviour. The refusal that matters as much: never put an irreversible, differently authorised action behind an argument value, because a permission is a guard and a description is not.

   Read chapter
6. 6

   Absorbing transport inside the tool

   A document search tool takes a collection UUID and a cursor, returns twenty records, and passes HTTP status through, including 404 for no matches and 409 while reindexing. Nine things have to go right per call and one of them is the query. Explaining the mechanics in 180 tokens of description took the invalid-argument rate from 18.6% to 11.2%; replacing the contract with two arguments, a bounded ranked set and a plain flag took it to 1.1%.

   Read chapter
7. 7

   Catalogue size and selection accuracy

   Adding 128 tools the tasks never need, and changing nothing else, took task success from 91.5% to 74.0% and input tokens per turn from 3,410 to 27,800. Neither cost is visible in any single tool's design, and only one of them caches away. How to measure both — a distractor-injection arm, a first-call-correct column, and an embedding screen for near-neighbour pairs — and why density rather than count is the quantity that actually predicts the damage.

   Read chapter
8. 8

   Generating tools from a spec

   One tool per endpoint from an OpenAPI document is complete, automatic and stays in sync — the third clause is the one that sinks it, because internal APIs change for internal reasons. What breaks at 140 tools and not at twelve, why filtering to the ten most similar descriptions recalls every needed tool on only 46% of tasks, and what to build from 30 days of traffic instead.

   Read chapter