Endpoint-per-tool surfaces

A tool surface is the complete set of operations a model is offered, together with the schema of each one: a name, a sentence saying what it does, and the arguments it accepts with their types and permitted values. A model cannot reach your database or your services directly. It emits a tool name and a JSON object of arguments; your agent loop validates that object against the schema, runs the matching function, and puts the result back into the transcript for the model to read on the next turn. Everything the agent is able to do is something somebody put into that set, and the model's picture of your systems is whatever the set describes. Tool design is the work of deciding where the boundary between the two sides falls: how coarse or fine each operation is, which is the question of granularity, and how much of your services' internal machinery is shown to the model rather than handled for it, which is the question of surface.

Endpoint-per-tool is the shape a surface takes when it is generated rather than written — one tool per route of an API that already exists, with the same path parameters, the same query string and the same response body. In practice you meet a surface as the tools array in a request to a model API, as the exported functions of an MCP (Model Context Protocol) server, or as decorated Python functions in whatever agent framework the team picked, and all three are easy to produce from a specification in an afternoon. The two sides of the interface want different things from it, though. Programs that call your API hold state between calls, keep a cursor in a local variable, know which of six identifiers they are carrying, and follow a written rule about which errors are worth retrying. A model carries nothing between calls except the conversation so far, and re-derives all of that from words, probabilistically, every time it is asked. What that re-derivation costs is measurable, and on one billing team's support agent it was measured.

Seven calls, and what each one is for#

Find this customer's most recent unpaid invoice and email it to them is the workflow this surface was assembled for: one instruction as a person would give it, seven operations as the surface requires them.

Here is the surface, abbreviated to the parts that matter. Each of these is a direct wrapper: the tool name is the endpoint, the arguments are the path and query parameters, the result is the JSON body verbatim.

[
  {"name": "search_customers",
   "input_schema": {"q": "string", "cursor": "string|null", "limit": "int"}},
  {"name": "get_customer",
   "input_schema": {"customer_id": "string (uuid)"}},
  {"name": "list_invoices",
   "input_schema": {"customer_id": "string (uuid)", "status": "string|null",
                    "cursor": "string|null", "limit": "int"}},
  {"name": "get_invoice",
   "input_schema": {"invoice_id": "string (uuid)"}},
  {"name": "get_invoice_document",
   "input_schema": {"invoice_id": "string (uuid)", "format": "string"}},
  {"name": "list_contacts",
   "input_schema": {"customer_id": "string (uuid)", "role": "string|null"}},
  {"name": "send_email",
   "input_schema": {"to": "string", "subject": "string",
                    "body": "string", "attachment_url": "string|null"}}
]

A successful run of that workflow looks like this. The user says email Acme their latest unpaid invoice, and the model has to get from that sentence to a send_email call with four correct arguments.

1  search_customers(q="Acme")
   -> {"results": [{"id": "c_8f2a...", "name": "Acme Manufacturing Ltd"},
                   {"id": "c_31bd...", "name": "Acme Digital (dissolved)"}],
       "next_cursor": "eyJvIjoyMH0=", "total": 24}

2  list_invoices(customer_id="c_8f2a...", status="open", limit=20)
   -> {"data": [ ...20 invoices, oldest first... ],
       "next_cursor": "eyJvIjoyMH0=", "has_more": true}

3  list_invoices(customer_id="c_8f2a...", status="open",
                 cursor="eyJvIjoyMH0=", limit=20)
   -> {"data": [ ...7 invoices... ], "next_cursor": null, "has_more": false}

4  get_invoice(invoice_id="in_4c19...")
   -> {"id": "in_4c19...", "amount_due": 18400, "currency": "usd",
       "status": "open", "due_date": "2026-06-30", ...}

5  get_invoice_document(invoice_id="in_4c19...", format="pdf")
   -> {"url": "https://files.internal/inv/4c19?sig=...", "expires_in": 900}

6  list_contacts(customer_id="c_8f2a...", role="billing")
   -> {"data": [{"email": "ap@acme.example", "role": "billing"}]}

7  send_email(to="ap@acme.example", subject="Invoice INV-2026-0412",
              body="...", attachment_url="https://files.internal/inv/4c19?sig=...")
   -> {"message_id": "msg_77a1...", "status": "queued"}

Read that trace looking only for invoice logic. There is one decision in it that is about invoices at all: which of the twenty-seven open invoices is the most recent, which requires knowing that status="open" means unpaid in this system and that the list arrives oldest first. Everything else in those seven calls is transport. The c_8f2a... string has to be copied out of result 1 and into the arguments of calls 2, 3 and 6, and it has to be the right one of the two customers that came back. The next_cursor from result 2 has to be recognised as a cursor, carried into call 3 unchanged, and then stopped on when has_more goes false. The in_4c19... string has to be extracted from a twenty-seven element array and threaded into calls 4 and 5. The signed URL from result 5 has a fifteen-minute expiry that only matters if the run stalls.

Where the turns actually go#

Instrumented to record model turns per run and categorise each tool call, the agent showed a median of 9.4 model turns across its 305 successful runs: seven tool calls, one or two turns of reasoning where the model re-read a result before deciding, and the final answer. Of the seven calls, one is the operation the user asked for and six exist so that the seventh can be made with correct arguments.

That ratio is the plumbing tax, and it has a direct cost as well as a reliability cost. Each hop is a full request to the model carrying the whole conversation so far, which by call 7 includes twenty-seven serialised invoice objects the model needed one field from. On this workflow the median run sent 31,000 input tokens in total across its seven requests, against roughly 4,200 for the redesigned surface later in this topic. The latency is worse than the token count suggests, because the calls are strictly sequential — call 3 cannot be issued until the cursor from result 2 comes back — so the median wall time was 14 seconds and the p95 — the figure only the slowest run in twenty exceeds — was 41.

The token figure is worth taking apart, because seven calls do not cost seven times one call. Each hop is a fresh request to the model, and a fresh request carries the whole conversation so far: the tool definitions, the user's sentence, and every tool result already returned. The first request carries almost nothing. The fourth carries three results, one of which is a page of twenty invoices. The seventh carries all six, both pages included. Summing across the seven requests therefore counts the early results several times over, once for each later turn that has to re-send them, and that repeated carriage rather than any single large payload is where the median run's tokens go. It is also why the total falls faster than the hop count when a workflow is shortened: removing a call removes its own request and shortens every request that would have come after it.

The reliability cost is arithmetic. Seven sequential hops, each with its own chance of producing an argument the API rejects or a result the model misreads, compound multiplicatively rather than additively. A per-hop success rate of 95.8% gives 0.958⁷ ≈ 0.74, which is the 74% completion measured across 412 production runs of this workflow in its first two weeks — 305 finished and 107 stopped somewhere in the middle. A per-hop rate of 99% would give 93%. Nothing about the workflow is hard; the workflow is long, and length is the thing that kills it.

Why the hops compound rather than adding up is the step worth being slow about. A run completes only if every one of its seven hops produces something the next hop can use, and one bad argument anywhere ends it. So the event this run finishes is the event hop one works and hop two works, and so on through hop seven — a conjunction, whose probability is the product of the parts rather than their sum. Multiplying seven numbers each a little below one drags the answer down quickly, because every factor shrinks whatever the factors before it had already produced. Addition is the operation for the failures, not for the successes, and even there only as an approximation: seven chances of going wrong at a little over four in a hundred each come to roughly three in ten, which is in the neighbourhood of the measured shortfall and overshoots it, because it double-counts the runs that go wrong twice.

Independence is an assumption inside that product, and it is worth naming rather than hiding. The seven hops are not independent in practice — a model that mis-copied an identifier at the first hop is more likely, not less, to mis-copy the next value it is asked to carry, and a customer holding twenty-seven open invoices strains hops two, three and four at once. Correlated failures land several at a time in the same run, so the same total number of bad hops is spread over fewer runs, and more runs finish clean than the product predicts. The product is still the right first model, because what it gets right is how completion responds to a change in the number of hops, and the number of hops is the quantity this chapter is about changing.

The stumble that gets a surface like this through review is a reasonable one. Four failures in a hundred calls sounds like a rounding error, and on a single call it is one. The instinct is then to read the workflow the way you would read a checklist — seven mostly-fine steps, therefore mostly fine overall — which is addition, and addition answers the question how many hops do I expect to go wrong rather than the question that decides whether a run is any use, which is how often do none of them. The seventh power is what turns a four-percent per-hop failure rate into a quarter of runs lost, and it is invisible in every dashboard that reports errors per call, because per call is precisely the unit in which the number looks acceptable.

Two levers are available, and the same arithmetic prices them against each other. One is to raise the per-hop success rate and leave the workflow at seven calls, which is what better descriptions and stricter schemas attempt. The other is to reduce the number of hops and accept the per-hop rate you already have.

Read that table by fixing one coordinate and sliding the other. Hold the row at seven hops and move left along it: at the per-hop rate this agent actually had, the run finishes about three quarters of the time, and at ninety-nine percent per hop it finishes a little over ninety percent of the time. The gap between those two cells is the whole of what a better prompt or a stricter schema could ever buy, because those interventions move the base and leave the exponent alone. Now hold the column at the measured per-hop rate and move up it instead: seven hops give about three quarters and two hops give about nine in ten. That second gap is what a shorter workflow buys, at an unchanged per-hop rate. Both numbers come off the same table because both are the same exponent doing the same work, and the only thing that changes between them is which coordinate you are permitted to move.

Along the seven-hop row the two levers are worth about the same: moving each hop from 95.8% to 99% buys nineteen points, and moving from seven hops to two at an unchanged 95.8% buys eighteen. The difference between them shows up when the workflow grows. Per-hop improvements have a ceiling, because a stochastic component executing an instruction correctly 99.5% of the time is already an unusually good number, and the exponent keeps working against you regardless: at twenty hops even 99.5% only returns 90.5%. Reducing the exponent is the lever that scales, and it is the one that requires changing the surface rather than the prompt.

The three things the model is reasoning about that are not invoices#

The 107 failures sorted into three categories, and each category maps to one piece of internal mechanics that the wrapper exposed rather than absorbed.

A cursor is the first of those mechanics and the one most often misread, so it is worth saying what one is before saying what it costs. list_invoices does not return every open invoice, because an endpoint that promised to would have to hold an unbounded result in memory and ship it down one connection. It returns a fixed number of rows and, beside them, an opaque token recording where it stopped. That token is not a page number and not an offset the caller can compute; it is a value the server chose, meaningful only to the server, and the caller's entire job with it is to hand it back untouched. Exactly that property is what makes a cursor easy for a program and hard for a model. A program stores it in a variable, where nothing can accidentally improve on it. A model is holding it as text among several thousand other tokens of text, and producing text that looks right is the one thing a model is unconditionally good at.

Cursors accounted for 38 of them. The most common single failure was the model stopping after result 2, taking the twenty invoices it had as the complete set, and emailing the most recent of those — which is wrong whenever the customer has more than twenty open invoices, and wrong silently, because the run completes and produces an email. Nineteen runs did that. Eleven more invented a cursor value that looked plausible ("20", "page_2", "eyJvIjo0MH0=") and got a 400 back. Eight looped, re-issuing the same cursor because the second page's next_cursor happened to be a value it had already seen, and hit the twenty-call ceiling.

Opaque identifiers accounted for 41. Two customers matched "Acme", and 14 runs picked the dissolved entity, which has no open invoices, at which point the model had to decide whether an empty list meant no unpaid invoices or wrong customer — a question the tool result cannot answer, because both look like {"data": []}. The rest were transcription problems: an id truncated at an ellipsis the model had seen in a log line, an invoice_id passed where a customer_id was expected, and in four cases a UUID that appeared nowhere in the conversation, assembled from fragments of two others.

Status codes accounted for the remaining 28. The API passes HTTP semantics straight through, so the model sees 404 when a customer has no billing contact, 409 while an invoice is mid-write, 429 under load and 403 when the requesting user lacks permission on a specific customer. Nothing in the surface says which of those are worth retrying. Nine runs retried a 403 — three, four, six times — until they ran out of budget. Twelve treated a 404 from list_contacts as a fatal error and reported failure to the user, when the correct behaviour was to fall back to the account owner's address. Seven retried a 429 immediately, with no backoff, three times in a row.

The tempting belief about that third category is that HTTP status codes are a published standard, so a model that has read most of the internet already knows what they mean and does not need telling. The first half is true and the second does not follow. What the standard fixes is the shape of the meaning — this class of code says the request was refused, that class says the server failed — and what it leaves entirely to the service is which real-world condition gets mapped onto which code. A missing billing contact and a mistyped customer identifier both arrive as the same code here, and one of them should be recovered from while the other should not. Nothing in the standard, and nothing in the response body, tells the two apart. So a model reading a code correctly and a model knowing what to do next are separate achievements, and only the first of them is something reading the internet could have supplied.

None of that is about invoices, and none of it is knowledge the model can be expected to hold, because it is knowledge about one company's internal service conventions. It is being re-derived, probabilistically, on every call of every run.

Completion is a property of the transcript#

The 305 completions are an upper bound rather than a count. Nineteen of the cursor failures above sit in the failure column only because somebody eventually looked: the run produced an email, the customer replied asking why they had been sent a March invoice they had already paid, and a support engineer traced it back to a first page of results the model had treated as the whole set. A run with the identical defect against a customer holding twenty-one open invoices instead of twenty-seven produces the same wrong attachment, and nobody writes in, because the invoice it sends is plausible and no human reconciles it. Completion is a property of the transcript and correctness is a property of the world, and only the first of the two is cheap to observe.

The signed URL from call 5 is the second failure of this kind, and it is worth following because it is created by the length of the conversation rather than by any single call being wrong. get_invoice_document returns a link with expires_in: 900, and in the ordinary case the gap between that result and the send_email call is 3.1 seconds at the median, so the expiry never comes into play. Six runs asked the support agent a clarifying question in between — did you mean Acme Manufacturing or Acme Digital? — and resumed when the answer came back, between four and twenty-three minutes later. In four of those the link had expired before the email was composed, and the model attached it anyway, because nothing in a conversation transcript goes stale and there is no signal telling it that a string it copied fifteen minutes ago has stopped working. Those four runs are counted in the 305.

The general shape is that a wrapper hands the model values that have lifetimes and no way to reason about them. An expiring URL, a cursor that is only valid against a stable snapshot, an id that a concurrent write may have superseded — each is a piece of state that calling code is expected to hold and refresh, and here it is being held in a context window by a component with no clock. A tool that fetches the document and sends it inside one operation cannot have this bug, because the URL never leaves the server and never waits on a human answering a question.

Where the diagnosis stops#

None of this is an argument that the model is the wrong component or that seven calls is inherently too many. Agents run twenty and thirty step workflows successfully all the time; the length that matters is the number of hops where an error can enter, and a surface that carries its own state across hops has far fewer of those than its call count suggests.

It is also a different problem from having too many tools. A seven-tool catalogue is small, and catalogue size is a separate cost with a separate measurement, which comes later in this topic. Selection was not where these runs died.

And it is not fixed by writing better descriptions. That is the tempting move, because it is cheap: document that the cursor should be passed back verbatim, that a 404 from list_contacts is not fatal, that ids must be copied exactly. Every one of those sentences is a rule that a stochastic component has to execute correctly on every single call, which is precisely the step that was already failing. A description cannot make a leaky contract deterministic; it can only ask, politely and repeatedly, for the leak not to matter this time. Topic 5.11, on writing tool descriptions, is exact about what that string can and cannot do.

The plumbing tax is the share of a workflow's turns that exist to move state between calls rather than to do the task, and on an endpoint-per-tool surface it is most of them. It is charged on every run, it compounds with workflow length, and it is invisible in any metric that looks at one tool at a time — which is why the on-call engineer's diagnosis reached for the model instead.

Before any of the redesign, though, there is a measurement problem to settle. The team above knew their numbers because they had instrumented the agent before they touched the surface. The far more common situation is a redesign that ships alongside a prompt rewrite, evaluated on a task set written the same week, producing a number that cannot support any claim at all — which is where the next chapter starts.