Idempotency and actionable errors

A model never touches the service behind a tool. It emits a tool name and a JSON object of arguments, and the agent harness — the loop that runs the model, dispatches the calls it asks for and feeds the results back into its context — turns that into a request and serialises whatever comes back into the transcript as text. Everything the model will ever learn about what happened to a payment arrives in that one piece of text, with no colleague to ask and no console to open, and its next call is generated from that transcript and nothing else.

The contract carried in that text is tool semantics: which repeats of a call are safe, and what a failure tells the caller to do about it. Its first half is idempotency, the property that performing an operation several times leaves the same state behind as performing it once, so a caller who cannot tell whether the request arrived may safely repeat it. Reading a balance is naturally idempotent. Paying a supplier is not, and making it so means the server has to recognise a second request as the same intent as the first and commit the effect at most once however many attempts arrive. The other half is the failure response, and the standard it has to meet is that the response be model-actionable: written so the caller can compute its next call from it, naming the precondition that was violated, the value that offended it, the values that would be accepted, and whether repeating could ever succeed. A body reading {"error": "Payment failed"} supplies none of those, which leaves a caller with nothing available to it except repetition.

Both halves exist because of who the caller is. An engineer integrating against your API who receives a bare 400 reads the documentation, asks in a channel, or opens the source, and then retries by hand at most once, having thought about it first. A model does none of that: it is non-deterministic, so two attempts at one intent are not textually identical; it runs unattended, so nobody is watching the fourth retry; and its only way to recover is to emit different arguments and see what comes back. You meet both contracts wherever an agent holds a tool that writes — the handlers behind a function-calling schema or an MCP (Model Context Protocol) server, wrapping payments, refunds, ticket updates, calendar bookings.

The two questions a failed tool call raises#

The first question is whether the failed attempt left state behind. A timeout, a dropped connection, a 502 from a proxy and a client-side cancellation all share the property that the caller does not know how far the server got. The write may have committed, or it may have been rolled back, or it may still be in flight against a database that will commit it two seconds from now. This uncertainty is a property of the transport and the commit boundary, and no amount of care in the error text can resolve it, because in the interesting cases the server never got to produce error text at all. What resolves it is a mechanism that lets a second attempt be recognised as the same intent as the first, so the side effect happens at most once no matter how many attempts arrive.

The chain that produces that uncertainty is short and every link in it is checkable, which is worth doing once because the conclusion is not obvious. The server commits, which changes durable state. It then begins writing a response, and writing a response is a separate act from committing, so it can fail on its own. The connection drops, or the client's read deadline expires, and the bytes carrying the outcome never arrive. What the client now holds is the absence of a response — and an absence is the same object whether the request never arrived, arrived and was refused before any write, or arrived and committed. Nothing distinguishable reached the caller in any of those cases. That is why no field you might have put in the response helps here: the response is precisely the thing that did not happen.

The second question is whether an identical repeat of these arguments could ever succeed. A rejection for exceeding an account limit is deterministic given the account and the amount: the same call at 23:10, 23:11 and 23:12 produces the same 400 with the same body. A rejection because a downstream service returned 503 is not deterministic in that way, and waiting two seconds is a reasonable response. This question is about the semantics of the failure and it is answered entirely by what the response says, because the model has no other source of information. A response that says only "Payment failed" leaves the model to guess, and the cheapest guess available to a system whose training is full of transient network failures is to try again.

These two questions are independent, which is the part a single heading over an incident review loses. Every failed call sits somewhere in the grid they form, and the treatments in each cell are different pieces of engineering owned by different parts of the request path.

A payment that times out after the ledger has committed is the bottom-right cell. A payment refused in 40 ms because the amount exceeds the account's per-transaction ceiling is the top-left. Those are the two corners furthest apart in the grid, and treating them as one class of problem means applying a remedy from one corner to the other.

Independence has an exact meaning here, and it is the whole reason the grid has four cells rather than a line with two ends: knowing the answer to one question tells you nothing about the answer to the other. A downstream 503 arriving before any write is a failure that may succeed later and left nothing behind. A timeout after commit may also succeed later and left something behind. An over-limit rejection can never succeed and left nothing. A write that partly applied and will now be refused can never succeed and left something. All four combinations occur, all four are failures somebody has actually had, and none of them is predicted by the other question's answer. That is the test, and it is what makes selecting a remedy from one answer a mistake rather than a shortcut — a design chosen from the transport question alone protects a call it cannot repair, and a design chosen from the semantics question alone repairs a call it cannot protect.

Idempotency makes a repeat harmless, not productive#

The remedy that gets proposed comes from the bottom-right: add idempotency keys to every write tool and let the agent retry freely with exponential backoff, each wait double the length of the one before it, on the reasoning that a safe retry is safe whether or not the original failure was transient. The first half of that is true and is worth doing. The second half treats "harmless" as if it meant "useful".

Retrying an over-limit payment through an idempotent endpoint produces the same 400 as before, five times, at the same 40 ms of server time and the same model round trip each — roughly 2.2 seconds and 1,100 to 1,500 tokens of context per attempt on a mid-sized model once the tool result, the assistant's reasoning and the next tool call are counted. Four wasted attempts is somewhere near 5,000 tokens and nine seconds, which is affordable in isolation. What is not affordable is the terminal state: the agent exhausts whatever retry ceiling the harness imposes, concludes that it cannot complete the task, and reports the failure in the only vocabulary it has, which is that the system is not responding. That report is wrong in a specific and expensive way, because it sends a human to look at infrastructure when the actual defect was a number in an argument that the agent could have fixed itself.

Both totals in that paragraph are the per-attempt figures multiplied by four, and the four deserves a word: five attempts were made, exactly one of them could ever have been the successful one, so four is the waste. Four attempts at the middle of the 1,100-to-1,500 band comes to somewhere near five thousand tokens, and four at 2.2 seconds is a little under nine. The instructive part is which term dominates. The 40 ms of server time is incurred four times as well, and four times 40 ms is under a fifth of a second, so essentially none of the cost is the work the server did. Almost all of it is the model round trip between one attempt and the next. That is the general shape of an agent's retry loop, and it is why a fast rejection is not a cheap one: the expense is the thinking between the attempts, and a server that answers instantly has no way to reduce it.

The inverse mistake is available too and is less commonly proposed only because it is less flattering. Writing an exemplary error body for a post-commit timeout accomplishes nothing, since the failure mode is that no body arrives. You can add retryable: true, a retry_after, a documented error class and a link to a runbook, and the agent will still see a socket timeout. The information a caller needs in that case cannot be delivered in the failed response; it has to be recoverable on the next request, which is what an idempotency key is for.

The two response shapes#

Concretely, the two failures want different shapes coming back. The timeout-after-commit tool needs to be callable twice with the same effect, which means the second call returns the first call's result rather than a fresh one:

POST /payments HTTP/1.1
Idempotency-Key: pay_9f2c1a7b4e08d3
Content-Type: application/json

{"account_id": "acct_4471", "amount": 4180.00, "currency": "GBP", "invoice_id": "inv_88213"}

{
  "payment_id": "pay_01HZX9",
  "status": "settled",
  "amount": 4180.00,
  "currency": "GBP",
  "created_at": "2026-03-10T14:02:31Z",
  "idempotent_replay": true
}

The idempotent_replay flag is there so the model can tell a fresh success from a replayed one and describe what happened accurately to a human, and so your logs can distinguish a duplicate attempt from a duplicate intent. Everything else in that response is the ordinary success body, deliberately, because a replay that returns a special shape is a shape the model has to learn separately.

The reasoning there goes a step past convenience. Whatever the model does next is generated from this response, and it should be the same thing whether the payment settled on this attempt or on the previous one, because in both cases the supplier has been paid and the invoice is closed. Return the replay as a different object — a wrapper naming it a replay, or a conflict status with the original result nested inside — and the model has to recognise a second shape and map it back onto the first before it can proceed. Every shape a caller has to recognise is a place it can fail to, and this is the one shape that only appears when something has already gone wrong, which is the worst moment to be exercising a rarely taken path. One extra field on the ordinary body keeps the common path single: additive for the caller that wants to know, ignorable by the caller that does not.

The over-limit rejection needs something else entirely, and none of it is about repetition:

{
  "error": "amount_exceeds_limit",
  "message": "Amount 8500.00 GBP exceeds the per-transaction limit of 5000.00 GBP for this account.",
  "field": "amount",
  "received": 8500.00,
  "limit": 5000.00,
  "retryable": false,
  "alternatives": [
    {"action": "split_payment", "max_amount": 5000.00},
    {"action": "request_limit_increase", "tool": "create_limit_review"}
  ]
}

Each part is doing a distinct job. The error code is a stable string that alerting and branching can rely on. The message states the violated precondition in a form a model can act on, including both the offending value and the boundary it crossed. The retryable: false tells the agent that time will not fix this, which is the field that keeps a deterministic rejection from being reported to a human as an outage. The alternatives block names the moves that are actually available, which is what turns a dead end into a next step; without it, a well-behaved agent correctly stops retrying and then correctly reports that it cannot proceed, which is better than paging someone but is still a task that did not get done.

Neither shape substitutes for the other, and the reason is worth stating plainly: the replay body carries information the failed attempt could not deliver, because that attempt produced no response at all, while the rejection body carries information the caller cannot obtain by trying again, because every attempt produces the same response. One is recovered on a later request; the other has to be present on this one.

At-most-once under at-least-once delivery#

None of this delivers exactly-once execution, and it is worth being precise about that because the phrase gets used loosely. What an idempotency key buys is at-most-once side effects under at-least-once delivery: the client may attempt any number of times, and the server guarantees the effect lands once. Whether the effect lands at all still depends on at least one attempt getting through, and whether the caller ever learns that it landed depends on a later attempt or a separate read. A model that times out on its final permitted attempt has an idempotent payment it does not know about, which is a real state your system has to handle and which the key alone does not resolve.

The two halves of that phrase are each carrying a party's obligation, and reading them separately is what makes the residue visible. At-least-once delivery describes the client: it may send the request any number of times, and it has to be allowed to, because it cannot tell a lost request from a lost response. At-most-once side effects describes the server: however many attempts arrive, the effect lands no more than once. Put the two together and the effect lands somewhere between zero times and one time. Zero is a real outcome — every attempt lost in transit, the caller having done everything correctly and nothing having happened — and no amount of care on either side turns that range into exactly one. What closes it is not a stronger key but a read: a status call on the same intent, made after the attempts are exhausted, which is a different call in a different direction. A design that stops at the key is a design with a state it cannot describe.

Neither granularity nor prompting carries this contract#

This is also not the surface-area question of topic 5.10, on how finely to carve a tool surface. Splitting a fat manage_payment tool into submit_payment, cancel_payment and get_payment_status improves what the model chooses to call; it says nothing about what happens when the chosen call fails halfway. A perfectly scoped tool with a vague error body produces Thursday's incident, and a perfectly scoped tool without a key produces Tuesday's. Granularity decides which calls get made, semantics decides what happens when one of them goes wrong, and the second is where duplicate payments and false outage reports live.

It is not a prompting problem either, though it can be papered over with prompting for a while. You can write "if a tool call fails twice with the same error, do not retry it again" into a system prompt and it will help, in the same way that a comment saying "do not call this twice" helps in a codebase. It relies on the model following instruction text at exactly the moment it is confused, applies uniformly to failures that need different handling, and evaporates when the prompt is rewritten by someone who does not know why that sentence was there. The contract belongs in the tool's responses, where every caller sees it and where it survives a change of model.

Sorting a failure into the grid takes one question about the transport and one about the arguments, and the answers select different engineering. A call whose commit status is unknown needs a key that lets the repeat be recognised, and the error text is nearly irrelevant because there usually is none. A call that was refused deterministically needs a body naming the precondition, the offending value and the permitted range, and idempotency does nothing for it. The remedy that feels general — make everything idempotent, retry freely — covers exactly one corner of the grid and quietly raises the cost of another.

The harder of the two is the key. A key that makes a retry safe has to be the same value on the retry as it was on the original attempt, which sounds trivial until you ask who computes it and from what — and the obvious answer, letting the caller generate one per request, protects nothing at all.