After five consecutive failures, stop calling the tool at all

The vendor behind web_search went down at 09:12 and came back at 09:34. During those 22 minutes a research agent — the one that assembles competitor briefs from public sources — ran 340 times, and every single run discovered the outage from scratch. A representative trace: step 4 calls web_search with the company name, five transport attempts, 31 seconds, ToolFailure(retryable=True). Step 5 calls it again with a narrower query, because the loop had been taught to feed failures back and the model was doing exactly what a model does with a failure it has been shown. Steps 6 through 14 repeat that with progressively more creative phrasings. The run ended at step 20 with the sentence "I was unable to locate recent public information about this company," having spent 6.8 minutes and about $1.20. Multiplied across the 340 runs, the outage cost roughly $410 in model spend and sent about 3,700 web_search calls at a vendor who was already failing, each of them five HTTP attempts deep.

Nothing in the previous chapter was wrong here. That chapter separated a transport failure — the request well-formed, the arguments correct, the dependency momentarily unable to serve it, and elapsed time the only thing that has to change — from a semantic one, where the model chose the arguments badly and no delay will ever help. The transport retries were correctly scoped and correctly capped. The failure was correctly classified as retryable and correctly kept away from the model's decision surface. The problem is that both policies so far treat each tool call as an independent event, and independence is exactly the assumption a sustained outage violates. Topic 2.11 made this argument about a provider brownout — a spell in which a dependency is degraded rather than dead — and answered it with a breaker in the HTTP client. A circuit breaker keeps a running count of a dependency's recent failures and, once that count crosses a threshold, stops calling it for a while, so that later requests fail immediately instead of each paying to rediscover an outage the system already knows about. An agent needs the same object, and it needs one thing more, because a breaker that only fails fast solves a smaller part of the problem than it looks like it does.

Failing fast makes the waste cheaper, not smaller#

Put a plain breaker under the tool, and after five failures the sixth call returns CircuitOpen in 40 microseconds instead of 31 seconds. The trace changes shape: the run no longer takes 6.8 minutes, it takes 40 seconds. It still uses all 20 steps, still pays for 20 model round trips over a growing context, and still ends with the same apology. The saving is real: the wall clock drops by an order of magnitude, and the vendor load drops as well, though by rather less, since the first five calls still reach it with their full complement of attempts and only the sixth onwards is stopped at the breaker. The money, which was the larger number, is untouched, because the money is in the model calls and the model is still being asked to plan around a tool it keeps being told to try again.

That result is worth taking apart, because its shape is the argument for everything after it. A run's wall clock is the sum of two terms: the seconds spent inside tool calls, and the seconds spent waiting on model calls. A plain breaker drives the first term to almost nothing for a tool that is down, turning 31 seconds of sleeping per call into 40 microseconds, and it does not touch the second term at all. A run's bill is not that sum. It is a count of model calls, priced by the tokens each one carries, and the count of model calls is fixed by the step budget rather than by anything a tool does. So the steps the sleeping calls used to occupy stay occupied, and what the breaker emptied out of them was time nobody was billing for. The gain is exactly the size of the slots it vacated, and those slots were free.

State that as the thing it licenses and the thing it refuses. Failing fast certifies that the run got faster and that the vendor got a rest. It certifies nothing whatever about what the run cost, which means a dashboard showing a ten-fold drop in run duration during an outage is entirely consistent with a bill that did not move. The two figures have different denominators over the same set of calls, and reading either as a proxy for the other is how a team concludes it has fixed something it has only accelerated.

The move that removes the waste rather than accelerating it is to change what the model can see. The tools array is rebuilt on every step anyway, since the whole request is resent; dropping the broken tool out of it means the model cannot emit a call to it, and a short note in the context tells it why so the plan adapts instead of the model wondering where a capability went.

Every link in that chain is checkable, and it is worth walking rather than asserting. The provider decides which tool calls are well-formed by comparing the model's output against the schemas in the tools array of the request it is answering, so a name absent from that array is not a call the model can successfully emit. The array is rebuilt from scratch on every step, because an agent turn resends the whole request rather than amending a session the provider is holding open. Therefore removing an entry takes effect on the very next step, and it takes effect as a constraint rather than as an instruction — the difference being that an instruction is something a model may decline. The note in the context is doing a separate job and carries no enforcement at all. It explains an absence the model would otherwise have to account for on its own, and a model that cannot see why a capability vanished tends to spend a step looking for it.

class ToolBreaker:
    """Consecutive-failure breaker, scoped to one agent run."""

    THRESHOLD = 5

    def __init__(self) -> None:
        self.consecutive: dict[str, int] = defaultdict(int)
        self.opened: dict[str, str] = {}   # tool name -> last error message

    def record_success(self, name: str) -> None:
        self.consecutive[name] = 0

    def record_failure(self, name: str, failure: ToolFailure) -> None:
        if not failure.retryable:
            return                      # a rejected argument is not ill health
        self.consecutive[name] += 1
        if self.consecutive[name] >= self.THRESHOLD:
            self.opened[name] = failure.message

    def available(self, tools: list[dict]) -> list[dict]:
        return [t for t in tools if t["name"] not in self.opened]

    def notice(self) -> str | None:
        if not self.opened:
            return None
        lines = [f"- {name}: unavailable for the rest of this task ({msg})"
                 for name, msg in self.opened.items()]
        return ("The following tools are out of service and cannot be called "
                "again:\n" + "\n".join(lines) +
                "\nContinue with the tools that remain. If the task cannot be "
                "completed without them, say so and report what you have.")

Wiring it in touches three lines of the loop from topic 10.3: the request sends tools=breaker.available(TOOLS), the notice is appended to the system block or to the tool result that carried the fifth failure, and every dispatch calls record_success or record_failure. Only the middle line has a real choice in it. Attaching the notice to the tool result puts the explanation next to the event that caused it, which is where a model is most likely to read it; putting it in the system block keeps it in front of the model for the rest of the run, at the cost of resending it on every step. Either works, and neither is what makes the tool unavailable — that was settled by the first of the three lines, before the model saw anything.

Counting the right events#

Three details in that class decide whether the breaker fires when it should. The first is that it counts consecutive failures rather than a failure ratio. A ratio is the right instrument for a high-volume HTTP client where thousands of calls per minute make a percentage meaningful; inside one agent run there might be four calls to a given tool in total, so a 50% error-rate rule would trip on two failures out of four and a 5-in-a-row rule will only trip on something that is genuinely not answering. Any success resets the count, which means a tool that fails intermittently keeps its place in the toolset and the retry policy from the previous chapter handles it, which is what should happen.

The reason those two rules diverge so sharply is sample size, and it is worth seeing rather than accepting. A ratio is an estimate of a rate, and an estimate built from four observations is a poor one: two failures out of four is a result a single bad minute at the vendor produces routinely, so a 50% rule fires on evidence entirely consistent with a tool that is basically fine. A consecutive rule asks for something structurally different — five failures with no success anywhere between them. Under a fault that is genuinely intermittent, each extra failure the rule demands multiplies the chance of an accidental trip by the failure rate again, so the requirement collapses as the threshold rises. A tool erring one time in five will hand you two failures out of four often enough to matter and five in a row almost never, and that gap between the two rules is the whole reason the second one is the one in the class.

The second is that only retryable failures count. This is the line that connects the two chapters: a 400 saying the date format is wrong tells you nothing about whether the tool is healthy, and counting it would let a model that is confused about arguments amputate a working dependency. The two failure classes need separate counters, and the semantic one — the semantic_failures dictionary from the previous chapter, capped at three — is a different intervention with a different remedy, since its message tells the model to change approach rather than telling it the tool is gone.

The third is that one exhausted tool call is one failure, not five. The transport wrapper already made five HTTP attempts before raising; counting each attempt would trip the breaker on the first call, which defeats the whole point of having a retry policy underneath. With the thresholds as written, opening the breaker requires 25 underlying HTTP attempts across five distinct tool calls, which is enough evidence and — because the agent is also spending a model round trip per call — takes about five steps to accumulate. That is the real tuning constraint: on a run capped at 20 steps, a threshold of 5 spends a quarter of the allowance proving the tool is down, and teams running short agents often drop it to 3.

The general form of that constraint is worth extracting, because it says which of those numbers a team can usefully change. Measured in HTTP attempts, the evidence the breaker demands is the threshold multiplied by the transport wrapper's attempt cap. Measured in steps, the evidence it demands is the threshold alone, because the attempt cap costs seconds and never steps: five HTTP attempts inside one tool call are one model round trip, exactly as one attempt would have been. So the fraction of a run's allowance spent proving a tool is down is the threshold divided by the step budget, and the attempt cap does not appear anywhere in that fraction. A team that raises the attempt cap to be more patient with a flaky vendor pays in wall clock and pays nothing in steps. A team whose runs are capped at ten steps instead of twenty has doubled the fraction without touching a single setting on the breaker, which is why the threshold is the number that moves.

Where the state lives changes what it protects against#

The class above holds its counters in a Python dict that dies with the run, which is the narrowest possible scope and the one to start with. It is not the only option, and the choice determines whether run 341 of the research agent learns anything from runs 1 through 340.

The 340 runs that each rediscovered the outage were paying the top row's cost. Moving the counter to a shared store would have cut the bill to roughly the first handful of runs plus the probe traffic, at the price of a new failure mode: one misconfigured worker whose network policy blocks the vendor can trip a fleet-wide breaker and take a healthy tool away from everyone. Teams that go fleet-wide generally require the failures to come from at least two distinct workers before opening, and they alert on the open event rather than letting it be silent, because a tool that is quietly absent from every run is a capability regression that no test will catch.

Read that table by its differences rather than by its rows and there is only one trade in it, carrying a sign on each side. Moving down the rows, the number of independent discoveries the fleet pays for falls — every run, then every worker, then roughly the first few runs anywhere — and the blast radius of a wrong decision rises through the same movement, from one run's remaining steps to every run in flight. The column that does not move is the one that looks most like a setting: Trips after reads five in all three rows. Widening the scope does not change how much evidence the breaker wants; it changes only the set of calls it may gather that evidence from. That is why the choice is a question about how much you trust your own fleet's uniformity, and not a question about tuning.

Reset behaves differently at each scope, and this is where the agent case genuinely diverges from the HTTP one. Within a run, the honest answer is that the breaker never closes. A run's p95 lifetime on this agent, meaning the figure that 95 runs out of 100 come in under, is 90 seconds, the outage that tripped the breaker lasted 22 minutes, and re-admitting the tool at step 14 would put a coin-flip dependency back into a plan that has already been rebuilt around its absence — the model re-plans toward it, fails again, and the last steps go the way the first ones did. Across runs, the half-open probe from topic 2.11 is exactly right and needs no modification: after 30 seconds open, admit one call, close on success, re-open on failure.

The case for never closing inside a run is arithmetic before it is judgement, and separating the two matters. The evidence this team had is that an outage capable of tripping the breaker ran for 22 minutes, while 95 runs in 100 finish inside 90 seconds. An outage long enough to open a breaker is therefore, on those figures, an order of magnitude longer than the run that found it, so a run re-admitting the tool is overwhelmingly likely to be re-admitting something still down — and paying, for that near-certainty, the steps the model spends planning back toward it. None of that is a requirement, though. The requirement is only that the breaker must not thrash. Never closing inside a run is one design decision that satisfies it, chosen because a run is short enough that the decision costs almost nothing; a fleet-scoped timer is a different decision satisfying the same requirement, and it is available precisely because a fleet outlives the outage.

One piece of the vocabulary in that diagram is borrowed, and a reader is owed the disclosure. The name is an analogy to the electrical device that opens a circuit when the current through it runs too high, and it carries one idea across cleanly: a component that takes itself out of service in order to protect what sits on the other side of it. Two things do not cross. The electrical breaker protects the circuit from the load, whereas this one protects the caller from the dependency, so the direction of protection is reversed and a reader reasoning from the picture will expect the wrong party to be spared. And an electrical breaker is closed again by somebody walking to the panel, which is a deliberate human judgement that the fault is gone; the object above either never closes at all, inside a run, or closes itself on a timer that knows nothing about whether the vendor recovered. HalfOpen exists exactly because the analogy has no equivalent of it and something has to stand in for the person at the panel.

The failures a counter has nothing to count#

It needs failures to trip, so it is strictly downstream of the retry policy and cannot replace it; a tool with a 20% error rate never accumulates five in a row and should not, because backoff handles it and removing the tool would be an overreaction to a dependency that is mostly working.

It is also blind to the failure mode that produces no failures. A search tool that answers 200 OK in 25 seconds instead of 400 milliseconds will never increment a counter, and a run that makes four such calls has burned 100 seconds without a single error to show for it. Latency needs a different instrument, and it is one of the four in the next chapter. The same blindness applies to a tool that returns confidently wrong data: record_success fires, the counter resets, and the breaker's model of health says everything is fine.

That blindness has a mechanical cause, and the exact version of it is stronger than "the breaker is not perfect". The counter has two writers and nothing outside the breaker reads it. record_failure increments it, and only on a failure the transport layer already classified as retryable. record_success sets it to zero. Neither function looks at the tool's result. The counter is therefore a function of status codes and of nothing else, which means a tool answering 200 OK in 25 seconds and a tool answering 200 OK with an inventory count from last week both drive it to zero, and drive it to zero exactly as fast as a healthy tool does. In those cases the breaker's estimate of health is not merely noisy. It is constant, and it reports improvement, while the property that actually matters gets worse without any limit.

Naming that is worth more than filing it under caveats, because the counter is usually the only health signal anywhere in the loop and it is genuinely correct about the one thing it measures, so a team that has one feels covered. What it licenses is a single sentence: a consecutive-failure counter certifies that the tool is answering and never that the answers are worth anything. The hang in the next chapter and the confidently wrong result in the last chapter of this topic are both consequences of that one missing term, which is why neither of them is fixed by tuning the threshold.

Most importantly, routing around a missing tool is a degradation and the run's output has to admit it. An agent that drops web_search and produces a competitor brief from a stale internal wiki has not recovered, it has substituted, and a caller who cannot tell the difference will treat the brief as current. That is why the notice text ends with an instruction to say so, and it is the first appearance of an obligation that gets its own chapter later: whatever the recovery path was, the terminal result has to carry enough information for the reader to know what happened.

What the breaker is, once the pieces are assembled, is a small amount of state that converts a fact the run has already paid to learn into a constraint on what the run is allowed to do next. Counting is the cheap half of that and the half most implementations stop at. The half that moves the bill is acting on the count by rebuilding the request, because a tool the model can still name is a tool the model will still spend a step naming, however fast the call comes back.

The breaker bounds the damage from one dependency that has stopped answering. It has nothing to say about a run where every tool answers quickly and correctly and the agent still never arrives — twenty successful calls, no errors, no broken dependency, and no result. Bounding that needs limits that count something other than failures.