One budget for the run, decremented at every hop

The finance alert for run_9188 came in at $46.61 against a per-run expectation of about $1.80. Nobody had disabled a limit. Every agent in that run finished inside its 25,000-token cap and its 12-tool-call cap, and the run terminated normally after fourteen minutes with a correct answer. The invoice reconciliation it produced was worth roughly two dollars to the customer.

The bill decomposes cleanly, which is what makes it useful.

run_9188  invoice reconciliation, 3 vendor bundles
  agents spawned            19   (1 router, 3 planners, 15 sub-extractors)
  max tokens by any agent   24,180 / 25,000     under cap
  max tool calls by any     11 / 12             under cap
  total prompt tokens       402,100  @ $15/M  =  $6.03
  total completion tokens    61,000  @ $75/M  =  $4.58
  case-law lookups              96  @ $0.22   = $21.12
  OCR pages                  1,240  @ $0.012  = $14.88
                                       total  = $46.61
  wall clock                14m 09s

Fifteen sub-extractors is not a number anybody wrote down. The router read three bundles, decided each needed a planner, and each planner decided its bundle needed five sub-extractors, one per vendor section it happened to find. On a two-bundle job with fewer sections the same code path produces six agents and costs $12. The per-agent cap is a bound on a leaf; the run's cost is the cap multiplied by a fan-out — the number of children each agent spawns, and therefore the width of the tree of agents a run turns into — that the model chooses at runtime, and nothing in the system bounds the multiplier.

Halving the caps buys more laps, not fewer#

The instinct is to tighten what already exists: 25,000 tokens per agent was clearly too generous, so make it 12,500 and 6 tool calls. Follow what that does to run_9188's actual work. Each sub-extractor was using about 21,000 tokens to read a vendor section and emit a clause table. At 12,500 it cannot finish a section, so it returns a partial result and — under the retry policy from 10.6, which sees an incomplete output and a recoverable error — the planner splits the section and spawns two sub-extractors instead of one. Thirty agents at 12,500 tokens is 375,000 tokens against the original 402,100, and the tool calls barely move because the OCR pages and case-law lookups are a property of the documents rather than of the agents reading them. The bill lands around $41 and the wall clock goes up, because thirty agents contend for the same 32-slot worker pool.

The general form is that per-agent caps bound the depth of any single agent's work and say nothing about the breadth of the tree, and a limit on depth in a system that can widen is a limit that the system routes around. This is the same shape as a per-request memory limit in a service that can fork: it prevents one request from taking the machine down and does nothing about a thousand small requests doing exactly that.

The budget is one object, and it travels#

What bounds a tree is a single counter that every node in the tree decrements. The budget is created once when the run starts, it is scoped to the run_id, and every hop — a spawn, a model call, a tool call, a message, a retry — reserves from it before acting rather than reporting after.

@dataclass(frozen=True)
class BudgetSpec:
    tokens: int = 250_000
    tool_calls: int = 120
    model_calls: int = 90
    messages: int = 400
    retries: int = 20
    agents: int = 12
    usd: float = 6.00
    wall_clock_s: int = 900

class RunBudget:
    """Backed by one durable row per run; all agents share the row."""

    def reserve(self, run_id, **amounts) -> Reservation:
        # atomic: decrement every named counter or none of them
        row = self.store.decrement_all(run_id, amounts)
        if row is None:
            raise BudgetExhausted(run_id, self.store.snapshot(run_id))
        return Reservation(run_id, amounts)

    def settle(self, res: Reservation, **actual):
        # refund the difference between reserved and actual
        self.store.increment_all(res.run_id,
                                 {k: res.amounts[k] - actual.get(k, 0)
                                  for k in res.amounts})

Reserving before the call rather than counting after it is what makes the bound hold under concurrency. With fifteen sub-extractors running at once and a check that reads the counter after each call returns, all fifteen can pass the check in the same window and then each spend up to a full model call's worth — an overshoot of up to fifteen times the largest single call, which for a 24k-token call is around 360,000 tokens past the line. Reserving first caps the overshoot at whatever the last successful reservation asked for.

The other half is that the handle travels by reference. A spawned agent inherits the run's budget row, not a copy of the remaining amount:

async def spawn(ctx, child_agent, task):
    res = ctx.budget.reserve(ctx.run_id, agents=1, messages=1)
    child_ctx = ctx.derive(agent_id=new_id(), parent=ctx.agent_id)
    #        ^ derive() copies identity and lineage; budget stays the same row
    try:
        return await child_agent.run(child_ctx, task)
    finally:
        ctx.budget.settle(res, agents=1, messages=1)

If derive copied the budget, each planner would start with a fresh allowance and the fan-out problem would come back one level down, which is the bug this section exists to prevent and the easiest one to write by accident.

Wall-clock belongs in the same object even though it decrements by itself. A deadline stored on the run row and checked at every reservation gives the same behaviour as the other counters — the run stops at a hop boundary, with a reason recorded, rather than at whatever arbitrary point a timer fires.

The wall-clock column is the one worth arguing about, because a fifteen-minute deadline on top of the existing caps is the fix most teams reach for and it is not wrong so much as late. It does bound the loss, and at fourteen minutes the $46 has already been paid — the deadline is measuring the symptom that appears last. It also collapses every cause into one outcome, so a run killed for looping and a run killed while legitimately OCR-ing a 400-page bundle produce the same record, which is the substitution chapter 7 takes apart. And it says nothing about the fifteen sub-extractors that were mid-call when it fired.

Where the eight numbers come from#

A BudgetSpec with eight fields invites eight arbitrary choices, and arbitrary choices are what make budgets get raised every time one fires. The defensible way to set them is from the distribution of runs that succeeded. Over the last 30 days this pipeline's successful runs used a median of 78,000 tokens, 34 tool calls and 196 seconds; the 95th percentile used 190,000 tokens, 88 tool calls and 512 seconds; the largest legitimate run on record — a 12-bundle merger due-diligence job — used 231,000 tokens and 731 seconds.

Setting each counter at roughly 1.3 times the observed maximum gives the spec above and puts the expected false-positive rate below one run in three hundred, which is the number to state out loud when someone asks whether the budget will break real work. Setting it at p95 instead — the 95th percentile figures above, the level that five runs in a hundred exceed — would cancel one run in twenty, and a limit that fires on five percent of legitimate traffic is a product decision — some customers' documents will simply never process — wearing the clothes of an operational setting.

The counters are not independent, which is why they are specified together rather than tuned one at a time. Tokens and model calls move together for a given prompt size, so a run that exhausts model_calls at 90 without approaching 250,000 tokens is making many small calls, which is the signature of a loop rather than of a large document. That correlation is useful diagnostically: recording which counter hit zero first turns a cancellation into a weak classification for free.

The one field that should not be derived from historical successes is agents. Fan-out is the variable the model controls most directly and the one with the least stable distribution — twelve is a structural claim about how deep this pipeline's decomposition is ever meant to go, and a run that wants a thirteenth agent is better off failing loudly than being accommodated.

Exhaustion is a procedure, not an exception#

BudgetExhausted being raised in one agent is the beginning of the work. Three things have to happen, and a budget whose enforcement stops at the first one is a budget that reduces spend by a fraction and leaves the run in a worse state than if it had run to completion.

Cancellation has to reach every descendant, which means it cannot be implemented as an in-memory tree of tasks held by the parent. In run_9188 the router spawned three planners on three different hosts; if the router is the agent that exhausts the budget and then exits, an in-memory tree dies with it and fifteen sub-extractors keep working. The reliable form is a durable flag on the run row plus a registry of live agents keyed by run_id, so cancellation is "set state='cancelling' and let every agent observe it", and an agent that is unreachable is handled by its lease expiring rather than by a signal that never arrives.

Every agent then has to observe the flag at a bounded interval. Reservations are the natural check point, since every hop already goes through one, but an agent inside a four-minute OCR call makes no reservations for four minutes. That gap is why the state diagram has a grace period: cooperative cancellation at hop boundaries, where nobody kills the agent and the agent notices the flag and stops itself, and after twenty seconds the supervisor — the process that owns the run row and keeps the registry of which agents are live — revokes the agent's lease epoch and reclaims its slot, so a straggler that returns later finds its writes rejected by the fencing check from chapter 3.

What was held has to be released. Ledger claims go back to pending or to cancelled depending on whether a resumption is wanted, page locks are dropped, worker slots return to the pool, and the open obligations from chapter 4 are closed with a reason rather than left dangling — a cancelled run whose obligation count never reaches zero is a run that no observer can ever classify. The one thing that cannot be released is an external side effect already in flight: the 96th case-law lookup was dispatched, it will complete, and the vendor will bill $0.22 for it whether or not anyone reads the answer. Budgets bound intent, and there is always a tail of committed spend beyond the line, which for this pipeline runs about 3 to 5 percent of the budget.

Finally, the run needs a durable terminal outcome that says it stopped, why, and what exists. cancelled: budget_exhausted (tool_calls 120/120), with pointers to the eleven clause tables that did land and the four sections that never started, is a record a human can act on and a resumption can build from. A run that simply stops emitting is indistinguishable from a run whose trace collector fell over.

The budget stops the run and does not explain it#

A composite budget gives a hard guarantee: no run costs more than $6.00 and 900 seconds, whatever the model decides to do. It gives no account of whether the money bought anything. run_9188 under this budget would have been cancelled at about the eleventh minute with eleven of fifteen sections done, and the record would say wall_clock_s exhausted — accurate, and silent on the question of whether the eleven sections took eleven minutes because the documents were large or because two planners spent four of those minutes re-asking a question they had already answered.

That question is answerable, and it needs a different kind of evidence than a counter. Counters measure consumption; distinguishing productive work from a run going around in circles needs a record of what changed between one step and the next.