Budget scopes and reservations

A budget scope is the thing a ceiling is counted against: one task, one user, one customer organisation. Each scope stops a different failure and is blind to the others, which is why a single number in a config file is never the answer to "how much can this cost". The per-task ceiling is the one covered so far — an iteration count, a cumulative token count and a euro estimate carried inside the loop — and it has a limitation by construction. It bounds one task and then resets. Nine hundred tasks that each halt correctly at the ceiling are nine hundred purchases at the ceiling price.

This chapter builds the two scopes that outlive a task, and the machinery that makes them safe under concurrency. Once a counter outlives the task it stops being a local variable and becomes shared state, with a store, a key, a window and a race condition, and each of those is a decision with a failure mode attached.

Three scopes, and the thing each one misses#

The fourth row is not a scope so much as a second dimension that every scope needs. A token cap counts tokens, and a search tool at €0.005 per query, a sandbox at €0.0004 per CPU-second, an OCR vendor at €0.012 per page and a data provider at €0.30 per lookup produce charges that appear in no usage block anywhere. A night of nine hundred document-summarisation tasks made roughly 31 search calls each, which is 27,900 queries and €139.50 from a vendor whose invoice arrives separately and whose spend was not visible in any of the token dashboards. On agents with expensive tools the ratio inverts entirely: a research agent I measured spent €0.42 on models and €3.10 on data-provider lookups per task, so a token-denominated cap was bounding 12% of the cost.

Both of those figures are worth reproducing rather than reading past, because each is one multiplication and each answers a different question. Nine hundred tasks making about thirty-one search calls each is the query count, and the query count multiplied by the per-query price is the €139.50 — a number that exists entirely outside the token ledger and would not appear in it however carefully the tokens were counted. The research agent's 12% is a division rather than a multiplication: the model spend divided by the model spend plus the lookup spend. That ratio is the thing to compute for your own agents before choosing a unit, because it is what decides whether a token cap is a cap or a rough correlate. On the summarisation workload the tool charges are a small fraction and a token cap is nearly right; on the research agent the same cap governs a minority of the bill and leaves the majority unbounded. The workloads differ by which tools they call, and nothing about the enforcement machinery changes between them — only the unit it counts in.

The consequence is that the ledger has to be denominated in euros, with tokens as one contributing line rather than as the unit. Tokens are convenient because they are returned to you for free on every response, and they stop being sufficient the moment an agent can spend money through anything other than the model.

Check-then-act is a race, and forty parallel tasks will find it#

The obvious implementation of a per-user daily cap is to read the day's spend, compare it to the limit, run the task, then add the cost. That is correct when tasks are serial and wrong the moment they are not.

Consider a €20 daily cap, a user at €18.10, and forty tasks starting within the same second — which is what a client with a retry storm, or a browser tab that fires a request per document, or an attacker with a shell loop produces. All forty read €18.10, all forty conclude they are under the cap, and all forty proceed. At €4.45 each the day ends at €196, nearly ten times the ceiling, with no bug in the loop and no bug in the cap logic beyond the gap between reading and writing. This is the same class of hazard as any read-modify-write over shared state under concurrency, which topic 2.12, on concurrency and throughput, works through in general; what is specific here is that the corrupted quantity is money and the corruption is silent. The arithmetic is the starting €18.10 plus forty times €4.45, and dividing that result by the €20 ceiling gives the factor of nearly ten.

The part worth generalising is that the overshoot is not a fixed amount of slop. Exactly one of those forty tasks was entitled to start. Each of the other thirty-nine adds its full cost on top of the ceiling, so the excess is the number of tasks that raced, less one, multiplied by what a task costs. Neither factor is bounded by anything in the cap logic: the first is decided by how many workers a client can provoke into starting at once, and the second by the per-task ceiling. Doubling the concurrency doubles the overshoot, and a ceiling whose error grows with a quantity the caller chooses is not a ceiling. That is also why narrowing the window between the read and the write — re-reading just before the call, polling more often — lowers the probability of a race without touching that relationship at all. The requirement is not a smaller gap. It is no gap.

The fix is to reserve before the work and settle after it, so the decision and the state change are one atomic operation, by which is meant that no other worker can read the ledger or write to it while the operation is half done. Redis is used below because a Lua script it runs has exactly that property against the whole store, and any store offering the same guarantee would serve.

-- reserve.lua: KEYS[1] = ledger key, ARGV = limit, estimate, ttl, reservation id
local spent = tonumber(redis.call('HGET', KEYS[1], 'spent') or '0')
local held  = tonumber(redis.call('HGET', KEYS[1], 'held')  or '0')
local limit, est = tonumber(ARGV[1]), tonumber(ARGV[2])
if spent + held + est > limit then
  return {0, tostring(limit - spent - held)}
end
redis.call('HINCRBYFLOAT', KEYS[1], 'held', est)
redis.call('HSET', KEYS[1], 'res:' .. ARGV[4], est)
redis.call('EXPIRE', KEYS[1], tonumber(ARGV[3]))
return {1, tostring(est)}

Two things in that script are doing the work, and they are separable. The first is the held field. The comparison adds three quantities rather than one: spent, which is money already settled from tasks that finished; held, which is money committed to tasks that are running and have not settled yet; and est, the amount the caller is asking to commit now. The middle term is the one the naive version has no representation for. A check-then-act implementation can only see spent, and thirty-nine of the forty racing tasks were invisible in it precisely because none of them had finished. Introducing held gives running work a presence in the ledger, and the moment it has a presence, the forty tasks are counting each other.

The second is that the whole script runs as one indivisible operation against the store. Between reading spent and held and writing the new held, no other worker can read that key or write to it, so no two callers can both observe the same headroom and both take it. Put the two together and the forty tasks are serialised by the store: each in turn reads a held that includes everyone ahead of it, and the first one whose sum exceeds the limit is refused, along with everyone behind it. Note which half supplies which property. held is what makes the ledger describe reality; atomicity is what stops two readers acting on the same description. Adding held without atomicity leaves the original race with an extra field in it, which is the version teams tend to ship first.

def reserve(scope_key: str, limit_eur: float, estimate_eur: float) -> str:
    res_id = uuid4().hex
    ok, remaining = redis.evalsha(RESERVE_SHA, 1, scope_key,
                                  limit_eur, estimate_eur, 172_800, res_id)
    if not ok:
        raise BudgetDenied(scope=scope_key, remaining_eur=float(remaining))
    return res_id

def settle(scope_key: str, res_id: str, actual_eur: float) -> None:
    """Move the held estimate into spent, using the real cost."""
    redis.evalsha(SETTLE_SHA, 1, scope_key, res_id, actual_eur)

A task reserves against every scope it touches, innermost first, and releases in reverse order if a later reservation is denied:

reservations = []
try:
    for key, limit in [(f"tenant:{tid}:{day}", tenant_limit),
                       (f"user:{uid}:{day}", user_limit)]:
        reservations.append((key, reserve(key, limit, ESTIMATE_EUR)))
    result = run_agent_task(budget=TaskBudget(task_id))
finally:
    for key, res_id in reservations:
        settle(key, res_id, actual_eur=budget.eur)

The release step on the denied path is not tidying, and the diagram spends a line on it for a reason. A task that reserved against the tenant and was then refused by the user scope never runs, so it never settles, so its tenant hold sits in the ledger doing nothing but reducing what every other task in that tenant can reserve. Leave it there and a burst of denials shrinks the tenant's usable budget by the size of the burst, which is a self-inflicted outage that looks exactly like a legitimately exhausted cap. The property to preserve is that a hold exists only while work it corresponds to might still happen; the release on the denied path and the sweeper for crashed workers are the two ways that property is kept, covering the two ways it can be lost.

The estimate used at reservation time should be pessimistic, because a reservation that is too small lets a task start that cannot be paid for, and the per-task ceiling gives you a principled value: reserve the maximum a task is allowed to cost. That over-reserves badly on a workload where p50 is €0.02 and the task cap is €5, since a user with €20 remaining could only start four tasks. The usual compromise is a two-stage reservation — hold the p95 cost up front, and top up the hold mid-task when the in-process budget crosses 80% of what was reserved — which keeps concurrency reasonable and still refuses to let a task run past what the scope can pay.

The held field also needs an escape hatch, because a worker that crashes between reserve and settle leaves a hold forever. Give each reservation a record with a timestamp and sweep holds older than twice the wall-clock task limit; the TTL on the ledger key — the expiry the store attaches to it, after which the key deletes itself — handles the day boundary, and the sweeper handles crashes within the day.

Calendar days are an attacker's schedule#

A cap keyed on user:{id}:{YYYY-MM-DD} resets at a moment everyone can predict, which produces two problems of different sizes. The small one is a thundering herd: every client that got a denial retries at 00:00 UTC, and the resulting spike lands on the ledger and on your provider's rate limits at the same instant. Jittering the retry advice by a few minutes is enough for that.

The larger problem is that a fixed reset lets an account spend the full cap twice inside a few minutes by running at 23:55 and again at 00:05, which turns a €20 daily ceiling into a €40 ten-minute burst on demand. A rolling window removes it at the cost of a slightly more complex counter: keep hourly buckets, user:{id}:{YYYY-MM-DDTHH}, each with a 25-hour TTL, and evaluate the cap against the sum of the last 24. The read is a single MGET of 24 keys, the reservation still needs to be atomic against the current bucket, and the arithmetic is a sum instead of a lookup. For most products the fixed day is acceptable for the entitlement counter and the rolling window is worth the complexity for the safety counter, since the safety counter is the one an adversary is deliberately timing against.

The reason the rolling version closes the burst is a property of where its boundary lives. A calendar-day counter has one boundary, at a moment in absolute time that is the same for every account and is written in a public specification of what "daily" means. An account that restarted a task at 23:55 and again at 00:05 crosses it, and the second run reads a counter that was set back to zero by the clock rather than by anything the account did. A rolling window has no such moment. Spend leaves the counter twenty-four hours after it happened, one bucket at a time, so the counter is only empty for an account that has genuinely spent nothing for a whole window. There is no instant an attacker can wait for, because the reset is indexed to their own past behaviour instead of to the calendar. What that costs is precise and small: the ceiling is evaluated against a sum of the last twenty-four hourly buckets rather than a single value, which is one multi-key read instead of one, and the reservation still has to be atomic against the current bucket alone.

Being denied has to be a first-class response#

When a reservation is refused the task never starts, which is a different event from a task that halted partway. Nothing has been computed, nothing needs checkpointing, and the client deserves to know which of the two it is.

HTTP/1.1 429 Too Many Requests
Retry-After: 3180
X-Budget-Scope: user-daily
X-Budget-Remaining-EUR: 1.20
X-Budget-Resets-At: 2025-03-14T00:00:00Z

{
  "error": "budget_exhausted",
  "scope": "user-daily",
  "message": "This account has reached its daily processing limit.",
  "remaining_eur": 1.20,
  "resets_at": "2025-03-14T00:00:00Z",
  "contact": "support@example.com"
}

Returning the scope matters because the remediation differs: a user-daily denial is something an admin can resolve by waiting or by asking for a raise, while a tenant denial means colleagues are affected and support needs to know now. Returning the remaining budget matters because a client that knows it has €1.20 left can choose a cheaper mode rather than retrying the same expensive request in a loop and generating a second incident out of your first one.

Two details are easy to get wrong. A 429 invites retries, so if the denial is a hard commercial stop rather than a temporary one, 402 payment required, with no Retry-After, communicates it more honestly and stops well-behaved SDKs from hammering. And the denial path itself must be cheap — resolving the tenant, summing 24 buckets and returning is around 2 ms, whereas a denial path that first builds the full request context and counts its tokens costs more than some of the requests it is refusing, which is a small denial-of-wallet vector of its own.

Shared quota is a fairness problem before it is a cost problem#

Tenant-level ceilings exist for a reason that is adjacent to money rather than identical to it. Your provider account has organisation-wide limits — tokens per minute, requests per minute, concurrent requests — and every tenant draws from the same pool. One customer running 200 concurrent agent tasks does not merely spend money; it consumes the throughput that everyone else's requests need, so the symptom your other customers experience is 429s and latency rather than a bill. That is resource exhaustion in the OWASP LLM10 sense sharing a cause with the cost problem, and a euro-denominated cap does not fix it because the euros are being spent legitimately and fast.

The control for that is admission by concurrency rather than by spend: each tenant gets a guaranteed number of in-flight tasks and may burst above it only when the global pool has headroom. Two slots guaranteed and twenty by burst is a reasonable starting shape for a mid-market product, and it means the worst a single tenant can do to everyone else is take capacity that was idle anyway. Weighted fair queueing over tenants, which takes work from each tenant's queue in proportion to a weight you assign rather than in arrival order, achieves the same thing more smoothly if you already have a queue.

There is a failure mode created by shared scopes that is worth naming, because adding a cap can introduce an attack. If one user can exhaust the tenant's daily ceiling, then any user inside a tenant — a disgruntled employee, a compromised seat, a badly written integration — can lock out every colleague for the rest of the day. The per-user cap has to be strictly and meaningfully below the tenant cap for the tenant cap to be a backstop rather than a shared fuse. A tenant of 40 seats with a €20 per-user daily cap and a €300 tenant cap means one seat can take at most 6.7% of the tenant's day, and the tenant ceiling only engages when a substantial fraction of the organisation is unusual at once, which is the situation you actually want a human to look at.

That percentage is the per-user cap divided by the tenant cap, and turning the division round is what makes the design rule usable. Dividing the tenant ceiling by the per-user ceiling gives the number of seats that must simultaneously reach their personal maximum before the tenant ceiling can engage at all — fifteen, out of a tenant of forty. So the ratio between the two caps is not a comfort margin; it is the quorum required to trigger the outer one. Set the per-user cap at the tenant cap and the quorum is one, which is the shared-fuse failure above. Set it far below and the quorum approaches the seat count, at which point the tenant cap can never fire and you have paid for a control that does nothing. The number to choose is the fraction of an organisation whose simultaneous unusual behaviour you would want a human to look at, and the two caps then follow from it rather than being picked separately and checked afterwards.

Safety caps and plan entitlements are different objects#

A commercial quota — "the Growth plan includes 5,000 agent tasks a month" — is a promise you negotiate, upsell against and occasionally waive for a customer on a call. A safety cap is a ceiling that exists so that no combination of bug, abuse and enthusiasm can produce a bill you did not choose. Keeping them as one object is tempting because they look like the same integer, and it ends with sales asking to raise your denial-of-wallet protection during a renewal. Model them separately, put the safety cap well above any plan's expected usage, and let the entitlement produce a friendly upgrade prompt while the safety cap produces an alert and a hard stop.

None of these scopes replaces the in-process budget either. Every one of them lives in a shared store, and the shared store can be unavailable. Decide in advance what happens on a Redis timeout — fail closed and refuse the task, or fail open with a reduced per-task ceiling — and be aware that failing open is a documented way for an attacker to get unlimited spend by first making your ledger unhealthy. The per-task budget, which needs no network at all, is the layer that keeps working when the others do not.

Three scopes, one ledger denominated in euros, and reservations taken before the work is what makes an exposure question answerable at all: exposure per user per day is the per-user cap, exposure per tenant is the tenant cap, and the total is bounded by the sum. Every number in this chapter has been asserted rather than derived, though, which is the next problem and a larger one than it looks — the €4,005 night is the reason it matters, because that account did have a per-user daily token cap configured and the night fitted comfortably underneath it.