Automation, not headcount, creates the account that breaks you

The account manager for Northwind Logistics was preparing a renewal and wanted to know whether there was room to discount. Northwind was a 38-seat customer on the standard $29 plan, $1,102 a month, ranked 61st of 96 tenants by revenue, and nobody had thought about them in a year. The cost query said they had consumed $2,840 of model spend in the last thirty days, which is $74.74 per seat against a $29 price, and that they had done it every month since March.

The largest customer on the platform, a 910-seat health system paying $26,390, had consumed $1,980 in the same period. The account that was destroying gross margin was one twenty-fourth its size, and every dashboard in the company sorted by contract value, so it had never appeared above the fold.

Rank by cost per contract dollar, not by cost#

The ranking that surfaces these accounts is cost divided by revenue, per tenant, which is dimensionless and comparable across contract sizes in a way that raw cost is not. On a $29 seat with $4.30 of non-model cost of goods, the tenant is unprofitable above roughly $0.85 of model spend per contract dollar, and the fleet median sits at $0.17.

The two tenants at the bottom account for $4,244 of a $20,273 monthly model bill, 21% of inference spend against 2.4% of revenue. Neither is large, both were invisible on a revenue-sorted list, and the query that finds them is short enough to run as a scheduled job:

select t.name,
       t.seats,
       t.mrr_usd,
       sum(s.model_cost_usd)                       as model_cost,
       sum(s.model_cost_usd) / t.mrr_usd           as cost_per_revenue_dollar,
       sum(s.model_cost_usd) / t.seats             as cost_per_seat,
       count(*)                                    as sessions,
       count(*) filter (where s.actor_type = 'service_account')::float
         / nullif(count(*), 0)                     as automated_share
from sessions s
join tenants t on t.id = s.tenant_id
where s.started_at >= now() - interval '30 days'
group by t.id, t.name, t.seats, t.mrr_usd
having sum(s.model_cost_usd) / t.mrr_usd > 0.35
order by cost_per_revenue_dollar desc;

The threshold of 0.35 is set at roughly twice the fleet median, which on this fleet returns seven tenants a month — few enough that a human reads every row, frequent enough that the list is never stale. Setting it at the break-even point of 0.85 instead would be tidier and would give you no warning, because a tenant crossing 0.85 has already been unprofitable for the weeks it took to get there.

The signature of the account is in the request shape#

Cost per contract dollar tells you which tenant to look at and nothing about why, and the why determines what you can do about it. Three mechanisms produce nearly all the outliers, and they are distinguishable from ordinary request metadata.

Northwind is the first kind. Their 38 seats generated 14,206 sessions in thirty days, but 14,080 of them came from three service accounts — credentials issued to a program rather than a person, so their traffic follows a schedule instead of a working day — and 126 from the 35 humans. The humans are unremarkable: about 3.6 sessions each per month, $0.19 a session, $24 of model spend across the entire staff. The integration is the customer.

Nothing here is abuse. Northwind read the documentation, obtained an API key that the plan entitles them to, and automated a task the product is good at, which is the outcome every product team says it wants. Their usage will grow with their shipment volume, not with their headcount, and their headcount is what they pay for.

It is also the outcome the company's own growth metrics reward. Northwind's dashboard shows daily active usage climbing steadily, an integration in production, and a product deeply embedded in a customer's operations, which is the profile of an account that will renew. Every leading indicator says this is the healthiest customer on the platform, and the only metric that disagrees is the one nobody had built. Accounts like this are not a failure of the customer relationship; they are a customer using the product correctly under a contract that priced the wrong thing.

Callisto Legal is the second kind, and the shape is different in a way that matters for the remedy. Their session count is proportionate — 3,900 in the month, about 63 per seat — but mean input is 46,000 tokens because their workflow attaches an entire executed contract to every request rather than retrieving the relevant clauses. Cost per session is $0.36 against a fleet median of $0.19, and their cache hit rate is 6% because every prefix contains a different document. A rate limit does nothing for Callisto; a retrieval step in their workflow would cut their cost by two-thirds and probably improve their answers, which makes it a solution both sides want.

Catching it on the night it starts#

A monthly aggregation found Northwind four months late, and the reason is arithmetic rather than negligence: 640 sessions appearing on one night against a tenant's thirty-day baseline is a 4% deviation, comfortably inside normal variation, and it only becomes obvious once it has repeated twenty times. Detecting it on the first night means comparing a tenant against itself on a short window with a multiplicative threshold.

with daily as (
  select tenant_id,
         date_trunc('day', started_at) as day,
         sum(model_cost_usd)           as cost
  from sessions
  where started_at >= now() - interval '28 days'
  group by 1, 2
),
baseline as (
  select tenant_id,
         percentile_cont(0.5) within group (order by cost) as median_day
  from daily
  where day < current_date - interval '1 day'
  group by tenant_id
)
select d.tenant_id, d.cost, b.median_day, d.cost / nullif(b.median_day, 0) as ratio
from daily d join baseline b using (tenant_id)
where d.day = current_date - interval '1 day'
  and d.cost > 25                       -- ignore noise from tiny tenants
  and d.cost > 4 * b.median_day
order by ratio desc;

The floor of $25 a day matters as much as the 4× multiplier. Without it, a tenant that spent $0.40 yesterday and $2.10 today fires an alert at 5.25×, and a team that gets six of those a week stops reading the alert. With both conditions, this rule fires about twice a month on this fleet, and on the night Northwind's job first ran it would have fired at 19×.

Triage in the first two days#

Once a tenant is flagged, the work is to establish which of the three mechanisms is operating and whether the usage is legitimate, before anyone from the commercial side makes contact. Four queries answer it. The share of sessions originating from service accounts or API keys rather than browser sessions separates automation from human use. The concentration of sessions by hour of day identifies a schedule. Mean and p99 input tokens per request against the fleet baseline identifies oversized context. Turns per session and error rate identify a retry storm.

Northwind's readout took twenty minutes: 99.1% service-account, 96% of sessions between 02:10 and 02:45, input tokens at the fleet median, turns per session at 5.0 with a 1.2% error rate. That profile is a well-behaved batch integration doing exactly what the product does, at a volume the contract never contemplated, and the correct next step is a conversation rather than a rate limit — the usage is not a defect, and treating it as one turns a growth opportunity into a support escalation.

The opposite finding is worth naming because it is common and easily missed. A tenant whose flagged spend turns out to be a retry storm is consuming inference on failures caused by your own service, and billing them for it, if you bill by usage, is indefensible. On this fleet one flagged tenant at $0.94 per contract dollar resolved to a tool that had been returning malformed JSON for their region since a deploy eleven days earlier, sending the agent into six extra turns per session. Fixing the tool returned them to $0.21 and nobody needed a commercial conversation at all, which is why diagnosis precedes negotiation.

Why revenue-shaped intuition keeps getting this wrong#

The mental model that produces the wrong answer is the one carried over from infrastructure-heavy SaaS, where cost of goods scales with data volume and data volume scales with company size, so the largest customer is the most expensive and account teams learn to watch the top of the list. Under that model, cost concentration and revenue concentration point at the same accounts, and watching one is watching both.

Inference cost scales with tasks requested, and tasks requested is a property of what a customer automated rather than how many people they employ. A human seat has a hard ceiling of a few hundred sessions a month even under enthusiastic use, so the human-driven part of the distribution is bounded and well behaved. An integration has no ceiling at all: the same code that sends 640 requests a night sends 6,400 when Northwind's parent company folds two more depots into the feed, and the change arrives as a deploy in someone else's repository with no notice to you.

That asymmetry is also why the mid-sized account is over-represented among the outliers rather than it being a coincidence. Very large customers negotiate custom contracts, get security reviews, and have their integration patterns discussed before they are built. Very small customers do not have engineers to spare. The 30-to-100-seat company has an engineering team, an API key, a real operational problem, and no commercial conversation attached to any of it.

What ranking tenants does not do#

It finds the accounts that are expensive now, and it is a lagging indicator by exactly one billing period. Northwind's nightly job was running for four months before anyone looked, and the query would not have fired on the night it started, because a monthly window dilutes a single night's spike below any sensible threshold. Running the same aggregation on a rolling seven-day window with a lower threshold catches it faster and produces more false positives from ordinary weekly workloads, which is the trade you have to pick deliberately.

Nor does it see the exposure that has not happened yet. Every tenant holding an API key is a Northwind that has not been written, and the ranking has nothing to say about which of the eighty-nine tenants currently below threshold are two weeks away from crossing it. The only forward-looking signal available is structural: how many tenants have provisioned keys, how many have non-zero automated traffic at all, and how concentrated their existing usage is around a schedule. On this fleet, 31 tenants held keys and 9 had any automated traffic, which bounds the population that can produce the next surprise and is a more useful thing to report to a pricing discussion than the current outliers.

It also stops short of telling you what to do. Having identified Northwind, the available responses are all commercial rather than technical: raise their price, move them to a usage-based plan, rate limit their key, offer a cheaper asynchronous endpoint for batch work, or accept the cost as the price of a reference customer in logistics. Choosing among those for one account is a negotiation, and doing it forty times is a full-time job for someone who would rather be selling. Choosing a structure that handles the next forty Northwinds without a negotiation each time is a pricing model, and the options there differ mainly in where they put the discomfort.