Ambiguity and error tolerance decide fit, not model capability

Four proposals arrived in the same fortnight, and each one came with a demo. The finance team had a screenshot of a chat window producing a revenue forecast for the Nordics that looked reasonable. The tax lead had a transcript where the model computed VAT — value-added tax, the sales tax levied at each step of a supply chain and reclaimed by the business buyer — on a three-line invoice across two jurisdictions, and got it right. The chief of staff had a board narrative the model had written from last quarter's deck, and three people had said independently that it read better than the human draft. The support director had an email routed correctly into the billing queue.

All four demos worked. That is the problem with demos as a selection instrument: on a handful of inputs that the proposer chose, a current model produces plausible output for very nearly any task that involves text, which means the demo has a near-100% pass rate across the candidate set and therefore carries almost no information about which candidate to build. Chapter 1's filter kills the board narrative outright — there is no reference narrative, no execution, no downstream system that accepts or rejects it, and no historical record of a decision it drove — but the other three all have something checkable in reach, and they are not equally good bets.

It is worth being precise about why the demo carries so little information, because the instinct to build one is strong and the reasoning against it is not obvious. A test is informative in proportion to how differently it behaves on the good and bad candidates. Across the four proposals here, the demo passed four times out of four; run it across the twenty-odd proposals that reached the same planning committee over a year and it would have passed something like nineteen. A test with a 95% pass rate on the population it is meant to filter is not selecting, it is admitting, and the residual 5% it rejects are the tasks where the model produced visibly broken output on the proposer's own chosen inputs, which is a very low bar to fail.

The two properties that actually separate them#

What distinguishes a task a model is the right instrument for is how ambiguous and language-shaped the input is, together with how much error the output can absorb. Both need stating carefully, because both have a lazy reading that leads teams astray.

Input ambiguity means that the same underlying intent arrives in an unbounded number of surface forms, that the mapping from surface form to intent requires world knowledge or context rather than a lookup, and that a competent human would have to read the input rather than parse it. A support email saying "the card on file got declined again and now everyone's locked out" is ambiguous in exactly this sense: nothing in it names a queue, the words "locked out" appear in access-control tickets as often as in billing ones, and deciding it is a billing problem requires knowing that a failed charge suspends the workspace. An invoice line arriving as {"sku": "PRO-SEAT", "qty": 40, "ship_to": "IE", "bill_to": "DE"} is text, and it is not ambiguous at all; every field is drawn from a closed set and the mapping to a VAT treatment is a table.

Error tolerance is not the same as low stakes, and treating it as though it were is how teams reject good problems and accept bad ones. What matters is the cost of a single wrong output, whether a correction path already exists and is already used, and whether the error is caught before anything irreversible commits. A misrouted support email is a high-frequency, low-cost, fully reversible error with a correction path that support agents perform dozens of times a day already. A wrong VAT figure on a filed return is low-frequency, high-cost, and commits to a tax authority. Both of these involve money; only one of them tolerates error.

Two sub-properties of error tolerance deserve separating, because proposals conflate them and they have different remedies. Reversibility asks whether the wrong output can be undone at all: a queue assignment can be changed, a filed tax return can be amended at some administrative cost, a sent email to a customer cannot be recalled, and a payment released on the 25th is gone. Time-to-correct asks how long the wrong output is live before somebody fixes it: a misrouted ticket is typically corrected within twenty minutes because an agent opens it and sees it does not belong, while a wrong liability cap in a CRM record can sit for months. A problem can be perfectly reversible and still dangerous if nothing looks at it for a year, and a problem with a short time-to-correct can absorb a surprising error rate, which is why the support queue tolerates 6% without anyone treating it as a crisis.

The intersection is what the answer looks like. A problem is LLM-shaped when the input is ambiguous enough that rules cannot enumerate it and the output is wrong often enough that it needs somewhere to go when it is — which means the environment must already contain a correction mechanism, or you must be willing to build one and pay for it.

Triaging the four#

The revenue forecast fails on ambiguity rather than on error tolerance, and it fails in a way that is easy to miss because the demo was fluent. A model asked to forecast Nordic revenue produces a number and a paragraph of reasoning, and neither the number nor the reasoning comes from the arithmetic of the series; the model has no access to a fitted trend, seasonality, or the covariance between regions unless something computes those and hands them over. The existing gradient-boosted model — an ensemble of small decision trees, each fitted to correct the errors of the ones before it — has a mean absolute percentage error of 4.8% at one quarter out, measured against nine years of actuals. On the same backtest, which means refitting on data up to a past date and then scoring the forecast against what actually happened next, the chat-window version came in at 19%, and — more damagingly — its errors were not stable between runs on identical inputs, which makes the interval around the forecast uninterpretable. There is a real language-shaped problem adjacent to this one, which is explaining a variance after the fact from CRM notes and deal records, and that is a different proposal that nobody wrote.

VAT fails on error tolerance and on the availability of a better instrument. Fourteen jurisdictions, place-of-supply rules, reverse charge, and rate tables that change on published dates are a rule engine's home ground, and the existing one has produced two errors in eighteen months, both traced to a rate table that was updated late. A model achieving 99.5% on the same task would be introducing roughly one error per two hundred lines into a process that files with tax authorities, and the errors would be distributed unpredictably rather than concentrated in a stale table you can go and fix. The tax lead's demo was correct on three lines. The relevant number is not 3/3; it is what happens on the 0.5%, and who finds it.

Why 6% is the strongest thing in the support proposal#

The support director volunteered the misroute rate almost apologetically, as evidence that the current process was poor. It is the single best fact in any of the four proposals, for three reasons that compound.

It establishes an error budget — the rate at which a replacement is allowed to be wrong before it is worse than the thing it replaced — with a number attached. A model routing at 92% is worse than the humans and should not ship; at 95% it is arguably a wash and the argument is about latency and cost; at 97% the conversation is about how to handle the residual. Without the 6%, every one of those numbers is just a number, and the discussion collapses into whether 95% "sounds good", which is a discussion nobody wins.

It proves a correction path exists and is exercised. Support agents reassign misrouted tickets as part of normal work; the queue change takes about fifteen seconds and the customer never sees it. An error the organisation already absorbs 204 times a week is an error the organisation can absorb, and that is a far stronger statement than any risk assessment written in advance.

It is also the label source. Every reassignment is a human correcting a routing decision, which is chapter 1's fourth row: a decision a human already makes, recorded in a table, going back years. The evaluation set does not need to be created, it needs to be extracted:

-- Tickets whose queue was never changed: the initial routing was accepted.
-- Tickets reassigned exactly once within 24h: the final queue is the label.
with routing as (
  select t.id,
         t.body_text,
         t.created_at,
         first_value(e.to_queue) over w   as first_queue,
         last_value(e.to_queue)  over w   as final_queue,
         count(*) over (partition by t.id) as reassignments,
         min(e.created_at) over w          as first_move
  from tickets t
  join queue_events e on e.ticket_id = t.id
  where t.created_at >= now() - interval '3 years'
  window w as (partition by t.id order by e.created_at
               rows between unbounded preceding and unbounded following)
)
select body_text, final_queue, (first_queue is distinct from final_queue) as was_misrouted
from routing
where reassignments <= 2
  and first_move < created_at + interval '24 hours';

Run over the ticket store, that returns 61,400 usable rows, of which 3,690 are misroutes — the 6.0% the director quoted, recovered independently rather than taken on trust. The class distribution matters as much as the rate: 71% of misroutes involve two queue pairs, billing confused with account-access and integrations confused with api-support, which tells you where the difficulty is concentrated and gives the eventual system something specific to be good at. It also tells you something about the ceiling. Where humans disagree with each other about the right queue, a model matching the recorded label is matching a coin flip, and roughly 900 of those 3,690 rows are cases where two agents moved the ticket in opposite directions.

What this filter does not settle#

Ambiguity and error tolerance are properties of the problem, and they are silent about whether the problem is worth solving. Support routing passes both tests handsomely and could still be a poor investment: 3,400 emails a week at roughly twenty seconds of triage each is about nineteen hours of human time, and if the model saves 80% of that it saves fifteen hours a week, which is worth having and is not worth six months. That arithmetic is a separate exercise, and it is the one most often skipped.

The filter is also not a claim that ambiguity is the same as difficulty. Plenty of ambiguous language tasks are easy for a current model and plenty of unambiguous ones are hard; the reason ambiguity matters is that it is what makes the deterministic alternative infeasible, since a rule can only fire on a pattern somebody enumerated. When the input is enumerable, rules are available, and once rules are available they are usually cheaper, faster and more auditable, which is the comparison chapter 4 takes seriously rather than assuming.

Neither does high error tolerance license a system with no verifier. The 6% and the reassignment events are two different things doing two different jobs: the first says errors are survivable, the second says errors are visible. A problem with a high tolerance for error and no way to see the errors is not a safe problem, it is a problem whose failures will accumulate somewhere you are not looking, and that combination is common enough — and expensive enough — that it is the subject of the next chapter.