Separation decides usability, and 0.88 against 0.87 separates nothing

Abstention is a system declining to answer a question rather than answering it: the "I couldn't find that" reply, the response greyed out behind a "we're not sure about this one" banner, the handoff back to the human support queue such an assistant was installed to relieve. It exists because a retrieval assistant that cannot find an answer does not fall quiet on its own. The system searches a document collection for passages related to the question and puts them into a model's prompt, which is the retrieval-augmented generation arrangement — RAG — that the rest of topic 9 has been about, and when the search comes back with nothing relevant the model answers anyway, out of what it absorbed during pretraining, in the same fluent register as the answers that are right. Because that answer arrives from the tool a company installed so people would stop guessing, it is read as documentation and acted on, which is how a gap in an index turns into misinformation somebody trusts.

Mechanically the defence is small. Something attaches a number to each answer — a confidence signal, an estimate of the probability that this particular answer is correct — and a threshold cuts on that number, so answers scoring below the cut are withheld, hedged, or escalated to a person. The hard part is the number rather than the rule, because a threshold can be no better than the score it sorts by, and the cheapest score available is the one most teams reach for first: ask the model to rate its own answer. A model stating its own confidence in words or digits is verbalized confidence. Every later chapter in this topic replaces that number with a differently-sourced one and measures it the same way, so what counts as "worth cutting on" has to be settled before any of them.

Two complaints that look alike and are not#

Suppose a graded set of 2,000 answers from such an assistant shows accuracy at 0.71 and a mean stated confidence of 0.88, and that splitting the column by grade gives this.

Before reading anything into those means, check that the table is the whole population, because a split that leaves a remainder cannot support any argument about what the column contains. Fourteen hundred and twenty plus five hundred and eighty is two thousand exactly, so nothing is unaccounted for. And fourteen hundred and twenty out of two thousand is seventy-one hundredths, which is the headline accuracy to the digit — the split reproduces the number it was split out of. There is no third bucket, no ungraded residue, and no disagreement between the two ways of counting the same answers.

The recombination works in the other direction too, and it is worth doing because it shows the overall mean carries nothing the two subsets do not. Fourteen hundred and twenty answers averaging 0.88 and five hundred and eighty averaging 0.87, weighted by their sizes, come to a value a couple of thousandths below 0.88 — which rounds to the 0.88 reported for the column as a whole. Two conditional means and two counts reproduce the marginal mean, so the column is exactly these two populations sitting on top of each other and nothing else.

Now the two complaints. The gap between 0.88 and 0.71, and the gap between 0.88 and 0.87, are different defects, and only one of them determines whether anything can be built on this column.

The first is a statement about level: when the model writes 0.88, the empirical frequency of correctness among answers carrying that number is not 0.88. That property has a name, calibration, and chapter 2 is about measuring it properly rather than by comparing two means.

The second is a statement about ordering: whether answers the model scored higher are in fact more often correct than answers it scored lower. That property is discrimination, or separation, and it is the one an abstention rule consumes. A threshold does exactly one thing — it cuts the sorted list at a point and refuses everything below the cut — so its entire value comes from whether sorting by the score puts wrong answers disproportionately below the cut. If the correct and incorrect populations sit on top of each other, as the table above says they do, then every cut discards the two in the same proportion, and refusing the bottom fifth of the list removes a fifth of the wrong answers and a fifth of the right ones.

The reason the distinction gets lost is that a single mean hides it, and the cleanest way to see that is to construct a case where the mean is identical and the ordering is perfect. Reporting "mean confidence 0.88, accuracy 0.71" invites the level reading, because the two numbers are on the same scale and one is bigger, so the natural conclusion is that the model needs to be talked down. Nothing in that pair of numbers is sensitive to ordering at all. A signal that scored every correct answer 0.99 and every wrong one 0.60 would report a mean on this exact population of fourteen hundred and twenty ninety-nine-hundredths plus five hundred and eighty six-tenths, divided by two thousand — which lands at 0.88, the same figure, from a signal that is a perfect abstention rule. One summary statistic covers the best case and the useless one.

That is the stumble this chapter is about, and it is worth naming as a belief rather than as a slip, because it is the belief that keeps a dead column in production. The mean is a quantity that stays constant while the dangerous property varies from perfect to worthless, so a team watching the mean is watching a number that cannot report the failure. The split-by-grade line is the first thing to run on any confidence column that lands on your desk, and it costs one groupby.

The standard summary of separation is the area under the ROC curve — ROC standing for receiver operating characteristic, a term inherited from wartime radar, and the whole quantity abbreviated AUROC — which for a binary grade has a reading that needs no curve at all: pick one correct answer and one incorrect answer at random, and AUROC is the probability that the correct one carries the higher score, with ties counted as half. At 0.50 you are describing a coin. Computed over these 2,000 rows, stated confidence lands at 0.52.

from sklearn.metrics import roc_auc_score

# graded: list of (stated_confidence, is_correct) over the 2,000 held-out answers
scores  = [g[0] for g in graded]
correct = [1 if g[1] else 0 for g in graded]

print(roc_auc_score(correct, scores))   # 0.521

Two percentage points of separation over a coin, on a sample where the standard error of AUROC is about 0.014, puts the estimate a little under one and a half standard errors from chance. That is a distance a coin produces routinely, so the measurement cannot distinguish this signal from no signal with the data available — which is a weaker statement than "the signal is worthless" and the correct one to make. Nothing downstream of it can be better than the number it consumes.

Why the shift-it-down fix cannot rescue the ordering#

The reason to be precise about which defect you have is that the two respond to completely different interventions, and the cheap intervention only fixes the harmless one. Post-hoc recalibration — temperature scaling, Platt scaling, isotonic regression, the binned lookup table you would write by hand — all work the same way. Each fits a function on held-out data that maps raw scores to corrected probabilities, and each of those functions is monotone by construction: if s1 < s2 then f(s1) <= f(s2).

Follow that through in three steps, because the conclusion is stronger than "recalibration usually does not help". First, AUROC is defined entirely by pairwise comparisons: for every pairing of one correct answer with one incorrect answer, it asks which score is higher, and averages the verdicts. Second, a monotone map preserves every one of those comparisons — if s1 < s2 then f(s1) <= f(s2), so a pair that was strictly ordered stays ordered the same way, and the worst that can happen is that two scores collapse to the same value and the pair moves from a full point to a half. Third, since AUROC is a function of nothing but those verdicts, and the verdicts do not change, AUROC does not change. It is invariant under every method in that family, and the invariance is exact rather than approximate. You can verify it on the same array rather than taking it on faith.

from sklearn.isotonic import IsotonicRegression

iso = IsotonicRegression(out_of_bounds="clip").fit(cal_scores, cal_correct)
recalibrated = iso.predict(scores)

print(roc_auc_score(correct, scores))         # 0.521
print(roc_auc_score(correct, recalibrated))   # 0.521  — unchanged, necessarily

Said plainly, recalibration relabels the scores and does not reshuffle them. It can move 0.88 to 0.64 and 0.87 to 0.61, which makes the numbers mean what they say, and afterwards the answer that ranked above another still ranks above it — including every wrong answer that ranked above a right one.

This is the load-bearing asymmetry of the whole topic. A signal that separates but is badly scaled is a fixable signal: fit the map, and afterwards 0.9 means roughly nine times in ten. A signal that does not separate is not fixable by any function of itself, because every function of a single score either preserves the ordering or scrambles it, and scrambling it makes things worse. Recovering separation requires a different measurement — one that looks at something the first one was not looking at — which is what chapters 3 through 5 are for.

There is a smaller problem in the same column that is worth seeing, because it is visible without any grading at all. Models asked for a number between 0 and 1 do not produce a continuum; they produce a handful of round values, because those tokens dominate the distribution over anything that could follow "CONFIDENCE: ".

That table closes as well: seven hundred and six, six hundred and fifty-one, three hundred and thirteen, one hundred and ninety-four and one hundred and thirty-six sum to two thousand, and the four named shares add to a little over ninety-three percent. Ninety-three percent of the traffic landing on four values is a statement about resolution, and resolution has a natural unit. A score that can take four distinguishable states carries as much information as two yes-or-no answers, since two binary choices enumerate four possibilities — two bits, and no more, whatever precision the digits are printed to.

What two bits licenses is a claim about thresholds rather than about accuracy. Even if the score separated perfectly, there would be four places to put a cut, so a threshold at 0.87 and a threshold at 0.83 select exactly the same answers on this population: both sit between the same two occupied values, and nothing lies between them to be moved across. Tuning within that gap is not tuning. A signal has to have both resolution and separation before a threshold is a knob rather than a switch.

What the same measurement says about the alternatives#

The value of putting a number on separation is that every candidate signal in this topic reports on the same scale, over the same 2,000 graded answers, so the comparison is arithmetic rather than argument. These are the figures from that graded set, and each of the later chapters is largely an account of why its number came out where it did.

The interesting reading of that table is not the ladder of AUROC values but what moves alongside them. Cost and separation rise together down the rows, which by itself would only say that better things cost more. The gap that carries information is between the third row and the fourth, because that is where the kind of cost changes. The first three signals are all computed from the model's own behaviour — a number it wrote, the probabilities of the tokens it emitted, the spread of its own resamples — and buying more of them means buying more generations. The fourth leaves the model entirely and reads the retrieved passages with a separate component. That is the only step in the column where a new object enters the measurement, and it is the step where the number jumps by more than the other three gaps combined.

None of those numbers is good in the sense that a fraud model's 0.97 is good. At 0.79, the probability that a random correct answer outranks a random wrong one is seventy-nine hundredths, so the complement — more than one pair in five — is a pairing where the entailment score puts the wrong answer above the right one. What the column establishes is an ordering of candidates worth engineering effort, and it establishes that the cheapest signal, the one that requires no infrastructure and reads as if the model is telling you something, sits at the bottom.

Coverage, operating points, and the reach of one graded set#

Separation is not accuracy, and a high AUROC does not mean the system answers more questions correctly. It means the system can tell, better than chance, which of its answers to withhold. Adding an abstention layer to an assistant with 71% accuracy leaves the underlying 71% exactly where it was; what changes is which answers reach the user and how many questions get no answer at all. The improvement is in the quality of what is served, purchased with coverage.

Nor is a usable separation figure a licence to ship the threshold that comes with it. Every cut refuses some answers that would have been correct, and at AUROC 0.79 a cut that removes half the wrong answers removes a substantial share of right ones with them; the ratio is a property of the two distributions and not something you get to choose. Whether that trade is worth making is a question about what a refusal costs relative to a wrong answer in the product you are building, and nobody in the evaluation can answer it. Chapter 6 is about measuring both sides of that trade and handing them to whoever owns the consequence.

It is also not a substitute for knowing what a specific cutoff does, and the reason is in the definition. AUROC integrates over every possible threshold, which makes it the right summary for deciding whether to build on a signal at all and the wrong one for deciding where to cut. Two signals with identical AUROC can behave completely differently at 0.8: one may put most of its wrong answers just below the cut and the other may spread them evenly across the range, and only the first gives you a usable operating point at that number. That is why "the correlation is significant, ship the threshold" does not follow, and chapter 2 is about the measurement that does answer the question.

And a graded set of 2,000 answers drawn from real traffic is not a permanent verdict on verbalized confidence. It is a verdict on this model, this prompt format, this document collection, and this question distribution, and the honest form of the claim carries all four conditions. Elicitation strategy in particular matters: asking for a percentage in words, asking for top-k guesses with probabilities attached, or asking the model to evaluate a completed answer in a second call all measure differently from a number appended to the answer itself, and the Just Ask for Calibration work reports meaningful movement between them. So a reviewer told that verbalized confidence does not work should ask which elicitation was measured before generalising from it. What survives across those variants is the structural problem, which is that the number is produced by the same process that produced the answer, in the same forward pass, conditioned on the same context — including the retrieved passages the model has already decided to treat as sufficient.

That structural point is what makes the sprint-planning question tractable when two engineers propose two different signals and only one can be validated. The choice is not settled by which sounds more principled; it is settled by which one's failure mode is not already implied by the answer's own generation, and by running the same protocol on both when there is time: a few hundred graded answers from real traffic, separation computed on the grades, and a reliability curve to read the operating point off. The protocol is cheap enough that "we didn't have time" usually means nobody had assembled a graded set, which is a one-week job that every subsequent decision in this topic depends on.

What a graded set cannot yet tell you is the thing a product team asks for at the start: if we suppress everything below 0.8, how many wrong answers stop reaching users, and how many good ones do we throw away with them. Separation says the question is worth asking of a signal, and it says so by summarising across every possible cut — which is exactly the operation that discards the answer for any particular one. Getting that answer requires looking inside the score's range rather than summarising across it.