Uncertainty, confidence and abstention

Read these first

• Citations and attribution
• Logprobs and token-level confidence
• Self-consistency, majority voting over N samples, best-of-N with a judge

What you will know

• Produce a usable confidence signal, not just a verbalised one

  A model-generated 0.9 can mean little unless its scores separate correct from incorrect answers on representative evaluation data.

• Measure verbalized confidence against correctness

  Calibration should be checked with reliability plots or expected calibration error, because confidence often changes with phrasing, answer length, and prompt format.

• Use self-consistency spread or an entailment check as a signal

  Repeated samples can agree on the same unsupported claim, while entailment models can fail on negation, numbers, and evidence spanning multiple passages.

• Set an abstention threshold with a measured false-abstention rate

  Choose the cutoff on held-out traffic using both missed-answer and unnecessary-abstention costs, then recheck it after model or corpus changes.

• Break a "say I don't know" instruction on supported and unsupported questions

  A refusal instruction alone usually produces either blanket evasiveness or confident guessing because it supplies no observation tied to answer correctness.

Chapters

1. 1

   Separation decides usability, and 0.88 against 0.87 separates nothing

   Across 2,000 graded answers the mean stated confidence is 0.88 while accuracy is 0.71, and on the wrong answers confidence averages 0.87. The gap between 0.88 and 0.71 is a calibration problem you can often correct; the gap between 0.88 and 0.87 is fatal, because no threshold on that number abstains better than a coin. This chapter separates the two properties and says which one decides whether a signal is worth building on.

   Read chapter
2. 2

   A reliability curve tells you what a 0.8 cutoff actually buys

   Reporting that stated confidence correlates with correctness at r = 0.31 and therefore thresholding at 0.8 names a statistic that says nothing about 0.8. You will bucket answers by stated confidence, plot accuracy within each bucket, read off the precision and abstention rate a cutoff would produce, and summarise the whole curve with expected calibration error. Also covers how stated confidence shifts with phrasing, answer length and prompt format.

   Read chapter
3. 3

   A logprob scores the phrasing, not whether the passages said it

   A sequence logprob is the model's own probability over the tokens it emitted, and normalising by length turns it into something closer to a fluency score than a correctness one: a smooth, well-formed sentence outranks an awkward true one. For RAG the deeper problem is that it is model-internal in exactly the way the verbalised score is — when retrieval comes back empty and the answer is produced from parametric memory, those tokens are highly probable, so the logprob agrees with the stated confidence rather than correcting it. Add the practical constraint that many hosted APIs do not expose logprobs at all, and you have both halves of why this cheap signal is rarely the one to build on.

   Read chapter
4. 4

   Five samples agreeing measures the model's peak, not the passages

   Sampling five answers at temperature 0.8 and treating disagreement as uncertainty fails exactly where RAG hurts most. On questions the corpus cannot answer, a confidently held parametric belief produces five identical wrong answers, because agreement measures how peaked the model's own distribution is. That is also why averaging this signal with stated confidence yields a composite most confident precisely where it should be least.

   Read chapter
5. 5

   Entailment reads the passages, and breaks on negation and numbers

   An NLI model scoring the retrieved passages against each claim is the first signal here that looks at the evidence rather than at the model, which is what makes it independent of the parametric-memory failure shared by the verbalised score, the logprob and the agreeing samples. It has a shape of weakness worth memorising: negation and numeric edits change meaning while leaving wording intact, and a claim supported jointly by two passages is entailed by neither one alone.

   Read chapter
6. 6

   An abstention threshold is a two-sided cost, so measure false abstentions

   Shipping at 0.7 with a 12% abstention rate and a 60% drop in complaints looks like success and is half a measurement. Wrong answers generate complaints; refusals generate silence, so a threshold tuned by watching complaints can only ratchet upward until the system refuses nearly everything. Here you measure false abstention on held-out traffic, questions refused that would have been answered correctly, and hand both costs to whoever owns them.

   Read chapter
7. 7

   Telling the model to say it does not know supplies no observation

   Adding a refusal instruction moves hallucinations from 14% to 3% on a set where every question is unanswerable, which is a set on which refusing everything scores a perfect zero. This chapter explains why the sentence produces either blanket hedging or unchanged confident guessing, and specifies the mixed evaluation, answerable and unanswerable questions together, that can see over-refusal and hallucination at the same time.

   Read chapter