Error analysis

Read these first

• Building a golden dataset

What you will know

• Read fifty real failures end to end without skipping to the fix

  A score or truncated diff hides interaction failures, malformed outputs, and cases where the model took a plausible but unsafe path.

• Group them into named failure modes rather than "it was wrong"

  Use concrete labels that another reviewer can apply consistently, or the counts will change with the analyst.

• Rank the modes by frequency times severity

  This makes the long tail visible and forces catastrophic low-frequency cases to compete explicitly with high-volume annoyances.

• Pick the one to fix and say why it beat the others

  The decision should include expected user or business harm and the cost of intervention, not just the largest bar on a chart.

• Show that a change moved the specific mode it targeted

  Use the same slice definition and evaluation procedure before and after the change, or regression elsewhere can masquerade as success.

Chapters

1. 1

   Reading a trace end to end

   A results grid shows an input, a truncated output and a score, which is enough to name a failure by its symptom and never by its cause. This chapter reads one failing conversation twice — as three fields and as the full trace — locates the turn where the system first diverged, separates symptom names from cause names, sets out the four things logging has to capture for the reading to be possible at all, and states what a pass rate can and cannot carry.

   Read chapter
2. 2

   Nominating failures from production traffic

   On an eval set a scorer hands you the failures; production hands you nothing, so the pool has to be nominated, and every surface that nominates is a filter with its own view of what counts as bad. This chapter compares what six surfaces catch and miss, measures their precision and recall against a hand-judged random audit, and draws a uniform random sample alongside the nominated pool because it is the only arm whose base rate means anything.

   Read chapter
3. 3

   Choosing which failures to read

   A pool of four thousand candidates and a budget of fifty traces means the draw decides the conclusion. This chapter stratifies by intent and by week with a floor for small strata, gives the thirty lines of code that do it reproducibly, redraws the strata for a post-release read where a sampled generator flips cases in both directions, argues for finishing all fifty before interpreting any of them, and separates the releases where reading should wait behind a revert.

   Read chapter
4. 4

   Open coding and the codebook

   Wrong, bad and confusing are moods rather than categories, and two people applying them to the same traces produce different counts, so nothing built on those counts can be checked. This chapter writes notes before categories, turns each mode into a rule stated from conditions observable in the trace, measures with Cohen's kappa whether a colleague applies the definitions the same way, and separates the merges that keep information from the ones that bury a compliance exposure in an \"other\" bucket.

   Read chapter
5. 5

   Turning counts into rates and cost

   A count out of fifty failures is a share of failures, not a share of conversations, and the two differ by the whole failure rate. This chapter projects mode counts through a hand-judged base rate and monthly traffic, gives severity a per-occurrence unit that can be multiplied, puts a Wilson interval around every count of three, reweights for the strata that were deliberately over-sampled, and shows the largest bar on the chart turning out to be worth nothing.

   Read chapter
6. 6

   Prioritising modes by expected cost

   A ranking by expected cost is an input to the decision rather than the decision: it ignores what each fix costs and it has nothing to say about a mode that cannot be priced. This chapter puts fix cost and payback beside monthly harm, splits every candidate into a mitigation and a root fix, routes unbounded-tail modes onto a constraint list instead of into the arithmetic, compares the frame against ICE and RICE, and states the five dependencies that would change the answer.

   Read chapter
7. 7

   Measuring whether a mode was fixed

   A pass rate rising four points after a targeted change is compatible with the mode being eliminated, with it being untouched while something else improved, and with a trade of one mode for another. This chapter re-codes the same cases with the same codebook, builds a precondition slice where the mode can actually occur and a control slice where the suppressed behaviour is correct, sets out the four things a flat dashboard is consistent with, and hangs each round of analysis on a decision that already has a date.

   Read chapter