A threshold is calibrated against labelled pairs, never inherited from a blog post

Deduplication needs one number: the similarity above which two passages are treated as copies of each other. The number looks like a constant of nature and behaves like a local unit of measurement. A cosine similarity is the score two vectors get for pointing the same way, running from one for identical directions downwards, and what counts as a high score depends entirely on how the model that produced those vectors spreads text out. Some models place every pair of documents in a narrow band near the top of that range; others spread them across most of it.

Calibration here means the ordinary thing it means for any classification threshold: labelling a sample of the decisions the cutoff will make, and reading off how often each candidate cutoff would be wrong in each of the two available directions. Two rates come out of it. The false-merge rate is the share of the pairs a cutoff merges that a human reader would call distinct documents, and the false-miss rate is the share of genuine duplicate pairs the cutoff leaves unmerged. Nothing about that procedure is specific to embeddings. What is specific to embeddings is how badly the number travels between models, which is where the argument has to start.

One number, three geometries#

Under E5 a cosine of 0.95 sits below the median duplicate and above a good deal of ordinary relatedness, so it merges promiscuously. Under MiniLM the same 0.95 is above almost every duplicate the corpus contains and merges nearly nothing. The number is not a property of near-duplication; it is a property of the geometry a particular model imposes, which chapter 8.5 established is exactly what a similarity metric is reporting on.

The table is more useful read across each row than down each column, because what a threshold sits inside is a gap rather than a level. In every row the workable cutoff falls just under the median labelled duplicate — by a few thousandths for E5 and by a few hundredths for the other two — which is the small margin you can afford to give up before you start losing real copies. Measure the other side of it and the picture changes completely: the same cutoff sits far above the point where ordinary unrelated text stops, by roughly a tenth for E5, a quarter for BGE and three tenths for MiniLM. That second distance is the working room, it is different in every row, and it widens as the model's mean random-pair cosine falls.

So the three models are not disagreeing about which pairs are duplicates. They are disagreeing about how much of the range they spend on everything that is not a duplicate, and a cutoff is placed relative to that spread rather than in the abstract. Any threshold quoted without naming the model it came from is a threshold about someone else's index.

Why a spot check always agrees with you#

The usual substitute for calibration is a spot check: pull thirty pairs near the proposed cutoff, look at them, confirm they are duplicates, ship it. The procedure has an aversion to disagreement built in.

Pairs are almost always sampled by score descending, because that is the order the file comes in and the order the ANN search produced. The top of that list is where the near-certain duplicates live, so a reviewer who reads thirty pairs starting at 0.999 sees thirty duplicates and concludes the threshold is safe. Sampling uniformly from all pairs above the cutoff is barely better, because the score distribution is heavily bottom-weighted and the pairs that decide whether a cutoff is right are a thin slice near the boundary. A uniform sample of thirty pairs above 0.95 on the payments corpus lands, on average, twenty-six of them between 0.95 and 0.965 — which sounds promising until you notice this tells you nothing about 0.94 or 0.97 and nothing at all about how much is being missed below the line.

The deeper flaw is that a spot check is a one-sided instrument. It shows you the pairs you would merge, so it can find false merges. It cannot show you a single pair you are wrongly leaving alone, because those pairs are not in the sample by construction.

That one-sidedness is worth naming precisely, because it is the shape of a defence that carries no information. The output of a spot check is a count of errors among pairs above the line. That count is identical whether the line is placed correctly, or placed so high that most of the corpus's duplicates sit unmerged below it — in the second case it is lower, and lower reads as better. The procedure returns its most reassuring result in the state of the world it is supposed to warn about. A reviewer who sees no errors has evidence for exactly one claim, which is that the threshold is not too low, and no evidence at all about the question they thought they were answering.

Sampling where the decision is, not where the mass is#

The fix is the same one used for calibrating any classification threshold: stratify the sample across the score range, label within strata, and reweight back to the population. Divide the candidate file into bands, sample a fixed number from each band regardless of how large the band is, and every band gets a rate estimate — including the sparse high bands that a proportional sample would have covered with two pairs.

The payments candidate set, after the exclusions from chapter 3, held 2.4M pairs distributed like this:

The last two columns are where the reweighting happens, and it is two operations rather than one. The Rate column divides the labelled duplicates by the number sampled, which is an estimate of the proportion of duplicates in that band and carries no information about how big the band is. The Implied true pairs column multiplies that rate by the band's whole population, which converts a proportion back into a count. Multiplying is the right operation because the sample within a band was drawn without regard to score, so the rate measured on sixty pairs is an unbiased estimate of the rate across all of them; scaling a rate by a population size is what turns "one in six of these" into "about eight thousand of those".

The design of the Sampled column follows from what the labels are for. It is nearly flat — a hundred and twenty in the two largest bands and sixty everywhere else — where the populations differ by four orders of magnitude. A proportional sample would have spent almost every label on the bottom two bands, which between them contain nearly all the pairs and almost none of the duplicates, and would have covered the 0.99 band with a pair or two. Flat sampling buys a usable rate estimate in every band, at the price of having to carry the population sizes around separately.

Seven hundred and twenty labels, at roughly 25 seconds per pair once the reviewing tool put the two texts side by side, is five hours of work for one person. Two people labelled independently and agreed on 91% of pairs, Cohen's kappa 0.79 — kappa being agreement corrected for the share two labellers would have reached by chance alone, on a scale where 1.0 is perfect and 0 is no better than guessing — with the disagreements concentrated between 0.92 and 0.96 and adjudicated by a third reader.

Agreement that high requires a written decision rule, because "is this a duplicate?" is not self-evident on a forked runbook with two edited paragraphs. The rule that worked was framed around the reader rather than the text: if a user has already read A, does reading B give them any information they would act on differently? Under that rule the release-notes siblings from chapter 3 are not duplicates, a PDF export whose only difference is a page footer is, and a forked runbook with one genuinely changed step is not. Whatever rule you choose, it has to be written down before labelling starts and shipped alongside the numbers, because the numbers mean nothing without it.

Reading the trade off the table#

With band rates and band populations, every candidate cutoff can be scored without labelling anything further. The implied-true-pairs column sums to about 51,000 residual duplicate pairs above the 0.85 floor, and each cutoff merges a subset of them:

BANDS = [  # (lower, population, labelled_dup_rate)
    (0.85, 1_450_000, 0.000), (0.88, 770_000, 0.008),
    (0.90,    96_000, 0.067), (0.92,  44_000, 0.183),
    (0.94,    14_500, 0.367), (0.95,  11_200, 0.567),
    (0.96,     9_400, 0.783), (0.97,   3_100, 0.933),
    (0.98,     1_300, 0.983), (0.99,     500, 1.000),
]
TOTAL_TRUE = sum(n * r for _, n, r in BANDS)          # 50,998

for t, _, _ in BANDS:
    merged = sum(n for lo, n, _ in BANDS if lo >= t)
    tp     = sum(n * r for lo, n, r in BANDS if lo >= t)
    print(f"{t:.2f}  merged={merged:>7,}  "
          f"false_merge={1 - tp / merged:.3f}  "
          f"false_miss={1 - tp / TOTAL_TRUE:.3f}")

Working one row of that table by hand is what makes the rest of it readable. Take the 0.98 cutoff. Everything at or above 0.98 gets merged, which is the 0.98–0.99 band and the 0.99–1.00 band, so the pairs merged are 1,300 and 500 added, which is the 1,800 in the second column. Of those, the labelled rates say 0.983 of the first band and all of the second are genuine, so the expected genuine merges are 1,300 multiplied by 0.983, plus 500. Dividing that expected-genuine count by the merged count gives the share that were right, and one minus it gives the 0.012 in the third column. The false-merge rate is a ratio of two expected counts, and the reason a ratio is the right operation is that both counts were produced the same way — population times rate — so the band sizes that made them large cancel between numerator and denominator.

Two things in that table are worth more than the cutoff it eventually justifies. The first is that there is no clean operating point: false merges climb steeply from 0.97 downward, and the false-miss rate never gets close to zero anywhere you would be willing to operate. The second is where the missed duplicates live — the 0.88 to 0.92 range holds roughly 12,800 genuine duplicate pairs, a quarter of the total, in a population of 866,000 pairs that is 98.5% non-duplicates.

Those three figures are one division, and seeing that is what makes the conclusion inescapable. The 12,800 is the Implied-true-pairs entries of the two bands added, 6,400 and 6,400. The 866,000 is the Population entries of the same two bands added, 770,000 and 96,000. Dividing the first by the second gives the duplicate share of that range, and its complement is the 98.5%. So a cutoff placed low enough to reach those duplicates admits roughly fifty wrong pairs for every right one, and no adjustment of the number changes that, because the ratio was fixed by the corpus before any threshold was chosen. Those duplicates are unreachable by any threshold, in the economic sense rather than the technical one.

That is the honest shape of the problem and it points at the real lever, which is not the number but the second condition. Adding the residual-similarity test from chapter 3 to a 0.96 cutoff moved the false-merge rate from 0.159 to 0.038 while the false-miss rate went from 0.764 to only 0.771.

The asymmetry between those two movements is the finding, and it has a cause. A one-dimensional threshold can only move along the score axis, and both populations are spread along that axis, so every step that excludes wrong pairs excludes right ones at some rate — the two rates are tied together by construction. The residual test measures something else entirely: how much of a pair's score survives deleting shared boilerplate. Nothing forces that quantity to track the raw score, and on genuine copies it does not move at all. A second, cheap, orthogonal signal can therefore remove a population without paying for it on the other axis, and that is generally where the improvement in a deduplication pipeline comes from once the score is calibrated at all.

Precision on the estimates deserves a note before anyone tunes to three decimals. Sixty labels give a standard error of about 0.064 on a rate near 0.5, the standard error being the typical distance between a rate measured on a sample this size and the rate in the population it was drawn from, so the 0.95–0.96 band rate of 0.567 carries a 95% interval of roughly 0.44 to 0.69, and the false-merge figures inherit that. That interval is the standard error taken about twice in each direction from the measured rate, which is where the two end points come from. The table distinguishes 0.97 from 0.95 with confidence. It does not distinguish 0.955 from 0.960, and pretending otherwise invites someone to re-tune on noise every quarter.

The candidate search bounds the column it does not appear in#

The false-miss column has a footnote that has to travel with it: those rates are conditional on the candidate search. Chapter 2 measured the self-join's pair recall at 0.961 for the configuration in use, so about 4% of the corpus's genuine duplicate pairs were never in the candidate file, never in any band, never sampled and never labelled.

The correction is a multiplication, and the reason it is a multiplication is worth stating. The pairs a cutoff actually catches are a fraction of a fraction: first the search has to have returned the pair at all, and then the cutoff has to be low enough to merge it. At the 0.96 cutoff the false-miss rate among reachable pairs is 0.764, so the reachable pairs it does catch are the complement of that, a little under a quarter. Only 0.961 of all true pairs are reachable in the first place. Multiplying those two proportions gives the share of every duplicate that exists which this pipeline merges, and one minus that share is 0.773. The 0.764 false-miss rate at cutoff 0.96 is therefore a rate among reachable pairs; against all duplicates that actually exist it is closer to 0.773.

The correction is small here and it is not always small — a pipeline running efSearch=32 with m=20 would be quoting rates conditional on a search that found 87% of pairs, and the gap between what it reports and what is true would be an eighth of the answer. That eighth is just the complement of the search's own recall, which is where the fraction comes from: when an eighth of the pairs are invisible, an eighth of the answer is missing before the threshold does anything at all.

The rates are also specific to one corpus, one chunker and one model. Re-embedding with a different model invalidates every number in the table, and so does a chunker change, since chunk length shifts the score distribution — shorter chunks push everything upward. Treat the labelled pairs as a durable asset and the table as a derived report: the 720 labelled pairs survive a model change and can be re-scored under the new model in seconds, which turns recalibration from five hours of human work into a script.

Nothing about the label set is per-source either, and on a corpus with strongly different families — support articles, code documentation, contracts — a single global cutoff can be visibly wrong for one of them while being right on average. Stratifying the labelled sample by source family as well as by score band costs more labels and gives per-family cutoffs, which is worth doing when one family dominates the false merges.

When someone says the threshold is arbitrary, the answer is the table above, plus the decision rule the labellers used, plus the candidate-search recall figure that bounds the false-miss column, plus the date and the model name. That is a document, it took under a day to produce, and it converts the question from a matter of taste into a choice between stated error rates. Choosing 0.96 with the residual condition, at 3.8% false merges and 77% false misses, is a decision someone can disagree with on the merits.

What the table cannot tell you is whether any of it improved retrieval. It reports how well the detector agrees with human judgement about pairs, and no column in it mentions a query, a ranking or a user. A pipeline can score beautifully on pair-level precision and change nothing anyone notices, and the standard retrieval metrics are, as the next chapter shows, structurally incapable of noticing either.