Cosine similarity cannot tell agreement from disagreement

Two documents in a corpus can assert incompatible values for the same fact without anything in a retrieval system noticing. One says a single sign-on session lasts eight hours, another says twelve, both are indexed, both are retrievable, and the instrument teams reach for to find pairs like that is a near-duplicate detector: something that scores every pair of passages by cosine similarity and flags the closest ones for a human. That instrument answers a question about wording. Whether two passages agree is a question about the values inside them, and the two questions come apart hardest exactly where it matters most — on a pair whose sentences match word for word and whose numbers do not.

Four terms are worth naming plainly, because the rest of this topic turns on them. The corpus is the body of documents a retrieval system has ingested, chunked and indexed — a wiki export, a folder of runbooks and a handful of policy PDFs somebody pointed an ingestion job at — and it is the whole of what an assistant knows about the company. A contradiction is two documents in that corpus asserting incompatible values for the same fact, which is what a handbook and a runbook do about session lifetime. A duplicate is the same content present more than once, usually as several revisions of one page that were copied rather than edited in place. A stale document is one whose content has been replaced by something newer that also sits in the index, so both remain retrievable and either can come back for a query. The three arrive together more often than separately, since the SSO pair is a contradiction between two near-duplicates of which one is probably stale.

The assumption underneath every ranking function#

None of these is a retrieval defect in the usual sense, which is why they need machinery of their own. The work of retrieval — chunking, embedding, ranking, reranking — is aimed at which passage best matches a question, and all of it rests on an assumption that is rarely stated: a passage that matches well is also true and currently in force. A corpus assembled over six years by two hundred people breaks that assumption as a matter of routine, because nobody deletes anything, replacements get written beside the things they replace, and the old page keeps its inbound links and its wording. Handling contradictions, duplicates and stale documents means deciding what the system does when that assumption fails, and the decision belongs between retrieval and generation rather than inside either.

It is worth being precise about why the assumption is invisible from inside the pipeline, because that is what makes the failure survive. A retriever is scored on whether the passage it returned is relevant to the question. A generator is scored on whether the answer it wrote is supported by the passage. Both of those are satisfied in full by a stale passage that a query legitimately matched, so every component reports success and the composite reports success, and there is no stage in the chain whose job description includes noticing that the passage is wrong about the world. The defect is not located in any component; it is located in a property of the corpus that no component was asked about.

The place you meet this is the first internal deployment over documents nobody curated for the purpose. A support knowledge base carries an article and its rewrite; an API reference keeps the v1 pages alongside v2; a shared drive holds four policy PDFs whose filenames end in _final_v3. The symptom is two people asking the same question, receiving different answers, and both answers citing something real, and it is a symptom no retrieval metric registers, because from the pipeline's point of view both runs succeeded.

The digit is two tokens in an otherwise matching sentence#

An embedding model maps a passage to a point whose position encodes what the passage is about. Two sentences that describe the same policy, in the same register, with the same vocabulary, land close together, and the number inside them is a small part of the token sequence that gets pooled away along with everything else. Changing 8 to 12 alters one or two token positions out of roughly sixty, and the pooled representation barely moves.

The step from that sentence to the scores below is worth walking, because it is what makes the result inevitable rather than surprising. A model of this kind reads a passage as a sequence of tokens — word fragments, of which a short paragraph has a few dozen — produces one vector per token position, and then pools those vectors down to the single vector that gets stored. Pooling is an averaging operation, so if a rewrite changes one or two positions and leaves the rest alone, the pooled vector moves by something on the order of the changed share of positions multiplied by the difference between the old token's vector and the new one's. One or two positions out of roughly sixty is a few percent, and the resulting displacement is a few percent of one token's vector rather than a few percent of the passage's.

Cosine similarity then absorbs most of what is left. Cosine measures the angle between two vectors rather than the distance between them, and for two vectors that already point nearly the same way, a small sideways nudge costs an amount that falls off with the square of the nudge — displace by a hundredth of the vector's length and the cosine falls by about half a ten-thousandth. The two effects compound in the same direction, and a one-token difference between otherwise identical sentences arrives as a change in the third decimal place.

Two caveats keep that from being more than a sketch of the mechanism, and both are worth stating rather than leaving for a reader to trip over. Real pooling is not a plain average of token vectors; production models apply attention and normalisation around it, so the changed share of positions is a rough bound on the displacement and not a formula for it. And the vector for 12 is not a fixed distance from the vector for 8, so the size of the difference term varies with the pair. What survives both caveats is the shape of the argument: the changed positions are a small fraction of the total, and the pooled result inherits that fraction. Nothing here is a defect in the model. It is the model doing what it was trained to do, which is to place texts about the same subject near one another, and a sentence about session lifetime is about session lifetime whichever number it carries.

import numpy as np
from openai import OpenAI

client = OpenAI()

PASSAGES = {
    "handbook":  "SSO sessions remain valid for 8 hours. When the session "
                 "lifetime elapses the user is redirected to the identity "
                 "provider and must re-authenticate.",
    "runbook":   "SSO sessions remain valid for 12 hours. When the session "
                 "lifetime elapses the user is redirected to the identity "
                 "provider and must re-authenticate.",
    "wiki":      "A single sign-on session lasts eight hours, after which "
                 "people are bounced back to the IdP login screen to sign in "
                 "again.",
    "tokens":    "Access tokens issued by the SSO gateway expire after 60 "
                 "minutes and are refreshed silently while the session is "
                 "still valid.",
}

def embed(texts):
    r = client.embeddings.create(model="text-embedding-3-large", input=texts)
    v = np.array([d.embedding for d in r.data])
    return v / np.linalg.norm(v, axis=1, keepdims=True)

names = list(PASSAGES)
vecs = embed([PASSAGES[n] for n in names])
sims = vecs @ vecs.T

for i, a in enumerate(names):
    for j, b in enumerate(names[i + 1:], start=i + 1):
        print(f"{a:9s} vs {b:9s}  cos={sims[i][j]:.3f}")

The output is the whole problem in four lines.

Read that table as gaps rather than as levels and it stops being four numbers and becomes an experiment with two variables. Two things can differ between any pair of these passages: whether the wording matches, and whether the value matches. The handbook and the runbook share their wording and differ on the value. The handbook and the wiki differ in wording and agree on the value. The runbook and the wiki differ on both.

The wiki passage appears in two of those rows, and that is what makes them comparable. Against the handbook it scores 0.943 and against the runbook 0.938. The rewording distance is the same in both cases, because it is the same wiki sentence being compared against two sentences that differ from each other by one number and nothing else. The only variable that moved between those two rows is whether the value agrees, so the drop of five thousandths is the entire contribution that agreement makes to the score. That is the signal a conflict detector would have to work with.

Now hold the value fixed and move the wording instead. The handbook and the runbook, wording identical, score 0.981; the handbook and the wiki, saying the same thing in different words, score 0.943. Rewording costs thirty-eight thousandths, between seven and eight times what disagreeing costs, and it costs it in the direction that pushes two agreeing passages apart. A score built out of those two effects will rank a contradiction above an agreement whenever the contradiction is the closer paraphrase, and for two documents copied from a common ancestor it usually is.

The contradicting pair scores 0.981 and the agreeing pair scores 0.943, so the pair the team most needed to see ranked as the most duplicate-like thing in the set, and a detector whose job is to suppress redundancy would have quietly dropped one of them. Which one it dropped would have been an artefact of insertion order. The relationship the score is tracking is paraphrase distance, and the two documents are near-paraphrases of each other that happen to disagree about the only part anybody cares about.

The share of conflicts a wider net catches#

The natural next move is to lower the threshold, on the theory that the conflict is a slightly weaker duplicate and a wider net will catch it. Here is what that net actually catches, measured over a 61,000-chunk internal corpus with values hand-labelled on a 2% sample of the flagged pairs:

The first thing to check is that the table closes, and it does, exactly. On every row the pairs flagged equal the agreeing pairs plus the conflicting pairs with nothing left over: 1,171 and 33 make 1,204, 6,702 and 178 make 6,880, 30,612 and 838 make 31,450, and 115,610 and 3,290 make 118,900. What that licenses is narrower than it looks. The header says the values were hand-labelled on a 2% sample, so the two labelled columns are not counts of anything anybody read — they are the flag count split by the proportions the sample showed, and a total split by proportions sums back to itself by construction. The closure is arithmetic rather than evidence of a complete labelling, and it does license reading the last column as a share of the whole flagged set rather than of whatever happened to get looked at, which is what makes the four rows comparable with each other.

What it does not license is equal confidence in all four rows. Two percent of 1,204 is around two dozen pairs, and a sample that size cannot resolve a rate of a few percent at all — it would be expected to contain either no conflicts or one. Two percent of 118,900 is a couple of thousand pairs, which resolves such a rate comfortably. The evidence for flatness is therefore weak at the tightest threshold and strong at the loosest, which is the opposite of the arrangement that would let you argue the trend away, since the loose thresholds are the ones where the queue becomes unmanageable.

Two further divisions confirm the two claims made in the next paragraph, and those divisions share no step. Dividing the pairs flagged at the loosest threshold by the pairs flagged at 0.95 gives the seventeenfold multiplication of the review queue, using only the flag column across two rows. Dividing the agreeing pairs at that loosest threshold by the conflicting pairs on the same row gives the thirty-five agreeing pairs a reviewer reads for every conflict, using only the two labelled columns within one row. Two instruments, no shared arithmetic, both landing where the prose says they land — which is what licenses reading the prose as a report of the table rather than as a gloss on it.

Dropping from 0.95 to 0.85 multiplies the review queue by seventeen and leaves the share of genuine conflicts flat at under three percent, because cosine similarity is not weakly correlated with disagreement — it is orthogonal to it. A reviewer working the 0.85 queue reads thirty-five agreeing pairs for every conflict, which is a job that gets abandoned in the second week. There is no threshold at which this works, and the flatness of that last column is the reason.

The stumble worth naming here is not an arithmetic slip. It is a belief that a defence works: that the threshold is a dial, and that turning it far enough will bring the conflicts into view. That belief is tempting because it is true almost everywhere else in retrieval, where recall really does climb as a cutoff is loosened and a reviewer really does start seeing what they were missing.

What kills it is the shape of the last column rather than the size of any number in it. Across two orders of magnitude of flag volume, the share of flags that are conflicts moves by a tenth of a percentage point and does not trend in either direction. A quantity that stays constant while the dangerous property varies is the signature of an instrument that is not measuring the dangerous property at all. If cosine were even weakly correlated with disagreement, conflicts would concentrate at the top of the ranking and their share would fall as the net widened; if it were anti-correlated, the share would rise as the net widened. It does neither. That is what the word orthogonal is claiming, and the flat column is the evidence for it.

Comparing claims instead of passages#

Detection has to operate on the thing that can disagree, which is a claim: a subject, a property, a value, a unit, and the scope the value applies within. Extracting those from a passage turns "are these texts similar" into "do these two assertions about the same property have the same value", and the second question has an answer.

EXTRACT = """Extract every factual claim in the passage as JSON.
Each claim: {"subject": str, "property": str, "value": str,
"unit": str|null, "scope": str|null, "qualifier": str|null}.
`scope` is any population or condition the claim is restricted to
(e.g. "contractor accounts", "EU tenants"). Use null when the claim
is unrestricted. Do not infer facts the passage does not state.
Passage: {passage}"""

def claim_key(c):
    return (c["subject"].lower(), c["property"].lower(),
            (c["scope"] or "*").lower())

def find_conflicts(claims):
    by_key = {}
    for c in claims:
        by_key.setdefault(claim_key(c), []).append(c)
    for key, group in by_key.items():
        values = {(c["value"], c["unit"]) for c in group}
        if len(values) > 1:
            yield {"key": key, "values": sorted(values),
                   "sources": [c["doc_id"] for c in group]}

One decision inside that code does most of the work, and it is a design decision rather than a requirement. The requirement is to notice when two documents assign different values to the same fact. The design decision is what counts as the same fact, and claim_key answers it with a tuple of subject, property and scope, lowercased, with a literal asterisk standing in wherever scope is absent. Two claims are compared if and only if their keys are equal, and find_conflicts then reports a group whenever the value-and-unit pairs inside it are not all identical.

That choice has a consequence in each direction, and both of them turn up later in this chapter as failure modes. Narrowing the key — folding in the qualifier, say — means fewer pairs are ever brought together, so fewer spurious conflicts and more missed real ones. Widening it — dropping scope — means more pairs are compared, so more real conflicts found and more false ones raised alongside them. The key is the knob, and unlike a cosine threshold it moves recall and precision in a direction you can predict by reading it.

Run over the two SSO passages, the extractor produces {"subject": "sso session", "property": "lifetime", "value": "8", "unit": "hours", "scope": null} from the handbook and the same tuple with "value": "12" from the runbook. The keys match, the value sets differ, and the pair surfaces as a conflict on the first pass. Normalising units before comparison matters more than it sounds: 480 minutes, 8h, and eight hours are the same claim written three ways, and a comparison that treats the strings as opaque produces conflicts that are not conflicts. A unit-normalisation step that converts durations to seconds, currencies to minor units, and dates to ISO-8601 removed 31% of the raw conflict candidates on the corpus above before a human saw any of them.

The cost profile is different from cosine similarity in a way that shapes where you can run it. Embedding 61,000 chunks costs a few dollars and finishes in minutes. Extracting claims with a mid-sized model over the same 61,000 chunks, at roughly 400 input and 150 output tokens per chunk, is about 24M input and 9M output tokens — a few tens of dollars and several hours at practical concurrency. That is affordable once and awkward nightly, which is why extraction usually runs over a filtered slice: chunks whose documents belong to classes where numeric policy claims live, or the chunks that appear in retrieval results for real queries. Restricting to the second set on the corpus above cut it to 4,100 chunks and made it a daily job.

Both totals in that estimate are the chunk count multiplied by a per-chunk rate, which is worth noticing because it says where the only lever is. The per-chunk rates are the estimate; everything after them is arithmetic, so nothing about prompt design moves the total by an order of magnitude and reducing the chunk count is the one thing that does. The filtered slice cuts that count by about fifteenfold, which is the whole of the difference between several hours and a nightly window.

Where claim comparison fails, and what that looks like#

The failure that matters is scope collapse. A third document in the same corpus says "Contractor accounts are limited to a 4-hour SSO session", which is not a contradiction of the 8-hour rule — it is a narrower rule sitting underneath it. If the extractor drops scope, the pipeline reports a three-way conflict between 8, 12 and 4 hours, and a human burns twenty minutes discovering that one third of it was never a conflict. If the extractor keeps scope but the two documents phrase the same population differently — "contractors" in one and "non-employee identities" in the other — the keys do not match and a real conflict between two contractor rules goes undetected. Scope is where extraction is least reliable, because scope is frequently carried by document structure rather than sentence text: the passage says "sessions last 4 hours" and the fact that it is contractor-scoped lives in the H2 three paragraphs above, which chunking removed.

Both of those failures are the key doing what the previous section said it would, which is worth pointing out because it means neither is a surprise to be patched. Dropping scope widens the key, so the contractor claim joins a group it does not belong in and a false conflict is raised. Keeping scope but phrasing the same population two ways splits what should be one key into two, so two claims that genuinely conflict land in different groups and neither is ever compared with the other. One knob, two directions, and the reason it cannot simply be set correctly is that the right setting differs from one document to the next.

On a hand-labelled set of 300 candidate pairs drawn from that corpus, an extraction-and-compare pipeline caught 41 of 52 real contradictions and raised 23 false ones, giving recall of 0.79 and precision of 0.64. The eleven misses split into scope-phrasing mismatches (five), claims stated only as prose without a value the extractor would tuple ("sessions are short-lived, in line with industry norms" against a concrete 12 hours, three), and claims split across a chunk boundary so neither half carried the subject (three). The false positives were dominated by the scope-collapse case and by conditional claims — "during an incident, sessions are cut to 1 hour" — where the qualifier is the whole content of the claim.

Those two rates close against the counts, and checking that is worth thirty seconds, because a recall and a precision quoted without their denominators are two numbers nobody can argue with. Forty-one caught out of fifty-two present is the stated recall. Forty-one caught, plus twenty-three raised falsely, is sixty-four flags in total, and forty-one out of sixty-four is the stated precision. The misses close as well: fifty-two present minus forty-one caught leaves eleven, and the three categories named above — five, three and three — sum to exactly eleven with nothing unaccounted for. Two independent closures over one labelled set, sharing only the count of real contradictions, which is what licenses treating the breakdown as a description of those eleven misses rather than as a selection from them.

Reaching for a natural-language-inference model instead of structured extraction is tempting, because NLI is exactly a contradiction classifier and a DeBERTa-v3 MNLI checkpoint runs at a few milliseconds per pair on a GPU, which is cheap enough for the whole cross product of retrieved sets. On the same 300 pairs it reached recall 0.71 at precision 0.38. NLI models are trained on premise-hypothesis pairs that share a context by construction, so they read two independently authored passages as if one were offered as a description of the other, and a narrower scope reads as a contradiction with high confidence. The score is also uncalibrated across domains, which means the threshold you tune on one document class does not transfer to the next.

That comparison is given as two rates with no counts behind them, and the asymmetry limits what can be drawn from it. The recall implies how many of the fifty-two the model caught and the precision implies a total flag volume, and neither implied count is stated anywhere in the chapter. What the two rates do license is exactly the comparison the paragraph draws — the NLI model finds fewer of the real conflicts and is wrong about a much larger share of what it does flag — and both differences are large enough that the conclusion does not turn on the missing counts. What they do not license is any claim about how much review work each instrument creates, because that is a volume question and the volumes are not given.

None of this makes claim extraction the wrong tool. It makes it a recall-and-precision instrument with a review queue attached, in a place where the alternative instrument has no signal at all. What it is not is a corpus cleanliness check: it detects disagreement between two passages that a specific query brought together, and it says nothing about whether either of them is true, which of them the organisation intends to be in force, or whether a third document that never retrieved is the one that actually governs.

It is also worth being precise about which kind of disagreement this addresses, because the literature separates two that behave differently. Knowledge Conflicts for LLMs: A Survey distinguishes context-context conflicts, where two retrieved passages disagree with each other, from context-memory conflicts, where a retrieved passage disagrees with what the model absorbed during pretraining. Everything in this chapter is the first kind, and it is the kind you can instrument, because both sides of the disagreement are objects you own and can inspect. The second kind is invisible from the retrieval side: the model's parametric belief that an SSO session is typically eight hours is not a row in your index, and no comparison over your corpus will surface it. A conflict-detection pipeline that reports zero conflicts is therefore reporting something narrower than it sounds — the passages you retrieved did not disagree with each other, which leaves open that a single retrieved passage is quietly being overridden by something the model already thought.

Cosine similarity ranks by how alike two passages read, and reading alike is precisely what two revisions of one policy do. The four scores in the first table put a number on what that costs: with the wording held fixed, agreement moves the score by five thousandths, and with the claim held fixed, rewording moves it by thirty-eight. The instrument is between seven and eight times more sensitive to the property nobody cares about than to the property everybody does, and it is sensitive in the wrong direction. The threshold table then closes the escape route, because a conflict share that does not move while the flag count moves a hundredfold is a measurement with no signal in it to tune.

Detection has to run on claims — a subject, a property, a value, a unit and a scope — because those are the objects that can disagree, and what you get for that is a recall-and-precision instrument with a review queue rather than a clean test. None of it answers the question the detector raises by firing. A detector that fires on the SSO pair has done its job the moment it fires, and the system is still holding two numbers with a user waiting. Someone has to decide what happens at that point, and if nobody decides it explicitly, the behaviour is whatever the generator does with two conflicting passages in its context — which is not a decision anyone can state in advance or review afterwards.