A span exists only where the model quoted instead of paraphrasing

The previous chapter specified a record with a character range in it and left open where the range comes from. Nothing upstream can supply it. A retriever returns chunks and has no idea which words inside them mattered; a reranker scores whole chunks; only the component that wrote the sentence knows which part of which chunk it was leaning on, and that component emits text. So the range has to be recovered from text: ask the generator for the supporting words, then find those words in the chunk and read the positions off. Both halves of that sentence turn out to be harder than they sound, and they fail in different ways.

An evidence string here is whatever the model returns when asked to show what supports a claim. A locator is anything that lets a machine go back and point at the same text again — in this topic, a chunk identifier plus a character range. The two are not the same object, and the gap between them is the subject of this chapter: an evidence string becomes a locator only when it is present, character for character, in text the system still holds.

Where the evidence strings went#

The first version of one evidence prompt said, in plain words, "for each statement in your answer, give the sentence from the context that supports it." Over a batch of 200 answers with 514 evidence strings between them, 61 of those strings were found in the retrieved chunks by exact string search. The other 453 were not there. Reading a sample of them makes the failure obvious: the model had understood "give the sentence that supports it" as a request for a summary of the supporting sentence, which is a reasonable reading, and it had rewritten the text on the way out.

chunk text : "Growth — included 500,000 calls/mo; overage $0.12 per 1,000 calls
              (annual contracts: $0.09)."
returned   : "The Growth plan includes 500,000 calls per month and charges $0.12
              per 1,000 calls in overage."

That string is a faithful rendering of the source and it is useless as a locator. There is no character range in the chunk that contains it, so there is nothing to store, nothing for a viewer to highlight, and nothing a later sweep can compare against. Worse, the paraphrase quietly dropped the annual-contract parenthetical, which is precisely the kind of loss a reader needs to see. The evidence field had become a second, shorter answer.

Breaking the 453 misses down by cause changes what you build. In that batch, 274 differed from the source only in typography — a curly apostrophe where the chunk had a straight one, a non-breaking space, an em dash rendered as a hyphen, a soft hyphen left over from PDF extraction. Another 108 were genuine rewrites like the one above. 47 had merged two non-adjacent sentences into one string. 24 contained text that appeared nowhere in any retrieved chunk, in any form. Those four groups need four different responses, and only the first is a matching problem.

That breakdown is worth adding up before going on, because the sum is what makes it usable. Two hundred and seventy-four typography cases, one hundred and eight rewrites, forty-seven merges and twenty-four inventions come to four hundred and fifty-three, which is exactly the number of misses. Add the sixty-one strings that were found and the total is five hundred and fourteen, exactly the number of evidence strings in the batch. Neither column leaves a remainder.

A column that closes licenses a stronger claim than a column that nearly closes. It says the four causes are exhaustive over this batch: there is no fifth failure mode hiding in a residue of unclassified strings, so a plan that answers all four answers everything that happened. It also says the four are disjoint, since overlapping categories would have pushed the sum past four hundred and fifty-three. That matters, because the four responses pull in different directions — normalisation for the typography group, output format for the rewrites, claim splitting for the merges, and permission to return nothing for the inventions — and if the categories overlapped you could not reason about them one at a time.

Making the format demand a copy#

The instruction has to make copying the only compliant behaviour, and the output shape has to make a quote a distinct object rather than a free-text field the model can fill with prose. A tool schema — the machine-readable description of the object you require the model to fill in, field by field, with a written description attached to each field — does both, because the field description travels with the field.

{
  "name": "emit_answer",
  "input_schema": {
    "type": "object",
    "required": ["claims"],
    "properties": {
      "claims": {
        "type": "array",
        "items": {
          "type": "object",
          "required": ["text", "evidence"],
          "properties": {
            "text": {
              "type": "string",
              "description": "One verifiable proposition, stated in your own words. If a sentence asserts two independent facts, emit two claims."
            },
            "evidence": {
              "type": "array",
              "description": "Empty if no retrieved passage establishes this claim. An empty array is an acceptable answer; an invented quote is not.",
              "items": {
                "type": "object",
                "required": ["chunk_id", "quote"],
                "properties": {
                  "chunk_id": { "type": "string" },
                  "quote": {
                    "type": "string",
                    "description": "Copied character for character from that chunk, including its punctuation, spacing and capitalisation. Do not reword, do not repair typography, do not join text that is not contiguous, do not trim a qualifying clause."
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Two details in that schema carry most of the improvement, and both are answers to specific rows of the breakdown rather than general good practice. Allowing evidence to be empty, and saying so where the model will read it, removes the pressure that produces the invented strings: a model told every claim must have a quote will manufacture one when it cannot find one, because the format leaves it no compliant way to say there is nothing. Splitting claims at the level of a single proposition answers the merges. A quote is scoped to a claim, so a sentence asserting two facts drawn from two places has no single supporting span, and the merge is what the model does when the format asks it for one anyway.

Rerunning the same 200 questions against that schema moved exact-match from 61 out of 514 to 402 out of 531, with 96 near misses — strings that are recognisably the source text with a character or two rendered differently — and 33 empty evidence arrays. Fabrications fell to 4. The remaining near misses are typography, and they are worth fixing in the matcher rather than in the prompt, because typography differences are also what happen when the same document is re-extracted by a different parser version.

Those figures do not partition as cleanly as the first batch did, and it is worth saying so rather than reading a closure into them. Four hundred and two exact matches, ninety-six near misses and thirty-three empty arrays come to five hundred and thirty-one, the batch total, which leaves the four fabrications with nowhere of their own to sit. They have to be a subset of one of the three named groups rather than a fourth category beside them, and the chapter this figure comes from does not say which. Take the fabrication count as a count within a group, not as an addend, and do not build any arithmetic that adds it to the others.

The chunk_id field needs validating rather than trusting, for the same reason the quote does. Models attribute a quote to the wrong chunk at a low but non-zero rate — 9 of the 531 in that batch — usually when two retrieved chunks come from the same document and the ids differ only in their offset. Since the set of chunk ids in the context window is known to the caller, an id outside that set is a hard reject, and an id inside it that fails to contain the quote is worth retrying against every other chunk in the window before giving up: if exactly one other chunk contains the quote, the citation is repairable and the mis-attribution was clerical. Retrying against the whole window costs a few hundred microseconds and recovered 7 of those 9.

The word exactly is carrying the argument in that sentence. One other chunk containing the quote identifies the true source uniquely, so rewriting the id preserves the citation's meaning. Two chunks containing it identify nothing, and picking either would be a guess dressed as a repair. Nine mis-attributions with seven recovered leaves two that stayed unresolved, and the honest outcome for those is a dropped citation rather than a chosen one.

A further prompt-only refinement is tempting and does not work: instructing the model to include character offsets alongside the quote. Models have no reliable access to character positions in text they were given as tokens, and the offsets that come back are plausible-looking and wrong — in a small trial, 11 out of 60 were within ten characters of the true position and the rest were off by hundreds. The quote is the only locator a language model can produce accurately, which is why the matching step exists at all.

An alternative worth knowing about is to move the whole problem server-side. Anthropic's Citations API takes the documents as structured input and returns spans over them directly, with the model constrained to reference locations rather than reproduce text, which eliminates the matching step entirely. It changes the failure surface rather than removing it: you still have to decide what the returned span asserts, still have to store it against a document version, and still have to check it. The rest of this topic applies unchanged whether the span came from a matcher you wrote or from an API that produced it.

Matching a quote back into the chunk#

Given a quote and a chunk_id, the job is to produce char_start and char_end over the chunk's stored text, or to refuse. The naive implementation is chunk.find(quote), and on the 402 exact matches it is correct and takes microseconds. It is the other 96 that determine the design.

The temptation is to normalise both strings — collapse whitespace, fold unicode, lowercase — and search in the normalised space. That is the stumble in this chapter, and it is tempting for a good reason: it works, in the sense that it finds the match. What it returns is the problem. Offsets discovered in the normalised string index into the normalised string, which exists nowhere in your storage, and the highlight therefore lands in the wrong place by however many characters the normalisation removed before that point. The error is not random and it is not bounded by anything you control. Every soft hyphen dropped, every run of non-breaking spaces collapsed, shifts everything after it left by one position. A PDF chunk with three soft hyphens and a run of collapsed spaces drifts by about a dozen characters over a couple of hundred, which is enough to highlight the middle of the wrong sentence — and the citation still resolves, still renders, and still looks correct to anybody who is not comparing it against the quote. Normalisation has to keep an index back to the raw string.

import re
import unicodedata

_QUOTES = {"‘": "'", "’": "'", "“": '"', "”": '"'}
_DASHES = {"‐", "‑", "‒", "–", "—", "−"}
_DROP = {"­", "​", ""}          # soft hyphen, ZWSP, BOM


def _fold(ch: str) -> str:
    if ch in _DROP:
        return ""
    if ch in _DASHES:
        return "-"
    if ch in _QUOTES:
        return _QUOTES[ch]
    folded = unicodedata.normalize("NFKC", ch).casefold()
    return " " if folded.isspace() else folded


def normalise_with_map(raw: str) -> tuple[str, list[int]]:
    """Return folded text plus, for each folded char, its index in `raw`."""
    out: list[str] = []
    back: list[int] = []
    prev_space = False
    for i, ch in enumerate(raw):
        for c in _fold(ch):
            if c == " ":
                if prev_space:
                    continue
                prev_space = True
            else:
                prev_space = False
            out.append(c)
            back.append(i)
    return "".join(out), back

The back list is what makes the result usable, and the translation it enables is worth doing one index at a time rather than quoting as a formula. A match at normalised positions [a, b) covers folded characters a through b - 1, since the range is half-open and b is the first position outside it. The raw index of the first of those is back[a], which is the start. The raw index of the last of those is back[b - 1], and the span must reach past it, so the end is back[b - 1] + 1. That gives raw [back[a], back[b - 1] + 1), a range into the exact bytes stored for that chunk.

The step that is easy to get wrong is writing back[b] for the end, by analogy with the normalised range. It fails in two ways at once. If the match runs to the end of the chunk, back[b] is off the end of the list. And when it does exist, it is the raw index of the next folded character, which skips every raw character the fold dropped between the two — so the span quietly stops short of a trailing soft hyphen or ends before a collapsed space that belonged inside it. Going through b - 1 and adding one keeps the arithmetic inside characters that survived folding, which are the only ones whose raw positions the map knows.

PDF hyphenation needs one more pass before folding, because a word broken across a line break appears as sub-\nscription in the extracted text and as subscription in anything a model reads back; a substitution of -\n before a lowercase letter, applied to the raw text at parse time rather than at match time, keeps the offsets honest for everything downstream. Doing it at parse time rather than at match time is a design decision rather than a requirement, and the reason is that the raw text is what every stored offset in the archive indexes into: repair it once, before anything points at it, and every consumer agrees. Repair it at match time and the matcher is working in a coordinate system nothing else shares.

With alignment preserved, fuzzy matching handles the residue, where fuzzy means a candidate is scored for how closely it resembles the target and accepted when that score clears a cutoff you choose. rapidfuzz.fuzz.partial_ratio_alignment returns both a similarity score and the source range of the best alignment, which is exactly the shape needed.

from rapidfuzz import fuzz

ACCEPT, REVIEW = 95.0, 85.0


def locate(chunk_raw: str, quote: str) -> dict | None:
    hit = chunk_raw.find(quote)
    if hit >= 0:
        return {"start": hit, "end": hit + len(quote), "score": 100.0,
                "how": "exact"}

    hay, back = normalise_with_map(chunk_raw)
    needle, _ = normalise_with_map(quote)
    if not needle:
        return None

    al = fuzz.partial_ratio_alignment(needle, hay, score_cutoff=REVIEW)
    if al is None:
        return None

    start, end = back[al.dest_start], back[al.dest_end - 1] + 1
    span = chunk_raw[start:end]
    ratio = len(span) / max(len(quote), 1)
    if al.score < ACCEPT or not (0.85 <= ratio <= 1.20):
        return {"start": start, "end": end, "score": al.score,
                "how": "review"}
    return {"start": start, "end": end, "score": al.score, "how": "fuzzy"}

The length-ratio guard is there because partial matching is happy to align a short needle inside a long haystack at a high score. A four-word quote can score 97 against a passage that contains those four words in a different sentence, and without the ratio check you would store a span shorter than the quote and highlight a fragment.

The two tests in that condition are worth separating, because they are looking at different things and neither can see what the other sees. A partial-match score is a measure of how well the needle aligns with some window of the haystack, and it is deliberately indifferent to how much of the haystack was skipped to find that window — which is the property that lets it locate a quote inside a long chunk at all. Being indifferent to what was skipped, it is also indifferent to a window far shorter than the quote. The ratio looks only at lengths and knows nothing about content, so it cannot tell a good alignment from a bad one, and it can tell a sixty-character window from a two-hundred-character quote instantly. Two instruments that share no step, agreeing, is what licenses storing the span; either one alone leaves a family of failures it is structurally unable to notice.

A trace from the run makes the three outcomes concrete.

claim  c3   "Growth plan overage is billed at $0.12 per 1,000 API calls."
quote       "Growth – included 500,000 calls/mo; overage $0.12 per 1,000 calls"
chunk       pricing/2025-q3-rate-card.pdf#tok:1200  (1,684 chars)
  exact find            : miss
  normalised alignment  : score 98.4, norm [231, 296)
  raw span              : [244, 310)  len 66, quote len 65, ratio 1.02
  raw text              : "Growth — included 500,000 calls / mo; overage $0.12 per 1,000 calls"
  verdict               : fuzzy accept

claim  c5   "Enterprise customers may negotiate custom overage rates."
quote       "Enterprise pricing is available on request"
  exact find            : miss
  normalised alignment  : score 88.1, norm [902, 941)
  verdict               : review  (score below 95)

claim  c6   "Support response times are contractual under Growth."
quote       "Growth includes a four-hour first-response SLA"
  exact find            : miss
  normalised alignment  : no candidate above 85
  verdict               : reject — no span, citation dropped

Read the first case one line at a time and the whole design is visible in it. The exact search misses, because the model returned an en dash where the chunk holds an em dash and the chunk has spaces around its slash. The normalised alignment scores high and returns a range in folded coordinates. Mapping that range back through back gives a raw span whose width is one character more than the quote, since the raw text contains a space the fold collapsed, and the ratio of those two lengths sits just above one — comfortably inside the accepted band, and nowhere near the failure the guard exists for. The third case never gets that far: no window of the chunk resembles the quote well enough to be a candidate, which is the correct answer for a quote about a support commitment scored against a rate table.

Claims with no single span#

Some claims are true, well-grounded, and unspanned. The answer says Enterprise customers get overage at $0.04 per 1,000 calls and unlimited seats; the rate is in the pricing table and the seat policy is three chunks away in the plan comparison, and there is no contiguous run of characters anywhere that establishes both. Fuzzy matching cannot rescue this. The matcher's whole job is to find text that differs from the target by rendering, and here the target differs from every candidate by content, because it was assembled from two sources that were never adjacent.

There are three defensible exits, and they are not equivalent.

Emitting two citations against one claim is the most common, and it is the weakest, because a reader sees a sentence with two markers and reasonably concludes that either supports it. Neither does. If you take this route, the union has to be checked as a union — the entailment test in the next chapter, which asks whether a piece of text makes a claim true, has to receive both spans as a single premise, not each span in turn — and the interface has to render both spans together rather than as alternatives. Both of those are consequences of the same fact: the thing that supports the claim is the pair, and any machinery that iterates over citations one at a time is asking a question neither one can answer.

Narrowing the claim is the version to prefer, and it is a generation-time fix rather than a citation-time one. The schema above already asks for one proposition per claim; enforcing it means the answer emits "Enterprise overage is $0.04 per 1,000 calls" and "Enterprise includes unlimited seats" as two claims, each with one span. The prose the reader sees can still be a single sentence, with two citation markers attached at the phrase level, because the claim structure and the surface text do not have to be the same object. That separation is a design decision, and it is what makes the fix cheap: nothing about the answer's readability has to be sacrificed to make its evidence checkable.

Declining to cite is the right exit for claims that are genuinely inferential — "the Growth plan is the cheapest option for a team sending 800,000 calls a month" is arithmetic over two cited numbers and is not stated anywhere. Marking such a sentence explicitly as uncited is more honest than attaching the two spans it was computed from, and it gives the reader the information that this line is the system's reasoning rather than the document's assertion. The cost is structural rather than cosmetic: uncited claims are invisible to any check that iterates over emitted citations, because they emit none, so a claim that opts out of citation also opts out of every test built on citations. That hole is real, it is the reason chapter 6 adds a check that iterates over sentences instead, and it is worth carrying in mind before choosing this exit for anything consequential.

The matcher has now done its whole job, and it is worth being exact about how small that job is. It can say that a particular run of characters exists at a particular range in a particular chunk, and that the string stored alongside the range is the string that was there. Both of those are claims about text. Neither is a claim about meaning. A quote can be located perfectly — exact match, offsets verified, width checked — and still fail entirely to establish the sentence hanging from it, because being present in the chunk and making the claim true are unrelated properties. Separating those two is what the next chapter builds tests for, and the tests only become possible because the span exists to feed them.