A good resolution keeps the intent and drops the spent qualifier

Knowing that a query has to stand on its own says nothing about what belongs in it, and several rewrites of the same follow-up can all be true, well-formed sentences about the right subject while retrieving completely different documents. Three terms make the difference sayable. The intent of a turn is what the user is asking for in it — an uptime figure, a price, a deprecation date. A qualifier is a phrase inherited from an earlier turn that narrows the question in some way. An antecedent is the earlier thing a referring expression such as the second one points back at.

The reason to be precise about which of those survive a rewrite is that the failure they cause is silent. A rewrite that keeps too much is still a true sentence, still names the right entity, and still reads correctly in a log six months later, and the only visible symptom is that the retrieved passages are about something adjacent. Chapter 1 established what a query string has to contain before retrieval can work at all; this is the question of what to put in it, and the answer is not "everything you know".

Five ways to make one follow-up searchable#

Two turns, both short, and five plausible ways to turn the second one into something searchable:

user:      Which EU regions support customer-managed encryption keys?
assistant: Frankfurt and Dublin both support customer-managed keys (CMK)
           for data at rest.
user:      What's the uptime SLA on the second one?

The first candidate rewrite is to leave the query alone, which chapter 1 disposed of. The second is What is the uptime SLA for Dublin?. The third tries to lose nothing and produces What is the uptime SLA for the Dublin region, which supports customer-managed encryption keys?. The fourth skips rewriting and searches with both turns concatenated. The fifth resolves the ordinal to Frankfurt, which is what a rewriter does perhaps one time in twenty when the list is long or the assistant's answer was prose rather than a numbered list.

Running all five against a 92,000-chunk documentation index, where the target passage is docs/sla/regions#eu-west-1, gives a clean picture of which parts of the rewrite are load-bearing.

Only one row puts the target first, and it is the row that adds the least. It names the entity, keeps the question, and stops. Every other row either adds something the second row does not have or substitutes something the second row gets right, and every one of them fails. That is a tidy result to have in front of you before reasoning about rules, because it means the rules do not need to describe what to include so much as what to leave out.

What an added clause does to the query point#

The third row is the one that surprises people, because it is the rewrite a careful engineer writes by hand when asked to preserve context. It is a true sentence, it names the right region, it reads well in a log, and it retrieves the wrong documents. Explaining that with "the extra clause confused the retriever" is not an explanation, and the mechanism underneath it generalises well beyond this conversation.

Start with what an encoder does to a sentence. It reads the whole string and returns one point, with no structure inside that point corresponding to the clauses that went in — there is no field holding "uptime SLA" and a second field holding "customer-managed keys". Every token participates in producing one set of coordinates, and the CMK clause contributes as much to that point's position as the SLA clause does. That single fact is where the intuition people bring to this goes wrong, and the intuition is worth naming, because it is imported from a system that works differently. In lexical filtering, adding a term narrows: a conjunctive term can only remove candidates that lack it and can never introduce one. A dense query has no candidate set to narrow. It has a position, and an added clause changes the position.

Movement is the whole of it. The point for What is the uptime SLA for the Dublin region, which supports customer-managed encryption keys? sits somewhere between the region of the space occupied by text about uptime commitments and the region occupied by text about key management, because both clauses pulled on the same coordinates. Moving toward one of those regions is moving away from the other; there is no direction in which the query can hold both at full strength. What comes back is whatever sits nearest to the in-between position that results.

Which side of the line wins is then decided by how the two regions are populated. The corpus contains a great deal of writing about customer-managed keys and comparatively little about Dublin's uptime commitment, so the crowded side supplies most of the near neighbours of a point sitting between them, and the search lands in the security neighbourhood. security/cmk-overview#0000 comes back first and the target falls to rank 8.

That is the general result, and it is worth stating in a form that settles cases nobody has measured. A qualifier that has finished its work is not neutral ballast in a query; it is a displacement, and the size of the displacement is set by how much of the corpus is written about the qualifier's topic. The better documented the thing the user has stopped asking about, the more damage carrying it forward does — which is exactly backwards from the instinct that a well-documented product is a forgiving one to search. The CMK clause did real work in turn one, where it narrowed all EU regions down to two. By turn two that work is finished, because the answer to which region we are talking about is now the literal token Dublin.

The fourth row is the same mechanism with the volume turned up. Concatenating both turns keeps every word of the CMK question, including the words that made turn one a question about encryption at all, so the point lands further into the security neighbourhood than the third row's did and the target falls further, to rank 14. Concatenation is not a conservative choice that declines to decide. It is a decision to weight the earlier turn as heavily as the current one.

The failure that no downstream check can see#

The last row deserves attention for a different reason, because its failure is invisible in exactly the way the third row's is not. Resolving the ordinal to Frankfurt produces a query that is well-formed and gets an excellent answer to it: docs/sla/regions#eu-central-1, the Frankfurt SLA page, comes back first, and the reply is confident and accurate about a region the user did not ask about. The Dublin page is still in the same result list at rank 6 and nothing will select it, because the query the passages were ranked against says Frankfurt.

Work out what any downstream check would see. Passage relevance is measured against the query, and the passages are relevant to the query. Citation faithfulness is measured against the passages, and the answer is faithful to them. Groundedness, hallucination detection and a human spot-check of "does the answer follow from the cited page" all return the same verdict in the world where the ordinal resolved correctly and the world where it did not. The quantity every one of those instruments measures is constant while the dangerous property varies, which means none of them is a weak check on this failure — none of them is a check on it at all.

That is a stronger reason to log the rewrite than convenience. The rewrite is the only artefact in the whole request that differs between the correct world and the wrong one, so it is the only place the error is recoverable from, which is why chapter 6 spends its time on traces and why the rewrite has to be logged in a form you can read.

Three rules, in the order they apply#

The resolution you want is produced by three separate decisions, and separating them makes it possible to say which one a bad rewrite got wrong.

Every referring expression in the current turn is replaced by its antecedent from the transcript. That covers pronouns, ordinals like "the second one", bare definite phrases like "the enterprise one", and demonstratives like "that limit". The antecedent may live in a user turn or an assistant turn, and in practice it lives in an assistant turn more often, because the assistant is the one producing lists.

The intent of the current turn survives intact. Whatever the user is asking for now — an uptime SLA, a price, a deprecation date — is the point of the query and must appear in the rewrite in words the corpus would use. This rule sounds too obvious to state until you watch a rewriter that has been told to "use the conversation history" produce a rewrite that reads like a restatement of turn one.

Qualifiers inherited from earlier turns survive only if they still constrain the answer to the current question. This is the rule that decides the CMK clause, and it needs a test rather than intuition.

The order is forced rather than chosen, and seeing why is worth a moment. The third rule asks whether removing a qualifier would change which document answers the current question, and that test cannot be run until you know what the current question is about. Before the first rule has fired, the current turn still says the second one, and a test asking which document answers a question about "the second one" has nothing to hold fixed. The second rule supplies the other half of the same test, since "which document answers this" depends on what this turn is asking for. So each rule produces an input the next one needs, and an implementation that evaluates them in any other order is evaluating the third rule against a question it has not finished reading.

Rule 1 has a case that trips implementations written with pronouns in mind, which is the follow-up whose antecedent is a proposition rather than an entity. "Why not?" after an assistant turn saying that Dublin does not support cross-region read replicas refers to the whole claim, and the only rewrite that retrieves anything useful is one that restates it, along the lines of Why does the Dublin region not support cross-region read replicas?. A resolver that looks for a noun phrase to substitute finds nothing to substitute and returns the query unchanged, which is the worst outcome available, because the pipeline then embeds two words. Treating rule 1 as "replace the reference with what it points at, whether that is a name or a statement" covers both cases with one instruction.

The test for a spent qualifier

Ask whether removing the qualifier would change which document correctly answers the current turn. Dublin's uptime SLA is Dublin's uptime SLA whether or not Dublin supports customer-managed keys, so the CMK clause changes nothing about the target document and is spent. Compare that with a conversation where the user established in turn one that they are on the enterprise plan and now asks "what's the rate limit?" — the rate limit document differs by plan, so enterprise is still doing work and must survive into the rewrite. The same applies to version numbers, environments and tenancy: on 3.2, in staging and for our single-tenant deployment all narrow the set of correct documents and belong in the rewrite until the user changes them.

The test is deliberately a counterfactual about documents rather than about meaning. It does not ask whether the qualifier is true, and it does not ask whether the user still cares about it; both of those questions lead into the user's head, where the answer is not available. It asks whether the correct target changes, which is a question somebody holding the corpus can settle without holding the conversation.

A useful way to hold the distinction is that some qualifiers select and some constrain. A selecting qualifier picks the item out of a set, and once the item has been named the qualifier has spent itself; a constraining qualifier narrows which document about that item is correct, and it persists until it is contradicted. Which EU regions support CMK selected Dublin. Enterprise plan constrains every rate-limit question the user asks for the rest of the session.

What the rewriter should return#

A rewriter that returns a bare string throws away the two things that make failures diagnosable later, which are what it resolved and what it decided to drop. Given that the rewrite is the only artefact distinguishing a correct resolution from a confidently wrong one, throwing away its internal decisions discards the evidence at the one point where it exists. Structured output costs nothing extra at generation time and turns the rewrite into a record.

CONTEXTUALIZE = """You rewrite the user's latest message into a standalone \
search query for a documentation index.

Rules:
1. Replace every pronoun, ordinal and definite reference with the entity it \
refers to, using the exact name that appears in the transcript.
2. Keep what the user is asking for in this turn. Do not answer it.
3. Carry a qualifier from an earlier turn ONLY if removing it would change \
which document answers this turn. Qualifiers that were used to pick an item \
out of a list are finished once you have named the item.
4. If the latest message already stands alone, return it unchanged and set \
rewritten to false.

Return JSON: {"query": str, "rewritten": bool,
              "resolved": [{"surface": str, "antecedent": str, "turn": int}],
              "dropped": [str]}"""

def contextualize(history: list[Message], latest: str) -> Rewrite:
    resp = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=300,
        system=CONTEXTUALIZE,
        messages=[{"role": "user",
                   "content": render_transcript(history) + f"\n\nLATEST: {latest}"}],
    )
    return Rewrite.model_validate_json(resp.content[0].text)

On the Dublin conversation this returns a record that is worth more than the string alone:

{
  "query": "What is the uptime SLA for the Dublin region?",
  "rewritten": true,
  "resolved": [{"surface": "the second one", "antecedent": "Dublin", "turn": 2}],
  "dropped": ["customer-managed encryption keys"]
}

The resolved array is what lets you check an off-by-one ordinal without reading the whole transcript, and the dropped array is what lets you find out, six weeks later, that your rewriter has been quietly discarding enterprise plan on every pricing follow-up. The fourth rule and the rewritten flag are the part of this that looks like housekeeping and is not.

Resolving into the vocabulary the index contains#

Rule 1 says to use the name that appears in the transcript, and there is a case where that is not enough. The assistant said "Dublin" because that is what humans call it; the SLA table is keyed by eu-west-1, and the phrase "Dublin" appears in that document once, in a parenthetical. A rewrite of What is the uptime SLA for the Dublin region? still retrieves it at rank 1 in a dense index because the parenthetical is enough for the encoder, but the margin is thinner than it looks — the same query against a BM25 index puts it at rank 19, behind every marketing page that mentions Dublin.

The gap between those two rankings comes from what each scorer does with a single occurrence. BM25 scores term by term, weighting a term by how rare it is across the collection and by how often it occurs in the document being scored, so one parenthetical "Dublin" in a long SLA page earns a small contribution while a marketing page that repeats the city name earns a large one. A dense encoder does not score terms; it produces one point for the whole passage, and a passage that is about uptime commitments and mentions Dublin once lands near a query about Dublin's uptime commitment. The single occurrence is doing very different amounts of work in the two systems.

That result carries a condition worth stating, because the whole dense ranking rests on it: the human-facing name has to appear in the target document at all. One parenthetical is enough, and zero is not. Where a document is keyed purely by an internal identifier and never spells out the name a user would type, no phrasing of the user's name reaches it and only the alias does. Emitting both surface forms, as in What is the uptime SLA for the Dublin (eu-west-1) region?, moves it to rank 1 in both indexes and is the form that survives that case as well.

Doing that requires the rewriter to know the alias table, which means either putting a short glossary in the system prompt or resolving aliases in a deterministic post-step. That is a design decision rather than a requirement, and it has two criteria. The post-step is cheaper and does not consume model attention, and it is also auditable, which the model's vocabulary choices are not.

Coreference resolution as computational linguistics does it#

Resolving referring expressions against prior text is one of the oldest tasks in computational linguistics, and there are decades of systems for it — mention-pair classifiers, the CoNLL-2012 shared task, and later neural end-to-end models that score spans against candidate antecedents. Pointing an off-the-shelf coreference resolver at a chat transcript gets you part of rule 1 and none of rules 2 and 3. Those systems are built to link mentions to spans that exist in the text, so they can connect "it" to "Dublin" when Dublin is a noun phrase in a preceding sentence, and they have nothing to say about a predicate that was never written down. The follow-up "and for staging?" contains no mention to resolve; what is missing is the verb phrase, and reconstructing it is a generation problem rather than a linking problem.

The current research direction runs the other way, treating the history as the thing to fix before the query is written at all. CHIQ, which is worth reading if you want the ablations — the experiments that remove one component at a time to show what each was contributing — enhances the conversation history first — resolving ambiguity within it, summarising it, and pruning turns — and only then rewrites, and reports that a well-prepared history lets a small open model match rewriting quality that otherwise needed a much larger one. The practical takeaway for a production system is that the rewriter's input is a lever equal in size to the rewriter's prompt, which is a thread chapter 5 picks up when the history stops being short.

An honest limitation of the three rules is that rule 3's test is not always decidable from the transcript. When a user asks about Dublin's uptime SLA after a security conversation, it is genuinely possible that they mean the SLA that applies to CMK-enabled deployments, if such a thing exists in the corpus. The rule resolves the common case correctly and the ambiguous case arbitrarily, and no amount of prompt engineering removes the ambiguity, because it is in the user's head. What you can do is make the arbitrary choice visible in the dropped array so that when the answer is wrong, the reason is one field lookup away.

The five rewrites at the top of this chapter differ from each other in ways that a reader would call trivial, and they span the whole range from a usable answer to a confident answer about the wrong region. What separates them is not care or fluency: the most careful of the five, the one that preserves everything, is the one that lands in the security neighbourhood, because in a dense index every word in the query is a vote about where the query sits. The rewrite that wins names the entity, keeps the intent of the current turn, and drops the qualifier that has already done its job.

Having a rule and a structured output tells you what a correct rewrite looks like for a conversation you are staring at. It says nothing about whether the rewriter you deployed produces correct rewrites across the traffic you actually get, and the retrieval benchmark already in the repository cannot answer that question either, for a structural reason worth being precise about.