Multi-turn query contextualization

Read these first

• Query transformation
• Conversation state

What you will know

• Make "what about the second one?" retrieve correctly

  The standalone wording omits the entity and constraints needed to identify the intended documents.

• Resolve coreference against conversation history before embedding

  The resolved query must preserve the relevant subject, qualifiers, and turn scope without importing unrelated context.

• Show single-turn evaluation missing this failure entirely

  A benchmark composed only of self-contained questions can report healthy retrieval metrics while never exercising omitted references.

• Compare query rewriting with history embedding on a labeled follow-up set, measuring retrieval recall, latency, and token cost

  Whole-history approaches increase context and privacy exposure, while rewriting can discard constraints or introduce model-generated errors.

• Trace a failed follow-up from chat state through rewritten query, retrieved passages, and answer, then patch the responsible stage

  A wrong answer may originate in stale conversation state, an incorrect rewrite, retrieval ranking, or generation, and the fix depends on locating that boundary.

Chapters

1. 1

   'What about the second one?' carries no retrievable signal

   After a solid answer comparing Postgres and Mongo retention policies, the user types four words and cosine search returns passages that merely sit near ordinal and list phrasing. The query vector holds no cluster, no retention and no Mongo signal, because the entity and the constraint identifying the target both stayed in the previous turn. This chapter fixes what has to be present in a query string before retrieval can do anything at all.

   Read chapter
2. 2

   A good resolution keeps the intent and drops the spent qualifier

   Turn one asked which EU regions support customer-managed keys and got Frankfurt and Dublin; turn two asks for the uptime SLA on the second one. The resolution you want names Dublin, preserves the uptime-SLA intent, and discards the encryption-key qualifier, which scoped the candidate list in the earlier turn and has since done its job. Carrying it forward drags the query toward key-management pages, so you need an explicit rule for what survives.

   Read chapter
3. 3

   A single-turn benchmark makes contextualization a no-op

   Four hundred self-contained questions reporting recall@5 of 0.91 are structurally unable to see this failure, since every question already stands alone and the contextualization stage never fires. The replacement is a labelled follow-up set whose item is a conversation prefix, the follow-up exactly as typed, and the passages that should come back, scored end to end from chat state. Follow-ups invented by someone who knows the answer yield clean pronouns and a gameable eval.

   Read chapter
4. 4

   Rewriting can invent a constraint; history embedding cannot

   On 300 labelled follow-ups, embedding the last four turns scores 0.78 with no added latency while an LLM rewrite scores 0.86 at +320ms and one extra model call. Neither the eight points nor the latency is the whole case: the rewrite is a readable artifact you can log, diff and regression-test, and it is also the only option that can silently drop or fabricate a qualifier. Whole-history embedding is lossy but never creative, and it ships more personal data downstream.

   Read chapter
5. 5

   An unscoped history welds the old entity onto a new question

   Users change subject and the assistant keeps dragging the previous product along, with logs showing rewrites like 'What is the Dublin SLA for the billing export API?' when only the API was asked about. The rewriter is obeying its instructions: it sees eight turns with nothing marking which are still in scope, so every earlier entity reads as a live constraint. Shrinking the window swaps this for silent failures on references that genuinely reach back five turns. The ambitious repair marks scope rather than length — segment the conversation into topics, or track which entities are still live and show the rewriter only those — and it breaks its own way: a segmenter cutting one turn early strands the reference the user did mean, and the resulting rewrite still reads perfectly well in the log.

   Read chapter
6. 6

   A correct rewrite with wrong passages is a ranking failure

   Chat state holds both turns, the rewrite reads 'What is the uptime SLA for the Dublin region?', the top-5 passages are all Frankfurt pages, and the answer cites them faithfully. That trace clears contextualization and generation and leaves ranking, where region names are carrying almost no weight in the embedding space. Enriching an already-correct rewrite with more conversation context degrades the query while attaching a convincing story to the same failure. The repairs are retrieval-side and priced differently: lexical or hybrid matching so a rare proper noun cannot be washed out, a metadata filter on the region parsed from the rewrite, or a reranker over a wider candidate pool.

   Read chapter
7. 7

   A wrong rewrite is often a truncated history, not a bad rewriter

   Two of the four stages the trace can indict never look like the ranking case. When chat state is at fault the rewrite is wrong for a reason the rewriter could not have avoided: the window held fewer turns than the user can see on screen, an assistant turn is missing so the antecedent was never passed in at all, or a session or ordering bug interleaved another conversation — and the tell is comparing what the rewriter received against the transcript the user is looking at. When generation is at fault the rewrite is right, the passages are right, and the answer still contradicts them, which is the one verdict no upstream artifact can explain. Naming which of the four boundaries you are at is what decides the patch.

   Read chapter