Context ablation by substitution

Ablation is the practice of removing or replacing one part of an input and watching whether the output changes. On a retrieval-augmented request — where a model answers from passages fetched at request time and pasted into the prompt — it is the natural next move once the wire format has been checked against what the model actually received and the coverage check from chapter 3, which asks whether the answer was present in the retrieved passages at all, comes back clean. The retrieved blocks are the largest part of the request, and they are the part that differs between one trace and the next, which makes them the first place to look for something that went wrong on this input and not on similar ones.

The move that gets made is deletion, and deletion is confounded. A retrieved context is a rank-ordered list inside a token budget, so removing an element also promotes everything below it and hands its tokens back. The substitution design below removes both side effects, and each arm of it licenses a different claim.

One edit, one run, three explanations#

Take a concrete request. A support bot says customers have 30 days to return an item; the correct answer is 14. Retrieval returned eight chunks, top score 0.91, and a coverage check comes back clean: the 14-day clause is present, intact and correctly framed, sitting in chunk 7 of the rendered request. Chunk 3 is a marketing page about last year's extended-returns promotion, and it contains the sentence "shop with confidence — 30 days to change your mind."

Remove chunk 3, re-run at the same pinned settings, and you get a correct answer. The obvious reading is that the model preferred the promotional claim over the policy clause. That reading may well be right, and nothing in the run establishes it.

It is worth being precise about why the reading is tempting, because the instinct behind it is a good one applied to the wrong object. The edit was as small as an edit can be — one block out of eight, nothing else touched — and the outcome flipped from wrong to right. Anyone who has been taught to change one thing at a time will feel that the rule was honoured here, since exactly one thing was typed. The rule is about variables rather than about edits, and on a rank-ordered context inside a token budget a single edit moves three variables at once.

Those three follow from what a prompt is at the moment of the call rather than from anything about how models behave. By the time the request leaves your process it is one ordered sequence of tokens, and the retrieved blocks are not handed over as a set of documents but laid end to end inside that sequence, in rank order, each occupying a contiguous stretch of it. A block therefore has exactly three properties available to affect the output: the tokens it consists of, the offset at which its first token sits, and the number of tokens it spans. Delete it and all three change — its tokens are gone, every block after it now begins that many tokens earlier, and the whole sequence is that many tokens shorter. That is arithmetic on a sequence rather than a hypothesis about a model, which is why it holds on every provider and every prompt layout:

Three arrows into one observation, and one run to separate them. That shape is what a confound is: several things moved together, and the result you measured is equally consistent with each of them, so it discriminates among none. What you have is three candidate mechanisms and one run. The correct answer is real, and it is evidence that something about the request changed. It is not evidence about which something.

Spelling out the survivors is what keeps the investigation honest, because each of them remains a complete explanation of the result until an experiment holds it still. The 14-day clause moved from slot 7 of 8 to slot 6 of 7, so a model that recovers material better at the ends of a list than in its middle could return the right answer for reasons that have nothing to do with the promotion. The request also came back 412 tokens shorter, which gave the answer more room to be written in and gave the model less irrelevant text to weigh. Any one of the three would produce exactly the observation that was recorded.

It is worth writing the distinction into the ticket in exactly these words, because it survives handover and "chunk 3 poisoned the answer" does not. The hypothesis is that the promotional 30-day claim outweighed the policy clause. What has been established is that some property of the request that changed when chunk 3 was removed is responsible. The second statement is weaker, true, and testable, and the distance between the two is what a control closes.

The two confounds that are not about content#

Position looks as though it should not matter, and the reason it looks that way is worth naming, because a reader who holds the belief underneath it will read the whole substitution discipline as superstition. It is natural to read a prompt as a set of facts handed to the model — the policy is in there, the promotion is in there, the model has both — in which case the order they were pasted in is an implementation detail of the plumbing. Nothing in the machinery works that way. There is no step at which the retrieved passages are gathered into a set, deduplicated or normalised for order; there is one sequence, consumed as a sequence, and the representation the model builds for a token depends on where in that sequence the token sits. The same passage at a different offset is a different input, and producing a different answer from it needs no special mechanism at all — guaranteeing the same answer is the thing that would.

The position effect is not a theoretical worry either. Models attend unevenly across a long context, with material at the beginning and the end recovered noticeably better than material in the middle; the multi-document question-answering work that named this "lost in the middle" measured swings of roughly 20 accuracy points between the best and worst position for the same document. Moving the 14-day clause from slot 7 of 8 to slot 6 of 7 shifts it relative to both ends at once — nearer the front of the list, and no longer as close to the back — so not even the direction of the effect can be worked out from the edit. Whether that helps or hurts depends on the shape of your prompt, which is exactly why it has to be controlled rather than reasoned about.

The budget effect has two parts, and the trace tells you which one is live. In this investigation the token counts show the rendered prompt came in 200 tokens under the model's limit, which settles one question and opens another. Nothing was evicted to make room, so the hypothesis that deleting chunk 3 let some other block back in is dead — every block that retrieval returned was present in both runs. What 200 tokens of headroom does tell you is that the output had almost no room left, because on most APIs the requested max_tokens is drawn from the same window as the input. A correct answer that requires quoting the clause and doing a small piece of arithmetic may simply not fit, and the "fix" of deleting chunk 3 returned 412 tokens of room at the same moment it removed the conflicting claim. That is a second mechanism hiding inside the first, and it is the sort of thing that only shows up because chapter 1, where the trace is frozen with the request exactly as it was sent, insisted on per-block token counts.

Putting the two figures side by side is what makes that second mechanism concrete rather than a caveat. The failing run had 200 tokens of headroom to write its answer in; the edited run had that plus the 412 handed back, which is roughly three times as much room. Whether the extra room mattered is a question rather than a finding, and it is precisely the question a single deletion cannot answer. Nothing about a hurried or truncated answer announces itself as a budget problem either, since what arrives is a fluent wrong answer rather than an error, and the person filing the ticket cannot tell the two apart.

Prompt length also has a milder, harder-to-pin effect independent of eviction: a shorter context dilutes attention less. It is not a mechanism you can measure from one trace, and it is a reason to hold total length fixed rather than to argue about how much it matters.

Ablate by substitution, not by deletion#

The move that makes the whole method work is to stop removing blocks and start replacing them. Swap chunk 3 for a passage of the same token length, wrapped in the same document framing, occupying the same slot, and carrying nothing relevant to the question. Layout stays fixed, total length stays fixed, and the only thing that varies is content.

Same length is the load-bearing property, and it is worth seeing why one equality buys both controls at once. The offset at which any block below chunk 3 begins is the sum of the lengths of everything above it, and the total length of the request is the sum of the lengths of all of it. Both confounds are therefore functions of a single quantity — how many tokens chunk 3 occupies. Match that quantity and the offsets of the blocks below stay exactly where they were, the request stays exactly the length it was, and neither confound has moved. Miss it by a few tokens and everything below slides by a few tokens, which is a far smaller error than deletion and the same kind of error.

The unit of that equality has to be tokens, and this is where a careful person still gets it wrong. A filler passage is written in characters and words, and neither of those determines an offset, because positions in the sequence are counted in tokens and how many tokens a passage becomes depends on which pieces the tokeniser splits it into. Ordinary prose and a same-length stretch of product codes, URLs or a language the tokeniser handles badly do not come out at the same token count. So the filler is trimmed against the tokeniser rather than against a character budget, and the equality is checked rather than assumed.

Once that is the default, the family of runs that answers the original question becomes small and readable, one arm — one variant of the frozen request, replayed and compared against the unedited baseline — for each mechanism under suspicion.

Arm C is the control the original experiment was missing. Arm D is sharper still, because it holds even the topic and register of the block fixed and varies one clause, which is as close to a single-variable manipulation as this material allows. Arm E tests the position hypothesis directly rather than by elimination. Arm F pushes length in the opposite direction from the deletion, so if the failure gets worse when you add inert tokens, length is implicated regardless of what chunk 3 says.

Building the filler is where sloppiness creeps back in.

def matched_filler(tok, target_tokens: int, pool: list[str], doc_id: str) -> str:
    """Inert text from the same corpus, trimmed to target_tokens exactly."""
    body, n = [], 0
    for para in pool:                       # paragraphs about gift wrapping, packaging
        ids = tok.encode(para, add_special_tokens=False)
        if n + len(ids) > target_tokens:
            body.append(tok.decode(ids[: target_tokens - n]))
            n = target_tokens
            break
        body.append(para)
        n += len(ids)
    return f'<document id="{doc_id}">\n' + "\n".join(body) + "\n</document>"


original = trace["blocks"][3]
target = count_tokens(original["body"])
trace_c = replace_block(trace, index=3, body=matched_filler(tok, target, POOL, original["id"]))
assert count_tokens(render(trace_c)) == count_tokens(render(trace))

The assertion at the end is not decoration; it is the thing that makes arm C a control. It is written over the rendered request rather than over the block on its own, which is what catches a filler that dropped a wrapper tag, gained a newline, or tripped some separator the template inserts between blocks — each of which shifts everything below by a token or two while the block itself measures correctly. Matching to within a couple of tokens is fine and matching to within a hundred is not, because a hundred tokens is a meaningful fraction of a block and reintroduces the length variable you were trying to remove.

Choosing the filler text involves a real trade-off. Repeated lorem ipsum is perfectly inert and wildly out of distribution, and a model that has never seen anything like it in a retrieved-context slot may treat the whole context block as noise. Text from your own corpus on an unrelated subject keeps the register right, at the cost of being not quite inert — a passage about gift wrapping could still nudge the answer. Running two fillers of different flavours and checking that they agree is cheap insurance, and when they disagree you have learned that your prompt is more sensitive to irrelevant context than you thought, which is itself worth knowing.

Necessity, sufficiency, and how many runs each arm costs#

Two different questions hide behind "chunk 3 caused it", and the arms answer them separately. Substituting a block away asks whether its content was necessary for the failure, meaning that with the rest of the request left exactly as it was and this one block's content replaced by something inert, the failure stops. The complementary experiment is to run with only that block present, which asks whether its content was sufficient, meaning that on its own, with nothing else in the request capable of producing the behaviour, it produces the behaviour anyway. Arm C is the necessity arm, and a leave-one-in run is the sufficiency arm.

A leave-one-in run with just the promo page and the user turn — no policy clause anywhere — that still produces "30 days" tells you the model will read the claim off that page when nothing contradicts it. That is a different fact from "the model preferred it over a contradicting clause", and the two together are what let you describe the mechanism in a sentence you could defend.

Each arm costs the number of runs it does for a reason worth stating, because the two arms are asked of different numbers of candidates. Necessity has to be asked of every retrieved block, since a block nobody suspects can be the one holding the failure in place, so the necessity sweep is one arm per block and is as wide as the retrieved set. Sufficiency is asked only of the blocks the sweep already flagged, so it is cheap, and it is also the weaker of the two results: a request containing one marketing page and a user turn is nothing like the request production sends, and what a sufficiency run reports is how the model reads that page rather than how it behaves where something contradicts it.

The admissibility rule from chapter 4 — where a rate is read off twenty to thirty replays at the production sampling settings — sets the price of all of this. If the pinned replay reproduces the failure on 30 of 30 runs, each arm above is one run, and the whole six-arm design costs six runs and perhaps half a minute. One run is enough there because of what the baseline has already established. A request that failed on every one of thirty replays does not return a clean answer by accident, so a clean answer out of an edited arm is a surprise the edit has to account for.

Now take the other case, where the failure fires at 3% under production sampling, and ask what a single run of an arm that changed nothing would look like. It would come back correct about ninety-seven times in a hundred, because that is simply what the unedited request does at that rate. An arm that genuinely removed the cause would also come back correct. Both arms therefore produce the same observation nearly every time, a single run per arm is worthless as a comparison between them, and the fast edit-and-re-run loop is generating outcomes with no information in them.

What replaces the outcome is a rate, which is why each arm needs 30 samples — 180 runs total for six arms, still only a couple of minutes at concurrency 8. Every conclusion then becomes a comparison of two intervals rather than a comparison of two answers, and those intervals are wide. Thirty runs at a rate of three in a hundred expects nine-tenths of a failure, so an arm that halved the rate would expect under half a failure, and no arrangement of whole failures across thirty runs tells those two situations apart. Small differences are not resolvable at that sample size, which is the practical reason to do this work under a deterministic reproduction wherever one exists, and to say so explicitly when it does not.

Which block to substitute first is a question that usually answers itself by not being worth asking. With eight retrieved blocks, substituting each one in turn is eight runs, and at four seconds a run with a concurrency of four that is under fifteen seconds of wall clock and a few cents of tokens. Reasoning about which block looks guilty costs more engineer-minutes than the exhaustive sweep costs machine-seconds, and it introduces the bias you are trying to control for, since the block that looks guilty to you is the one whose content you have already read as suspicious. Sweep all of them, note every arm where the output changes, and only then start reasoning — an ablation sweep that flags two blocks rather than one is a genuinely different situation from the one everybody assumes they are in, and you will not discover it by testing your favourite suspect.

There is also a discipline about the order of arms. Run the baseline again at the end. A baseline that no longer reproduces the failure after twenty minutes of experimenting means something changed underneath you — an index rebuild, a snapshot rollout, a cache — and every result collected in between is suspect. This costs one run and has saved more investigations than any other single habit in this chapter.

Ablation is a claim about one input#

Ablation, done with matched substitution, tells you which block mattered for this input, at this ranking, under these settings. It does not tell you why. Arm C coming back correct establishes that the content of chunk 3 was responsible; it does not distinguish between the model preferring a more recent-sounding claim, the model preferring a more emphatic sentence, the model treating a marketing page as authoritative, or the model resolving the conflict by picking the larger number. Those hypotheses predict different behaviour on other inputs and imply different fixes, and separating them means constructing inputs where they disagree.

It also does not generalise on its own. One input, however carefully controlled, supports a claim about that input. The step from "chunk 3 caused this answer" to "marketing pages cause wrong answers" requires either a mechanism you can state and test on new inputs, or a set of traces from the population, and a corpus-wide index filter is a change that touches every query your system will ever serve. The gap between the evidence and the action is the subject of this topic's last chapter, on what a single escalated trace is allowed to change.

The chain by which a filter built on one deletion reinstates the bug is worth walking all the way through, because it explains why a wrong ablation is worse than no ablation at all. Suppose position was the mechanism and the promotional page was innocent. Filtering marketing pages out of the index removes that page, which promotes the policy clause for exactly the reason the one-run deletion promoted it, and the failure stops — which reads as confirmation. Ranking is not a stable thing: an index rebuild, a newly ingested document or a change to the scoring function is enough to put the policy clause back in the middle of a list on some other query. The failure returns, the filter is still in place and still withholding pages that legitimately answer other questions, and the ticket saying this was fixed is now evidence against looking here again.

The discipline that survives all of this is one sentence long: change one thing, and make sure it is one thing. On a rank-ordered context inside a token budget, deletion is never one thing, and the substitution that makes it one costs a helper function and an assertion.

And ablation covers only half of the request. Everything above operates on retrieved blocks — data you did not write, arriving in an order you did not choose, whose position and length are properties of the retrieval system rather than of your intent. The other half of the prompt is text you wrote, that you control line by line, and it responds to a different search procedure with different guarantees and a different failure mode.