The fresh window wrote the script four failed attempts had poisoned

The task was a backfill on webhook_deliveries, a Postgres 15 table with 412 million rows, adding a retry_url column and populating it from webhook_config by merchant. The agent had the schema, the row counts, the migration tool's configuration, and a MIGRATIONS.md that states the two hard constraints: no statement may hold a lock for longer than two seconds, and no transaction may stay open for longer than thirty. It failed four times over about ninety minutes.

The first attempt was a single UPDATE ... FROM across the whole table, which was rejected with the comment that it would run for hours in one transaction. The second attempt batched, using SELECT ... ORDER BY id LIMIT 10000 OFFSET n, and was rejected because the offset makes every batch rescan everything before it. The third used a keyset cursor, which asks for the rows after the last id it saw instead of counting past them, and was correct in isolation, but the migration runner wraps each file in a transaction by default, so the whole loop still ran as one, and the correction said so. The fourth turned off the wrapper, kept the keyset cursor, and added CREATE INDEX ON webhook_deliveries (retry_url) at the end without CONCURRENTLY, reintroducing a long exclusive lock at the last line of the file.

At that point an engineer copied the instruction out of the failing turn, without editing a character of it, and gave it to a sub-agent along with the schema, the row counts, the runner configuration, and MIGRATIONS.md. The script that came back on the first try batched with a keyset cursor, disabled the runner's transaction wrapper with the correct pragma, slept 50 ms between batches to leave room for the replication lag budget, which is the allowance for how far the read replicas may fall behind the primary, created the index concurrently in a separate file, and added a guard that aborts if pg_stat_activity shows the migration's own backend blocking anything.

Why the fifth attempt was different from the first four#

By the failing turn the parent's window held roughly 118,000 tokens, and about 24,000 of those were the four rejected scripts and the four corrections that followed them. That content is not inert. It is the model's own prior output, which makes it the strongest available evidence about what an acceptable answer to this instruction looks like, and it sits alongside corrections that are each phrased as a local repair — replace the offset with a cursor, disable the wrapper — because that is how a person reviewing a diff talks. Conditioned on that, the natural next token sequence is a fifth local repair to the fourth script, and a fifth local repair is what the loop produced every time. The index-without-CONCURRENTLY line survived from attempt one through attempt four, because nobody had ever objected to it and there was no point at which the model re-derived the file from the constraints instead of editing the last version of it.

This is a different failure from the one people usually mean by long context. Retrieval degradation is a model failing to use something that is in its window; here the model used what was in its window with complete fidelity. A hypothetical model with perfect recall over 118,000 tokens would be worse at this task, not better, because the wrong approach would be even more firmly established. Anthropic's Context Management in Claude Code is a good picture of the window filling up and what that does to a session, and the mechanism in this chapter is the part that survives even when you are nowhere near the limit — 118k tokens is comfortable in a 200k window, and the run still failed four times.

One run is a story, so run forty#

The anecdote above is worth exactly one hypothesis, because the two arms differed in a dozen uncontrolled ways: different time of day, different sampled tokens, a fresh model call versus a fifth continuation. Establishing that the window is doing the work means constructing two arms — two versions of the same run, identical except in the one respect under test — that differ in what else is present and in nothing else.

TRACE = load_trace("run_8814")                # the failing session, frozen
INSTRUCTION = TRACE.turns[-1].content         # byte-identical in both arms

INPUTS = {                                    # everything the subtask needs
    "schema": read("schema/webhook_deliveries.sql"),
    "row_counts": {"webhook_deliveries": 412_000_000, "webhook_config": 403_912},
    "runner_config": read("migrate/runner.toml"),
    "constraints": read("MIGRATIONS.md"),
}

def arm_inherited() -> list[Message]:
    # the parent window exactly as it stood, plus the same instruction
    return TRACE.turns[:-1] + [user(INSTRUCTION)]

def arm_fresh() -> list[Message]:
    # only the subtask's inputs, plus the same instruction
    return [system(BASE_SYSTEM), user(render(INPUTS) + "\n\n" + INSTRUCTION)]

def arm_pruned() -> list[Message]:
    # inherited, minus the four rejected scripts and their corrections
    return drop_turns(TRACE.turns[:-1], TRACE.tag_ranges("failed_attempt")) \
        + [user(INSTRUCTION)]

The third arm is the one that turns a comparison into an explanation. If the fresh window wins and the pruned window wins by about as much, the failed attempts were the cause; if the fresh window wins and the pruned window does not, something else in the parent's history was responsible and you have not found it yet.

Three details of hygiene decide whether the numbers survive contact with a sceptic. The model version has to be pinned to an exact identifier for the duration, since a silent point upgrade halfway through forty runs puts the change in the same place as your independent variable. The arms have to be interleaved rather than run in blocks, because provider-side latency and load vary across an afternoon and blocked arms let that variation line up with the thing you are measuring. And the frozen trace has to be genuinely frozen — loaded from a stored artifact rather than re-generated by re-running the parent, since a re-run produces a different history and the inherited arm stops being the history that failed.

Grading has to be mechanical, because the whole exercise collapses if a human reads forty scripts knowing which arm produced them. Ledgerline's checker restored a 10-million-row fixture, ran the candidate migration under pg_locks sampling at 100 ms, and asserted four properties: the backfill completes, no lock is held over two seconds, no transaction stays open over thirty seconds, and the final row count matches. It is about ninety lines of Python and it was the most valuable thing built during the whole investigation, because it is the only reason any of the numbers below mean anything.

def grade(script_path: str) -> Result:
    db = restore_fixture("webhook_deliveries_10m.dump")   # ~40 s
    with lock_sampler(db, interval_ms=100) as locks:
        outcome = run_migration(db, script_path, timeout_s=900)
    return Result(
        completed        = outcome.exit_code == 0,
        max_lock_ms      = locks.max_held_ms(),           # must be < 2_000
        max_txn_ms       = locks.max_txn_open_ms(),       # must be < 30_000
        rows_backfilled  = db.count("webhook_deliveries", "retry_url IS NOT NULL"),
    )

Binary grading was the right call here and is not always. A script that completes and holds a 2.4-second lock is a failure by the rule and is much closer to correct than a script that never completes, and collapsing both to zero throws away information that a partial-credit scheme would keep. Ledgerline used binary anyway, because the constraint in MIGRATIONS.md is genuinely a threshold — a 2.4-second lock on that table is an incident — and because a graded scheme requires deciding weights, and weights chosen after seeing which arm scores better are not a measurement. Where the target has no natural threshold, the honest version is to define the rubric — the fixed list of scoring criteria a grader applies to every candidate, written out so that two people scoring the same script agree — and freeze it before the first run.

The inherited arm's two passes are worth noting rather than rounding away: the window does not make the correct answer unreachable, it makes it unlikely. And the gap between pruned and fresh says that the failed attempts explain most of the effect but not all of it, which matched what the traces showed — the remaining difference came from earlier turns in which the agent had been told to prefer small diffs to existing files, an instruction that was correct for the task it was given at turn 9 and actively harmful at turn 61.

Sample size deserves a sentence of arithmetic rather than a hand-wave. At n=40 and a pass rate near 0.7, the standard error on a single arm's rate — roughly how far the measured percentage would wander if you ran the same forty again with nothing changed — is about 7 percentage points, so the difference between two arms has a standard error near 10 points and differences smaller than roughly 20 points are not something forty runs can distinguish. The 77-point gap between inherited and fresh is far outside that. The 14-point gap between pruned and fresh is not, and Ledgerline correctly wrote it up as suggestive and moved on rather than building a design around it.

What this does not establish#

The comparison shows that removing the parent's accumulated context helped this subtask. It says nothing about whether the model can do the task at all, and that distinction decides whether a split is worth building. If the fresh arm had come back at 22% against an inherited arm at 15%, the finding would be that the migration is beyond what this model does reliably from these inputs, and the right response is a better checker, a worked example in the prompt, or a human writing the file. A split cannot manufacture capability; it only stops other content from competing with the task.

It also says nothing about the cases where inherited context is the reason the subtask succeeds. Ledgerline ran the same three-arm comparison on a second subtask — writing the release note for a change the loop had just made — and the inherited arm won 90% to 45%, because the release note depends on the discussion of why the change was made, and the fresh arm had only the diff. That result is as useful as the first one and points the other way. The comparison is a measurement procedure, not an argument for splitting, and a team that runs it only on the subtasks it already wants to split will find what it went looking for.

The selection problem has a cheap partial fix, which is to choose the subtasks before looking at the results. Ledgerline's second pass listed every subtask the loop performs more than ten times a day, sorted by how much of the parent's window was present when it ran, and took the top six regardless of whether anyone believed a split would help. Two of the six went the way of the migration, one went the way of the release note, and three showed no difference at either end, which is the most common outcome and the one that never gets written up. Knowing that three of six subtasks are indifferent to the window is worth as much as either of the interesting results, because it is the base rate against which the next enthusiastic report should be read.

Where this goes next is a research question rather than an engineering one. The measurement above treats the two arms as opaque model calls, which is why it needs forty runs to say anything; work on long-horizon agent serving, such as Efficient Long-Horizon LLM-Based Agent Serving, attacks the same problem from underneath by managing what an agent's context physically holds at the runtime level, sharing prefixes across sibling calls and paging state in and out. If that line matures, some of what this chapter measures by experiment becomes a property the runtime can report. It has not matured yet, and forty runs and a fixture database are what is available today.

There is one more thing the migration result does not establish, and it is the reason this is a chapter rather than the end of the topic. The engineer who ran the sub-agent copied the instruction verbatim, which is the only reason the arms are comparable at all. Most extractions in practice do not work that way: somebody moves a subtask into a sub-agent, tidies the instruction because they are looking at it properly for the first time, trims the tool list because the child clearly does not need to write files, and reports that the split worked. Three things changed and one of them gets the credit.