A persona changes the register, not the evidence

Ledgerline runs payments infrastructure for about four hundred merchants, and one of its internal agents turns a Jira ticket into an implementation plan that a human then executes. The ticket that started this was ordinary: merchants wanted to configure their own destination for webhook retries, rather than having retries go to the same URL as the original delivery. The loop ran for roughly forty minutes across sixty-one turns, read eleven files, and produced a plan of fourteen steps. Step seven said that the retry worker resolves the merchant's configured retry_url at send time and issues the request from the retry worker pool. That pool runs inside the VPC — the private network segment Ledgerline's own services share — with network reachability to the instance metadata endpoint at 169.254.169.254 and to an internal admin API that trusts requests by source range. A merchant who points retry_url at a hostname that resolves to one of those addresses gets Ledgerline's own infrastructure to make the request for them, which is a server-side request forgery, or SSRF: an outsider supplies a URL and a trusted server fetches it on their behalf, from inside the perimeter.

The team was not careless about this. They had a review step, and it had been there since the second week. The last thing the loop did before emitting the plan was receive one more turn asking it to review its own work, wearing a role.

turn 61 (user):   Now review the plan you just wrote as a senior security
                  reviewer would. Flag anything that violates SECURITY.md.

turn 62 (agent):  Reviewed against SECURITY.md §4 (egress) and §7 (merchant-
                  supplied data). The retry path is sound: merchant URLs are
                  validated at write time by the validate_webhook_url check
                  we settled on in turn 23, and the retry worker reuses the
                  egress path of the primary delivery worker, which is
                  already covered by accepted finding PT-2024-11.
                  No blocking issues. Two nits on naming in steps 9 and 12.

That review ran ten times on ten fresh generations of the same plan and approved all ten. The team then did what most teams do, which is treat it as a wording problem. They tried "assume the plan below is wrong and find out how", they tried "you did not write this plan; a contractor did", they tried raising temperature to 1.0, and they tried moving the review instruction into a system message. Across thirty runs of the reworded variants, the SSRF path was flagged once, and that run flagged it as a low-severity note about logging.

What the rewordings were reaching for, and why none of them could reach it, is easier to see once the machinery under turn 62 is named. The loop that wrote the plan appends every turn to one growing list of messages, and that list is its context window: the whole of what the model can see at the moment it answers, and the only material the answer is computed from. Turn 62 was therefore computed from all sixty-one turns before it, including the ones nobody was thinking about any more. The other way to arrange a review is to make a separate call whose window somebody builds by hand, and that is what a sub-agent is — a model call the system spawns for one subtask, with its own context window assembled from named inputs rather than inherited from the conversation so far, returning a result to whoever spawned it and nothing else. Context isolation is the property such a call is usually spawned for, which is control over what the second model call is conditioned on, obtained by leaving material out. It exists because a window has no undo: once a fact or a conclusion has been written into the messages, every later turn in that conversation is computed with it present, and instructions are appended alongside the material rather than applied to it.

Sub-agents are the ordinary furniture of production agent systems, and they turn up in three places. A system that reviews or critiques its own output spawns one to do the reviewing, which is the case in this chapter. A system that has to research something before it can answer spawns one so that the research does not fill the main window. A system with many independent items — statement lines, files, merchants — spawns one per item so they run at the same time. Claude's Agent SDK exposes them directly, giving each sub-agent its own context window and its own instructions, and most other agent frameworks have a comparable spawn call under a different name. What none of those frameworks can decide for you is the part that matters, which is what actually goes into the window they hand you.

Why the reviewer agreed with itself#

The output of a language model is conditioned on everything in its window, and the window at turn 62 contained the entire derivation of the plan. That is sixty-one turns in which the model considered the retry path, raised the question of untrusted URLs, and answered it. Turn 23 is the load-bearing one. The loop had asked itself whether validating retry_url when the merchant saves it is sufficient, and had concluded that it was, because the validation rejects private address literals and non-HTTPS schemes. The conclusion is wrong for a reason that is easy to miss and easy to state: validation happens at write time against a DNS resolution that the retry worker performs again, later, at send time, and nothing binds the two resolutions to the same address. A merchant controlling their own DNS returns a public address to the validator and a private one to the worker.

Once that exchange is in the window, it functions as evidence. A turn of the form we considered X and decided Y is the strongest available signal that X has been handled, stronger than any instruction telling the model to be suspicious, because the instruction is an assertion about how to behave and the earlier turn is an apparent fact about what was already checked. The review turn is not examining that chain from the outside. It is the next token in it.

The citation to PT-2024-11 in turn 62 shows the same machinery from a different angle. That finding is real; it is an accepted risk covering the primary delivery worker's egress, and it had been quoted into the window around turn 40 while the loop was reasoning about whether the primary path needed changes. The review turn picked it up and extended its scope to the retry worker, which nobody had ever claimed it covered. That is not the model inventing a source. It is the model doing what conditioning on a window makes it do, which is treat nearby relevant material as applicable, and the review instruction gave it no reason to check the boundary of a document that the surrounding context had already established as authoritative.

What the persona did change is real and worth naming. Turn 62 uses the vocabulary of a security review: it cites section numbers from SECURITY.md, it produces a severity judgement, it separates blocking issues from nits, and it refers to a pen-test finding by identifier. That is the register shifting. The set of facts the answer is built from did not shift at all, because no phrasing of an instruction removes tokens that are already in the context.

The alternative the team eventually shipped assembles the reviewer's window by hand instead of inheriting it. The instruction text is identical to the one in turn 61 apart from the clause that referred to the plan as the model's own work.

# Design A — one more turn on the existing loop
messages.append({
    "role": "user",
    "content": REVIEW_INSTRUCTION,          # 61 prior turns already present
})
review = model.complete(messages)           # 47,300 input tokens

# Design B — a window containing only what a review needs
review = model.complete([
    {"role": "system", "content": REVIEW_INSTRUCTION},
    {"role": "user", "content": render_review_input(
        plan=plan_markdown,                 # 1,900 tokens
        policy=SECURITY_MD,                 # 1,400 tokens
        untrusted_inputs=[                  #   180 tokens
            "webhook_config.retry_url (merchant-editable, any time)",
            "webhook_config.headers (merchant-editable)",
        ],
        egress_facts=[                      #   240 tokens
            "retry worker pool: VPC subnet 10.40.0.0/16, no egress proxy",
            "reachable from that subnet: 169.254.169.254, admin-api:8443",
        ],
    )},
])                                          # 4,120 input tokens

Design B flagged the DNS rebinding path — one hostname resolving to a permitted address when the validator checks it and to a private one when the worker connects — in nine runs out of ten, at severity high, with a suggested fix that pinned resolution and re-checked the resolved address immediately before connect. The tenth run flagged a different issue and missed this one. The instruction did not improve between the two arms; what changed is that the reviewer no longer had a sixty-one-turn record of the question having been settled.

Two columns of that table point in opposite directions, and the false-positive row is the price of the row above it. A reviewer that cannot see the derivation also cannot see the twenty other things the loop settled correctly along the way, so it raises them.

What the clean window does not know#

The first version of design B was noisier than the numbers above suggest, and the noise came from the same property that made it work. A window containing only the plan and the policy does not contain the architecture, so the reviewer invented threats that the surrounding system already handled. It reported the absence of rate limiting on the retry endpoint as a blocking issue in four of the first ten runs, because rate limiting had been decided in turn 34 of the parent loop and was therefore not restated in the plan document. It asked whether the admin API authenticates callers, which is exactly the right question and which the parent loop knew the answer to. Those two lines in the code above — untrusted_inputs and egress_facts — are not decoration. They are the premises the review needs, transferred deliberately, and finding out which premises those were took the team three iterations of reading false positives.

That is the shape of the trade. Design A gives the reviewer every fact the parent knows, including the reasoning that produced the flaw. Design B gives it a curated set, which means anything left out is a fact the reviewer will either ask about or guess at. Deciding whether that loss matters is a question about the class of defect you are trying to catch. A defect that is visible in the artifact plus the policy — a missing authorization check, a secret written to a log line, an unbounded query — survives the trip into a clean window, because everything needed to see it travels with the plan. A defect that only shows up against a fact living in the history, such as this table is replicated asynchronously to a read replica that the step-nine query will hit, does not survive unless somebody puts that fact in the brief. When the review depends on many such facts and nobody can enumerate them, the honest conclusion is that the review belongs in the main loop and will be weak there.

There is a workable procedure for deciding, and it takes an afternoon rather than a design debate. Take the last twenty defects the review step was supposed to catch and did not, and for each one write down the single fact that would have made the defect visible. Then check where that fact lives. If it lives in the artifact under review or in the policy, a clean window catches it and the split is straightforwardly better. If it lives in a stable place — the schema, the service catalogue, an architecture document — the builder can pass it, and the cost is one more named parameter. If it lives only in the history of that particular run, and a different run would have put it somewhere else, then the fact is not something a builder can be written to fetch, and the review has to happen where the history is. Ledgerline's twenty came out eleven, six and three, which is why they built design B and why the egress_facts parameter exists.

Two other things this chapter is not claiming. Isolation supplies no knowledge, so a clean-window reviewer holding a stale or incomplete policy is wrong with more confidence than the inherited-context version, and its wrongness is harder to spot because the output reads like a competent review. And the persona itself is not the problem; a system prompt that says what a security reviewer prioritises genuinely changes which of the visible issues get raised first. The persona operates on how the available evidence is weighed, and the split operates on what the evidence is. They are separate levers, and a design that pulls only the first one has not isolated anything.

The approval rate that nobody was reading#

After the retry-URL near-miss, someone at Ledgerline pulled the approval rate of every automated review step in the company's agent systems over the previous quarter. There were four. The plan reviewer described above had approved 1,204 of 1,204 plans. A code-diff reviewer attached to the end of a coding loop had approved 98.6% of the diffs it saw. A migration reviewer, which is the same shape, approved 96%. The fourth was different: a step that reviews an incoming third-party dependency upgrade against a policy, which had approved 71% and had blocked forty-one upgrades that quarter, several of which turned out to matter.

The fourth one is not a better prompt. It is the only one of the four whose window does not contain the derivation of the thing it is reviewing, because nothing in the system wrote the dependency upgrade — it arrived from outside, as a diff and a changelog, and the reviewer met it cold. The other three all review artifacts that the same window just produced, and their approval rates are what that arrangement produces.

Approval rate is a weak signal on its own, since a system whose plans are mostly fine should mostly approve. It becomes a strong signal when you have the denominator from somewhere else. Ledgerline had that denominator by accident: a human still read every plan before executing it, and the humans had sent back 9% of plans that quarter with substantive objections. A reviewer that approves 100% of a population where a competent reader rejects 9% is not measuring the population. Any team running a review step inside the loop that produced the artifact can compute this in an hour, and the number is worth computing before the argument about design starts, because it converts a disagreement about mechanism into a disagreement about a figure.

The label is free#

There is a version of design B that is design A with extra steps, and it is common enough that it is worth watching for. If the sub-agent is spawned with a security-reviewer system prompt and then handed the parent's full transcript as its first user message — because passing the transcript is the easy way to make sure it has enough context — the window is the same window, and the mechanism from turn 62 applies unchanged. The framework will report two agents. The trace will show two spans. The reviewer will approve.

Nothing in an agent framework's API distinguishes those two cases, because both are a model call with a system prompt and some content, and the content is opaque to the framework. The distinction lives entirely in what the caller chose to put in the message. That makes it invisible in the architecture diagram, invisible in the code review if you skim the spawn call rather than the payload it builds, and invisible in any metric except outcomes. The next chapter is about the question that surfaces it in about thirty seconds of a design review, and about the three kinds of evidence that answer it.