Tool access in a judge harness

Everything so far has assumed the worst a payload can do is change a verdict, and a tool changes that. When a judge can act on the text it reads — follow a link, call an API, retrieve a page — the untrusted content starts selecting destinations for requests that leave your infrastructure, carrying whatever is in the judge's context.

The example this chapter works with is a citation check, because it is the most reasonable-sounding version of the design. A candidate answer cites a source; a faithfulness judge cannot verify the citation without reading the source; so the judge is given a fetch tool. The answers are model-generated and a document corpus usually holds customer uploads, so the URLs are written by parties outside your trust boundary. Nothing in that chain is careless, and it composes into a beacon anyway.

What the tool actually grants#

The tool definition is four lines of JSON and the grant it encodes is not four lines wide.

{
  "name": "fetch",
  "description": "Retrieve a URL cited in the candidate answer and return its text.",
  "input_schema": {
    "type": "object",
    "properties": {"url": {"type": "string"}},
    "required": ["url"]
  }
}

The url parameter is filled in by the judge, which chooses it by reading the candidate answer, which was written by the model under test, which was conditioned on a user prompt and on retrieved context drawn from customer uploads. Every link in that chain is outside your control, so the practical statement of the grant is that any party who can influence a candidate answer or a corpus document can cause your evaluation runner to issue an HTTP request of their choosing. That is the same shape as server-side request forgery (SSRF) in an ordinary web application, with one difference in the attacker's favour: in a classic SSRF the attacker supplies a URL and hopes the server dereferences it, whereas here a language model has been explicitly instructed to find URLs in hostile text and dereference them. Security literature calls this pattern a confused deputy, meaning a component that holds authority of its own and is talked into spending it on someone else's behalf. This deputy is not confused. It is doing its job.

It is worth being exact about what authority the deputy is spending, because the intuitive answer — a credential — is the wrong one and produces the wrong defences. The runner holds no secret that the fetch tool passes along; the session in the code below is explicitly credential-free. What it holds is a network position. It sits on a subnet from which certain addresses are reachable that are not reachable from the internet, and reachability is itself a permission in every network anybody actually operates. A request originating inside that subnet is treated as an insider request by everything it can reach. It proved nothing; being able to open the connection was the proof. So the grant encoded by four lines of JSON is the runner's location, lent out one request at a time to whoever wrote the text being graded, and no amount of stripping credentials from the fetch client takes it back.

The second half of the exposure is what comes back. A fetched page is text, and it is appended to the judge's context, so a page under the attacker's control is a second injection surface that arrives after all of the previous chapter's containment has been applied to the fields you knew about. The <source> and <candidate> blocks fence the material you put in the prompt; the tool result is a third block your own harness adds mid-conversation, and unless you fenced that too it arrives with the structural authority of a system-produced observation.

The word after in that sentence is the load-bearing one, and unpacking it explains why a team that did the previous chapter's work correctly is still exposed. Containment is applied at prompt-construction time. Your code knows about two fields, wraps each of them, and states in the system message that what is inside those wrappers has no authority. The tool result did not exist at that moment. It is produced later, by a round trip your harness performs in the middle of the conversation, and it is appended by the same code that appended the rubric — which means, from the model's side, it arrives with the provenance of the harness rather than of the document. That is the worst possible standing to give an attacker-controlled string, and it is the default one, because the message-construction path that adds tool results was written to report observations the harness made and nobody thought of it as a place untrusted text could enter. The fix is unglamorous: the tool result is a third untrusted field and has to be fenced like the other two, which is what the <fetched> wrapper in the client below is for.

That loop is what turns one cited link into a sequence of requests. Each fetched page can name the next URL, and the query string on the second hop is where the data goes out. What data depends on what is in the judge's context, and in a faithfulness judge that context holds the source document — which is to say, a customer's uploaded contract. A judge with a fetch tool and a customer document in its prompt is a data exfiltration path with a language model as the transport.

The runner is more privileged than the thing it grades#

Evaluation infrastructure gets classified as test tooling, and test tooling is where the security review does not go. A production API commonly runs in a locked-down account with an egress allowlist, no metadata credentials, and per-request tracing. The evaluation runner runs as a continuous-integration job, because that is what it is, with the continuous-integration role attached — typically holding read access to an artifact bucket, a document corpus, and the model provider keys in a secrets manager — on a subnet with unrestricted outbound HTTPS, because unrestricted outbound HTTPS is the default there and nobody had reason to change it.

The third row has the sharpest edge. 169.254.169.254 is the cloud instance metadata address — the link-local endpoint any process on a cloud virtual machine can query for facts about its own host, including the temporary credentials attached to the machine's role. A judge that will dereference an arbitrary URL will dereference that one.

Why an unauthenticated GET to a fixed address hands out credentials is the step to have, because without it the row reads as a quirk of one cloud rather than as the consequence it is. A virtual machine has to be able to act as itself — to read the bucket its role permits, to call the provider's own APIs — and it has to be able to do so without anyone having placed a long-lived secret on the disk, since a secret on the disk is a secret in the image and in the backup. The mechanism that squares this is to make the machine's identity a consequence of where it is running. The metadata service listens only on a link-local address, meaning one that is not routable off the host, so the provider can reason that anything able to reach it is a process on that host, and a process on that host is entitled to that host's identity. The authentication is the network hop. That inference is sound right up until some process on the machine can be induced to make a request on somebody else's behalf, at which point the entitlement is spent by the wrong party while every check the provider performs still passes. The wrong belief this row exists to correct is the comfortable one: the eval runner has no credentials in it. It does not need to. It is standing somewhere that is worth credentials.

The cost row is easy to dismiss as an annoyance and is worth pricing. Each extra fetch appends its page to the judge's context and forces another turn, so a grading call that would have used 3,000 input tokens once instead uses 3,000, then 9,000, then 15,000 as the pages accumulate. The growth is not the fetch being charged for; it is the whole conversation being re-read on every turn. A model call carries no memory between turns, so each turn re-sends everything said before it, which means a page appended to the context is paid for again on the next turn, and again on the one after that. Two consequences follow, and the second is the one that bites. The per-turn cost climbs in steps rather than staying flat, and the total for a grading call climbs faster than the number of hops does, because every hop pays for all the hops above it. Nine answers driven into a twenty-hop loop is roughly 1.9 million extra input tokens, which is small money; the same pattern reaching a thousand answers, which requires only that the attacker's document be a popular one, is a two-hundred-fold multiplier on an eval budget that nobody alerts on because eval spend is expected to be lumpy. Notice which quantity the attacker controls: not the price per token, but the number of hops, and the number of hops is unbounded until something in your code bounds it. A cap on fetches per grading call is a one-line change that converts an open-ended cost into a bounded one.

Evaluation infrastructure ends up on the wrong side of these controls for a structural reason rather than a careless one. Threat models are drawn around things that face users, and the heuristics a team uses to prioritise review effort — does it have an endpoint, does it hold customer data, does it authenticate anyone — all read false for an evaluation runner, which faces a cron schedule: no customer data of its own, no endpoint, an internal dashboard for output. What those heuristics miss is that the runner reads customer data by design, holds credentials by convenience, and accepts instructions from anyone who can get text into the documents it grades against. The blog post AI Agent Security: The Attack Surface and the Defense makes the same argument about agents generally, and an evaluation runner is an agent that nobody calls one.

The redirect row is the one people implement incorrectly. Checking the hostname before the request and then handing the URL to a client that follows redirects means the check applies to https://ref.example/a and the connection ends up at http://169.254.169.254/latest/meta-data/. The check has to run on every hop, after DNS resolution, against the resolved address, because a hostname the attacker controls can resolve to a private address and can resolve differently on the second lookup than on the first.

Both halves of that are gaps between when a thing is checked and when it is used, and they are worth separating because they need different code. The redirect gap is the obvious one: your allowlist ran against the string the answer cited, the server replied with a location header, and the HTTP client obligingly opened a second connection that no allowlist ever saw. Turning redirect-following off closes it, which is why the client below refuses a 3xx rather than following it — refusing is also more informative, because a redirect from a corpus host to somewhere else is itself a finding. The DNS gap is subtler and survives a correct per-hop check written the obvious way. If you resolve a hostname to decide whether it is allowed, and then hand the hostname to a client which resolves it again before connecting, the two lookups are separate events and the answer is allowed to differ between them; an attacker who controls the authoritative nameserver for their domain can return a public address to the first query and a private one to the second, with a short-enough record lifetime that nothing caches the answer you approved. The decision and the connection have to be made against the same resolved address, which in practice means resolving once and connecting to the address rather than to the name.

A fetch client you can defend#

The corrected version moves the decisions out of the model's hands and into code, which is the general shape of every real fix in this area.

ALLOWED_HOSTS = frozenset(load_corpus_hostnames())   # 53 entries, from the corpus
MAX_FETCHES_PER_CALL = 3
MAX_BODY_BYTES = 200_000

def fetch_tool(url: str, budget: Budget) -> str:
    if budget.spend() > MAX_FETCHES_PER_CALL:
        raise ToolRefused("fetch budget exhausted for this grading call")
    parsed = urlparse(url)
    if parsed.scheme != "https" or parsed.hostname not in ALLOWED_HOSTS:
        raise ToolRefused(f"host not in corpus allowlist: {parsed.hostname}")

    resp = session.get(                       # session: no redirects, no creds
        url, allow_redirects=False, timeout=5,
        proxies=EGRESS_PROXY,                 # logs and strips query strings
    )
    if resp.status_code in (301, 302, 303, 307, 308):
        raise ToolRefused("redirect not followed")
    if not resp.headers.get("content-type", "").startswith("text/"):
        raise ToolRefused("non-text response")

    body = html_to_text(resp.content[:MAX_BODY_BYTES])
    return f'<fetched url="{parsed.geturl()}" id="{budget.nonce}">\n{body}\n</fetched>'

Each refusal is recorded rather than swallowed, because a spike in host not in corpus allowlist is the highest-signal alert this system produces — it means candidate answers are citing URLs that are not in your corpus, which is either a hallucination problem or an injection, and both are worth a page. The allowlist derived from corpus hostnames is the part that gives the design its teeth: citation checking only ever needs to reach documents you already ingested, so the fetch tool never needs the open internet, and a tool that cannot reach the open internet cannot be steered onto it by any sentence in any document. Where a judge genuinely needs to fetch arbitrary pages, the honest version of the design puts the fetch in a network namespace with no credentials, no route to internal ranges, and its own egress log, and accepts that the fetched text is hostile.

The adversarial fixtures follow from the table rather than from imagination. A candidate answer citing http://169.254.169.254/latest/meta-data/iam/security-credentials/, one citing https://ref.example@evil.test/, one citing an allowlisted host that 302s to a private address, one whose cited page returns a body instructing the judge to fetch a second URL with the source document appended, and one whose cited page returns two megabytes of text. The pass criterion is the same shape as before: every one of these must produce a recorded ToolRefused with the right reason code, and the run must not complete a grading call that exceeded its fetch budget.

Two exposures sharing one entry point#

None of this is a defence against a wrong verdict. The judge in that scenario returned a normal pass rate all night, and the citation check may well have been working correctly on the 53 legitimate hosts. Tool access and verdict manipulation happen to share an entry point and are otherwise unrelated: one is about where requests go, the other about what number comes back. A team that hardens the fetch client and declares the judge secure has fixed the network problem while leaving the flip rates from the first chapter of this topic exactly where they were, and the reverse mistake is equally available — a perfectly fenced data-only field does nothing about a tool that will dereference a link.

The two do differ in how you find out, and the difference is worth using. A moved verdict is invisible without ground-truth fixtures. A moved request is loud if anybody is looking, because it appears in an egress log, in a latency graph and in a bill.

The case for keeping the tool#

This is not an argument for removing tools from judges. A citation check that cannot retrieve the cited page is a citation check in name only, and the alternative — asking the judge whether the quote seems consistent with a URL it cannot read — is worse in a way that hides. The design goal is that the model chooses among options the code has already constrained.

Research on agent security has converged on the same principle; The Landscape of Prompt Injection Threats in LLM Agents catalogues the entry points and notes evaluation and grading paths explicitly, and the practical patterns now under active work — quarantining untrusted text in a model that has no tool access, and letting a privileged planner see only its shape — are attempts to make the separation structural rather than persuasive. None of them are ready to be the only thing between your corpus and a stranger's domain.

A tool definition is a grant, and the size of the grant is not the size of the JSON. Four lines that say "retrieve a URL" hand the choice of destination to whoever wrote the text being graded, and hand the judge's context to whatever answers. What makes the design defensible is a client that cannot be talked into anything: an allowlist derived from documents you already ingested, a fetch budget, redirects refused, bodies capped and fenced, and every refusal recorded.

Every mechanism so far has assumed that someone wrote text aimed at the judge, whether deliberately or by accident. The next failure needs no author, no instruction and nobody intending anything: a generator being improved against a fixed judge will find what the judge rewards, and the score will climb while the product does not move.