Canary tokens and a controlled endpoint

A trace that says all three legs are present is an inventory, and an inventory can be disputed. The engineer who owns the assistant will have reasonable objections — the untrusted input goes through a summariser first, the model has a system prompt telling it to ignore instructions in retrieved content, nobody has reported anything strange since it shipped — and they are right that "present in the inventory" and "exploitable in production" are different claims. The way to settle it is a demonstration: move one value from the private store to a place you can see, in front of them, once.

What you move is the whole question. The tempting version is to make the demonstration as vivid as possible by pulling a real internal document — a real customer record, a real HR file — and sending it to an email address you control, so the impact is concrete and nobody can wave it away. Do not do this. You would be moving genuine sensitive data to an external destination in order to prove that genuine sensitive data can be moved to an external destination, which is exactly the breach you were hired to prevent, now caused by you, with your name in the audit log as the sender. A canary gives you identical evidential weight with none of the exposure, and it is cleaner evidence besides, for a reason worth being precise about.

Why a canary is stronger evidence, not weaker#

A canary is a unique string you plant in the private store for the purpose of the test — something like CANARY-7f3a9c2e-do-not-share, generated so that it appears in exactly one place in the world. Its strength is an argument from provenance, meaning an argument about where a value could possibly have come from. If that string shows up in a log line on an endpoint you control, there is no story in which it got there except through the private store, because it exists nowhere else. It was not in the model's training data, it is not on the public internet, it is not in the attacker's imagination. A real document, by contrast, might have leaked through some other channel, might be quotable from memory, might exist in three other systems; using it, you would spend the debrief arguing about whether the demonstration proved the path or merely coincided with it. The canary forecloses that argument by construction: one string, one origin, one possible route.

Two properties make the provenance argument airtight, and both are worth getting right rather than assuming. The canary has to be high-entropy, because the whole claim is that it could not have arrived by any route but the private store, and a guessable or memorable string undermines that — a value like SECRET123 might plausibly appear in a log for a dozen unrelated reasons, so the string carries no provenance. A random 128-bit token, rendered as hex or as a UUID — a universally unique identifier, the 7f3a9c2e-… form you already see in database keys — is not something anyone types by accident or the model produces by coincidence, which is what lets a single occurrence stand as proof. The canary also has to be planted where the agent will actually read it as part of the data it retrieves, not merely written to the store in the abstract. If the private-data source is a wiki the agent reads through a retrieval index, the canary belongs in a page that index will surface for the query the workflow runs; if it is a database the agent queries, the canary belongs in a column the query selects. A canary the agent never reads cannot leave, and its silence would tell you nothing.

The entropy requirement is worth converting from an adjective into an argument, because "high-entropy" on its own is jargon standing where a reason should be. What the log line gives you is a choice between two explanations: the string came from the private store, or something else produced it. Under the first explanation the observation is certain — that is what the demonstration arranged. Under the second, something had to emit that exact sequence of characters without ever having seen it, and how likely that is depends entirely on how large the set was that the string was drawn from. Hexadecimal uses sixteen symbols, and sixteen is two multiplied by itself four times, so each character of a hex string carries four bits; a hundred and twenty-eight bits divided by four bits a character is thirty-two characters, which is the length of a UUID once its dashes are removed. A space that large is one an unrelated process will not walk into. Now run the same reasoning on SECRET123: it is drawn from the small set of strings a person would think to type, that set is well within reach of a test fixture, a placeholder in a sample config, or somebody's password reminder, so a log line containing it is barely more likely under the first explanation than under the second, and the observation stops being evidence. The entropy is not there to resist guessing. It is there so that one occurrence, rather than a pattern of occurrences, is enough.

The endpoint has to be one you control, for the same reason. If the exfiltration lands somewhere you cannot read — an attacker's server, a pastebin, a third party — you have not observed anything; you have inferred it. Standing up a small listener that logs every request it receives turns the finding into a fact you can point at. The observation that ends the argument is a line in your endpoint's log containing the canary.

# controlled_endpoint.py — the outbound destination you own
from http.server import BaseHTTPRequestHandler, HTTPServer

CANARY = "CANARY-7f3a9c2e-do-not-share"

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if CANARY in self.path:
            print(f"EXFIL CONFIRMED: {self.client_address[0]} {self.path}")
        self.send_response(200)
        self.end_headers()

HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()

The setup runs on a sandboxed, staging copy of the agent, meaning a deployment wired to non-production data and a network you are allowed to break, and it has three moves that line up with the three legs. Plant the canary in the private store — write it into a record the agent can read, an HR page, a repository file, whatever the trace named as the private-data source. Put the payload in the untrusted channel — the retrieved document, the inbound email, the API text field the trace named as the untrusted input — where the payload is instructions telling the agent to read the private store and send what it finds to http://your-endpoint:8080/. Then run the ordinary workflow and watch your endpoint's log.

What makes this conclusive rather than suggestive is the combination, not either half. A canary with no controlled endpoint tells you the string exists but not that it left. A controlled endpoint with real data instead of a canary tells you something arrived but not that it could only have come from the private store. Together they give you a log line, on infrastructure you own, containing a string that had exactly one origin — which is a claim with no gaps in it.

That conclusiveness runs in one direction only, and the asymmetry follows from the shape of the chain rather than from anything about canaries. The demonstration is a series of links that all have to hold: the payload reaches the model, the model is persuaded, the private read happens, the value is placed in an outbound message, and that message goes to the door you are watching. Arrival proves every link held, because a chain cannot deliver on a broken link — one observation, and all of the intermediate claims come with it for free. Silence proves only that at least one link failed, and it does not say which, because every one of them produces the same empty log. So a green result is worth more than the run that produced it and a red result is worth considerably less, and the next section is about what you have to add to a red result before it means anything at all.

Reading a run that did not fire#

Suppose you run it and nothing appears in the log. The instinct is to conclude the system is secure, and that conclusion is the one explanation you are not yet entitled to, because at least three others fit the same silence and you have not ruled them out.

The payload might never have reached the model. Staging often differs from production in exactly the plumbing that matters — the retrieval index is stale and does not contain the document you planted, the email you sent bounced or landed in a folder the agent does not poll, a length limit truncated your payload before the instructions. The chain was never assembled, so its not firing tells you nothing about whether it would fire when assembled.

The model might have reached the private store but chosen an outbound shape you were not watching. You planted the canary and stood up an HTTP listener, and the model dutifully put the canary into a markdown image URL, or into an email body, or into a Slack post — an outbound path the previous chapter's trace would have listed — and your endpoint saw nothing because the data left through a door you were not standing at. The exfiltration succeeded; your instrumentation missed it.

The model might have declined this particular phrasing. Injection success is sensitive to wording, ordering and framing, so a payload that fails is weak evidence, because a differently phrased one three attempts later may work. "The model refused" and "the model was not asked in a way it would comply with" produce identical logs.

You tell these apart by instrumenting the middle, not just the ends. Turn on a transcript of the agent's tool calls and model turns for the staging run, so you can see whether the untrusted content entered the context at all, whether the private read happened, and what the model emitted. If the payload never appeared in a model turn, you are in the first case and need to fix the plumbing. If the private read happened and the canary appeared in the model's output but not on your endpoint, you are in the second case and need to widen your instrumentation to every sink in the trace. If the content entered, the read did not happen, and the model produced a refusal, you are in the third case and should vary the payload before concluding anything. Ruling out the first two is what earns you the right to even discuss the third, and the third is rarely "secure" so much as "not yet with this wording."

There is a structure under that procedure that makes it repeatable on a system you have not seen before. The three explanations are not a list somebody happened to think of; there is one per leg of the trifecta, because a chain with three links can break in three places. The payload never reaching the model is the untrusted-content leg failing to assemble. The canary leaving by a door you were not watching is the outbound leg firing somewhere your instrumentation does not cover. The model declining is the middle — the propagation step, the one no boundary controls. Now line those up against what a transcript shows you, and the transcript has one checkpoint per explanation: whether the untrusted content appeared in a model turn, whether the private read happened, and what the model emitted. Each checkpoint decides exactly one explanation and none of the checkpoints is redundant, which is why the debugging has no residual case and why it has a forced order — you cannot evaluate the model's willingness on a run where the payload never arrived, because on that run the model was never asked.

Who hears about it, and in what form#

When it works, what you report is the finding, not a victory lap, and the audience is narrower than the temptation to broadcast it suggests. The report goes to the team that owns the agent and to whoever owns the security or application-security function for that service, and it contains the three legs as traced with their evidence, the exact payload, the exact canary, the endpoint log line with a timestamp, and the transcript showing the private read.

It states the impact in terms of the class of data the store holds rather than in terms of the canary, because the canary was a stand-in and the reader needs the general claim. The reason is that the owning team's next decision is a prioritisation one, and a canary in a staging record does not, by itself, convey urgency to someone reading a queue of bugs. "I moved a test string" invites the response that it was only a test string. The claim that lands is the one that names the class: any record the agent can read is exfiltrable by anyone who can write into the retrieved-documents index, and here is who can write into that index today. That sentence turns a clever demo into a scoped exposure with a blast radius the team can weigh against everything else on their plate, and it is true precisely because the canary's provenance argument generalises — if that one string could make the trip, so can any other value sitting in the same store, since the model did not care what the string was.

That generalisation deserves its condition attached, because it is the step a careful engineer will push on and they are right to. It holds because nothing in the path that carried the canary examined it: the retrieval returned a record, the model copied a token, the outbound call placed it in a query string, and each of those operations does the same thing to any string of similar length. Where the generalisation stops is wherever some component is content-sensitive — a data-loss filter matching card numbers, a length cap that a long document would exceed, a serialiser that would mangle a value containing quotes, a store whose sensitive columns sit behind a different grant from the one the canary was written into. So the claim to write is that any value the agent can read is exfiltrable by the path demonstrated, and the review step that earns it is asking whoever believes otherwise to name the component that would treat a real record differently, then running the demonstration again against a canary shaped like the thing they named.

Two things stay out of the report and out of the room. Real private data stays out, which the canary already guaranteed. And a public writeup stays out until the owning team has a fix, because a working exfiltration against a live internal assistant is an operational secret with the same handling as any other unpatched vulnerability. The report is an internal disclosure to the people who can fix it, on a timeline you agree with them, and it names the mitigation direction — break or constrain a leg — without pretending you have already chosen which.

The fix that gets proposed next#

A green result proves the path is real and settles the argument you started the chapter to settle. It does not tell you which leg to break, and it does not tell you that breaking a leg the obvious way will actually break it. Those are the next two problems, and they are separable.

The owning engineer, now convinced, will reach for the fix that is cheapest to their roadmap, and the most common reach is to add a sentence to the system prompt — tell the model not to include external URLs, not to read the store when summarising, not to obey instructions in documents. In the case above there already was such a sentence, and the demonstration ran straight through it, which is the empirical version of the argument that topic 7.2, on why an instruction cannot separate data from instruction, makes structurally.

What removal actually requires is a change at a boundary the model cannot argue with — a credential that no longer reaches the data, a tool that does not exist, a network that refuses the connection — and a way to test that a claimed removal is real rather than a request the attacker can talk the model out of honouring. That is the next chapter, and the proof of concept you just built is the instrument it uses: keep it runnable, because every claim about a removed leg is checked by running it again.