Chapter 6 of 9 · 8 min read
The safest credential is the one the agent never holds
On this page
The token had been scoped exactly as the last review asked. It was valid for 20 minutes, it was issued per run, it could deploy only to the staging environment, and it could deploy only the platform service. On the Thursday a run holding one of those tokens deployed commit e91f4a2 to staging, which took the payments integration tests down for 50 minutes and blocked four other engineers who were using staging to verify their own changes.
Nothing about the token was wrong. The task had said "verify the fix in staging", the agent had built its branch, and deploying its branch to staging is a plain reading of the instruction. e91f4a2 had never been through CI, had never been reviewed, and was the agent's own work in progress. Every constraint on the token was a constraint on where the deploy could land, and the failure was about what was deployed, which is a property of the arguments rather than of the target. A credential cannot see arguments. It is evaluated once, when it is issued, against facts known at that moment, and after that it is a bearer secret that answers yes to everything inside its scope.
Moving the decision to the moment of the call
A broker inverts the arrangement. An action broker is a service that sits between the run and the systems it wants to affect, holds the credentials for those systems itself, and exposes instead a small set of named actions that a run may request. The run holds no deploy credential; the broker does, and it decides per request whether to use it. When the run wants a deploy it asks, the broker evaluates policy against the specific arguments in front of it, and either performs the action with its own credential or refuses.
The property that makes this different from a narrower token is that the check on the right-hand side depends on the argument. No IAM policy, OAuth scope or deploy key can express "only commits with a passing CI run on a branch belonging to this task", because the credential system has no way to look up CI status, and even if it could, it is consulted at issue time when the commit does not yet exist. The broker is consulted at call time and can consult anything.
@action(name="deploy", allowed_envs={"staging"})
def deploy(ctx: RunContext, sha: str, env: str) -> Result:
if env != "staging":
return deny("policy.env_not_permitted")
if not ctx.task.repos & {sha_repo(sha)}:
return deny("policy.repo_outside_task_scope")
status = ci.status(sha)
if status != "success":
return deny("policy.ci_status_required", detail=status)
if not github.branches_containing(sha) & ctx.task.branches:
return deny("policy.sha_not_on_task_branch")
if deploys.count(env=env, window="10m") >= 3:
return deny("policy.rate_limited")
return deploy_api.release(sha, env, actor=f"broker/{ctx.run_id}")Five checks, each of which is a sentence somebody would have said out loud in a design review and none of which is expressible as a permission. The refusal is a structured reason rather than a 403 with no body, which matters because the agent has to be able to do something sensible with it — policy.ci_status_required tells the run to wait for CI and retry, while policy.repo_outside_task_scope tells it that it has misunderstood its own task and should stop.
The arrangement is not new and the closest familiar version is sudoers. Giving an engineer the root password is a credential: it is evaluated once, at the moment they are told it, and afterwards it answers yes to everything. Giving them a line permitting /usr/bin/systemctl restart nginx and nothing else is mediation: sudo holds the privilege, the engineer holds an identity, and every invocation is checked against the arguments and written to a log. The pattern spread through operations for the same reason it is worth applying here, which is that the interesting question is almost never who someone is and almost always what they are asking to do right now. The reason sudoers is also a byword for misconfiguration is instructive too — a rule ending in /usr/bin/vi grants a root shell, because vi can spawn one, and the equivalent mistake in a broker is an action whose effect is broader than its name suggests.
What the run does hold
The honest accounting is that the run holds an identity, and an identity is a thing that can be stolen. The difference from a token is what stealing it yields. A deploy token lifted out of a transcript works from anywhere until it expires and does everything in its scope; a run identity presented to the broker gets whatever policy allows for that run, which is bounded by the task's repositories and branches, rate-limited, logged with the run identifier, and revocable in one call the moment the run is cancelled.
POST /actions/deploy HTTP/1.1
Host: broker.dev.acme.internal
Authorization: Bearer <run-assertion>
Content-Type: application/json
{"sha": "e91f4a2", "env": "staging"}{
"ok": false,
"reason": "policy.ci_status_required",
"detail": "no completed workflow run for e91f4a2",
"run_id": "run-b71e04",
"correlation_id": "act-7d219b",
"retry_after_s": null
}The run-assertion is minted by the harness at spawn, carries the run identifier and the task's scope as claims, is signed, expires with the run, and names exactly one audience — the single service permitted to accept it, which every other service checks and rejects. It grants nothing directly — presented to the deploy API it is meaningless, presented to the database it is meaningless, and presented to the broker it is a name that policy is written about. Where the infrastructure supports workload identity — a scheme in which the platform running the process vouches for what that process is, so that no secret has to be handed to it at all — the assertion can be dropped entirely and the broker can attest the calling workload itself, which removes the last bearer secret from the run's environment.
The audit trail changes character as well. The deploy API's own log records that broker released e91f4a2 to staging at 14:02, which is true and nearly useless for answering why. The broker's log records that run b71e04, working task PLAT-2261, requested a deploy of e91f4a2 to staging, was refused for ci_status_required, requested it again eleven minutes later after CI went green, and was allowed. Denials are the valuable half and they only exist in a design where something is in a position to deny. A token that is never used to do something forbidden produces no record of the attempt, because the attempt succeeded.
How the denial reaches the model matters more than it looks. The broker's structured reason is surfaced to the run as the tool result, which means the agent reads no completed workflow run for e91f4a2 and can act on it, and in the Thursday scenario the run did: it pushed its branch, waited for CI, found two failing payments tests, fixed them, and deployed a green commit forty minutes later without a human intervening. A refusal that arrives as an opaque 403 Forbidden produces a different and much worse behaviour, where the agent retries, then tries a neighbouring endpoint, then invents a workaround, because it has been given no information to reason with. Policy engines written for humans tend to be terse on the assumption that a person will go and ask someone; policy engines in front of agents should say what would have to be true.
The enumeration cost is the design
Every privileged action has to be written down in advance, and this is where teams decide the approach is too heavy. The number is worth having: on the harness this chapter is drawn from, fourteen actions covered 96% of runs after three months — deploy to staging, open a pull request, comment on an issue, request a fixture, invalidate a CDN path, roll back a staging release, and so on. Adding a fifteenth is a pull request against the broker with a policy function and a test, reviewed by whoever owns the underlying system, which takes about a day of calendar time.
The 4% that the fourteen do not cover is the part to be clear-eyed about. Some of it is genuinely novel work that nobody anticipated, and the correct handling is that the run stops and a human does it. Some of it is a request that gets an action added, and after three months the rate of new actions had fallen to roughly one a fortnight. What the enumeration buys in exchange is a written, reviewable list of everything an agent is capable of doing to the outside world — a document that no credential-based design produces, because the answer there is spread across IAM policies, OAuth scopes, deploy keys and whatever the CI system grants, and nobody can read it in one sitting.
The failure mode to watch for has a recognisable shape. Someone adds an action called run_query, or execute, or kubectl, taking an arbitrary string, because it unblocks a class of tasks in an afternoon. That action re-creates the credential it replaced, with a broker in front of it adding latency and a false sense of mediation, and every policy check in the file above becomes decorative. An action whose argument is a program is a credential.
Where a broker is the wrong tool
It is a poor fit for exploratory work. A run whose job is to characterise an unfamiliar system needs to poke at it in ways nobody enumerated, and a broker either refuses everything interesting or acquires the generic escape hatch above. Teams doing that kind of work are better served by a genuinely disposable environment holding real credentials scoped to genuinely disposable resources, where the blast radius is the environment itself.
It is also a new dependency in the hot path of every run, with the availability and latency consequences that implies. The broker on this harness added 40 ms per action at p50, the median call, and 210 ms at p99, the slowest one call in a hundred, which is invisible against a 14-minute task and is not invisible if an action is called in a loop. When the broker was down for 12 minutes during a deploy of its own, every run that needed an action failed, and the pressure that afternoon to add a bypass was the same pressure chapter 2 described — which means the broker has to be run with the reliability of the systems behind it, not as a side project.
There is a scope limit that is easy to overstate the value of. A broker mediates the actions the run takes through it, and the run still has a network stack. It can still reach the package registry, still resolve internal hostnames, still open a socket to anything the network policy permits, and still send whatever it has read to any host that will accept it. Removing the credential removed one class of capability and left the substrate that capability travelled over untouched, which is why the next boundary is not about what the agent is allowed to do but about where its packets are allowed to go — a question that, on most teams, has already been answered badly in both directions.
Sandboxed developer workspaces — read the current version at https://daviddominguez.dev/learn/en/sandboxed-developer-workspaces/the-safest-credential-is-the-one-the-agent-never-holds