Tool execution safety

Read these first

• Architectural mitigations
• Tool design — semantics

What you will know

• Put a destructive tool behind a dry-run and a confirmation

  A preview must capture the exact arguments and intended side effects; confirmation of a vague summary is not meaningful.

• Enforce an allowlist rather than a denylist, and say why

  Default-deny scope should include the operation, resource, tenant, and caller context, since a permitted verb can still target the wrong data.

• Set a timeout on every tool and justify the number

  Choose budgets from dependency behavior and end-to-end SLOs, then ensure cancellation reaches subprocesses and network calls; otherwise a timed-out agent may leave work running.

• Produce an audit log that reconstructs what an agent did

  Record request and tool IDs, actor, approval, timestamps, arguments or safely redacted representations, results, and errors so an incident can be correlated without leaking secrets.

• Reproduce a timeout, denied call, and confirmation bypass attempt in an integration test

  These scenarios expose hangs, retries, partial writes, and approval races that isolated unit tests commonly miss.

Chapters

1. 1

   Dry runs and confirmation

   A dry_run parameter in a tool schema is an argument the model fills in, which leaves the model deciding whether to be careful and lets an injected instruction skip the preview. The design that holds separates the preview from the commit, pins the resolved plan on the server behind an opaque identifier, and requires a different actor to approve it — and it fixes what a preview must contain: resolved arguments, concrete before-and-after values, and the side effects a write triggers.

   Read chapter
2. 2

   Scoping a tool allowlist

   A denylist of forbidden SQL keywords fails on the capability nobody anticipated, which is the argument for deny-by-default. The subtler mistake is allowlisting only the operation: an agent permitted to post to Slack posted one customer's data into another customer's channel, because the scope never expressed which channel. A usable scope names operation, resource, tenant and caller context, resolves the resource's real owner, and is tested from the wrong tenant.

   Read chapter
3. 3

   Tool timeouts and cancellation

   Setting a tool timeout at the dependency's p99 plus headroom gives twelve seconds, and six calls against a thirty-second product target breaks the budget before anything is retried. The number has to be divided out of the end-to-end target, held as a task deadline rather than a per-call constant. Two further problems follow: a fired timeout that leaves a subprocess running or a request in flight has stopped nothing, and a timed-out write is an unknown outcome rather than a failed one.

   Read chapter
4. 4

   The tool execution audit log

   A log of every model request and completion answers almost nothing about what an agent did during a forty-minute session two weeks ago. The artifact that reconstructs it is written on the tool side: one record per invocation attempt, carrying correlated run and tool ids, actor and approver, policy decision, timestamps, the arguments actually sent or a safely redacted stand-in, downstream request ids, outcomes and retries — including the denials, and including what to record when privacy forbids recording arguments.

   Read chapter
5. 5

   Testing approval races and partial writes

   Unit tests with hand-written fakes miss the failures that matter here, because the fake agrees with the mental model that produced the bug. Approve a destructive call, change the underlying record before it executes, and watch it run against arguments that were correct when a human saw them. The same suite forces a timeout, a denied call, a bypass attempt and a mid-flight failure, so hangs, retries, half-completed writes and approval races surface in a test rather than in an incident.

   Read chapter
6. 6

   Confirmation cost and reversibility

   Support cancels 200 subscriptions a day by hand, and a confirmation on every agent cancellation makes the agent three times slower than the process it replaced, so somebody reading the telemetry removes it. Landing this well means scoping, previewing, timing out and logging the tool, then deciding which slice of cancellations genuinely stops for a person once reversibility, customer confirmation and velocity limits have taken the rest — and arranging attribution before launch.

   Read chapter