An agent that always returns something may be recovering badly

A month of runs with no failures in it is the shape a working system is supposed to have, and on an agent with recovery machinery it is the shape a broken one has too. That is the problem this chapter is about. Once a run can route around a dead dependency, retry with a different approach, and hand back a partial result rather than an exception, the number that used to fall when something went wrong stops falling, and it stops falling whether the recovery was good or bad.

Three terms carry the argument. Completion rate is the fraction of runs that returned a result rather than an error. Grounding is whether the facts a run asserts match the system of record — the store whose copy of a fact is the authoritative one, so that when a draft and a customer-relationship database disagree about whether a refund was processed, the database is right by definition. And a run is degraded when it reached its goal with a tool missing or a fallback used, which is a third state between success and failure that most dashboards have no column for. The gap between completion and grounding is where a fluent, confident, wrong answer lives, and nothing in the previous six chapters looks at it.

Completion rate stopped measuring quality the day recovery shipped#

Before any of the mechanisms in this topic existed, a failed dependency produced an exception, the run died, and the completion rate moved. It was a bad experience and a good signal, because the metric was coupled to the truth: when the CRM went down, the number went down.

Every mechanism added since then has deliberately weakened that coupling. The breaker removes a failing tool and asks the model to continue without it. The changed-strategy retry asks the model to find another path. The terminal result exists so a stopped run hands back something useful. Each of those converts a hard failure into a softer outcome, which is the point, and the arithmetic consequence is that completion rate now measures whether the machinery produced output rather than whether the output was right. A 100% completion rate on an agent with good recovery and an agent with no dependencies at all look identical from the dashboard.

It is worth being exact about what happened to that metric, because "less reliable" undersells it. A metric is useful when its value varies with the thing you want to know. Completion rate used to vary with dependency health, through one mechanism: a broken dependency raised, and a raised exception ended a run without a result. Every chapter since removed exactly that mechanism, one route at a time — the breaker catches the failing tool before it can end the run, the changed-strategy retry catches the rejected argument, the terminal result catches the exhausted budget. What is left is a quantity computed from inputs that no longer include whether the dependencies worked. So completion rate did not get noisier. It became constant with respect to dependency health, and a constant carries no information about the variable it no longer depends on, however carefully it is plotted.

That is why the flat line is the finding rather than the reassurance. A control that is firing correctly and never needs to and a control that was never wired in produce the same reading on a metric that cannot see either of them, and so does a wrapper that catches every stop reason and converts it into a polite apology. What a completion rate certifies is that the machinery produced output. It never certifies that a dependency was healthy, and on this architecture it cannot be made to.

The number that should have been questioned first is the zero. On a fleet with the instruments from the previous chapters, a healthy month has a nonzero rate of every stop reason: budget exhaustion in the low tenths of a percent, loop detection lower still, and a visible spike of tool_unavailable on the day a dependency was down for 34 minutes. Those numbers are the evidence that the controls are live. A perfectly flat 0.00% across 612,340 runs means one of three things — the controls are not wired in, the stop reasons are being caught by a wrapper that turns them into a polite apology, or nothing has gone wrong in a month, which for a system with four external dependencies is the least likely of the three.

The rate to expect is not a matter of taste, and the earlier chapter on budgets already fixed it: the budget that fires most often should fire on a few runs in every thousand, which is the band that keeps a limit tight enough to catch a regression and loose enough not to truncate real work. A stopped-run rate is therefore a quantity with a predicted value, and a predicted value is what makes a measurement able to disagree with you. Zero disagrees with it, in the direction that should worry a reader most, because the only way to read a control that has never once fired is that nothing is behind it.

Read the rows against each other rather than one at a time, and they form a chain in which each row measures something the row above it cannot see. Completion sees output and not stops. Stopped-run rate sees stops and not degradations, because a degraded run completes. Degraded-run rate sees that a fallback was used and not whether anybody was told, which is why disclosure is a separate line rather than an assumption. And disclosure sees what the draft said about itself, not whether what it said was true. Each row exists because the one above it is blind to a failure that still reaches a customer, and the last row is where the chain has to end, because the only way to check an assertion against reality is to go and look at reality.

The last row is the only one that would have caught the six percent, and it is the only one that cannot be computed from the run's own telemetry. Somebody has to compare the draft against the CRM, whether that is a human sampling 200 drafts a week or an automated check that extracts asserted order states from the draft and re-queries the system of record. Both are more work than reading a completion rate, and the completion rate cannot substitute for either.

There is a trap in that last row that a reader is likely to walk into, because two figures on this agent happen to be the same number. Sampled grounding accuracy came out at roughly ninety-four drafts in a hundred matching the system of record. The human approval rate — the share of drafts an approver passes unchanged — is also roughly ninety-four in a hundred. Nothing whatsoever says they are the same ninety-four. An approver reads a draft for about eleven seconds and never opens the CRM, so the approval decision is computed from how the draft reads and has no input that could depend on whether its claims are true. It is the same shape as the completion rate one section above: a quantity computed from inputs that exclude the thing being measured. What an approval rate licenses is that a draft read plausibly to somebody in a hurry. It never licenses that the draft was correct, and a draft that confidently states a refund was processed is more likely to be approved than one that admits it could not check, which points the error in the worse direction.

What separates a recovery test from a test that happened to pass#

The test suite for this agent had a case called test_recovers_from_crm_failure and it was green. What it did was fail crm_lookup once, let the second call succeed, and assert the run completed. That test passes on an agent with no recovery logic whatsoever, because the transport retry underneath the tool handles a single transient before the loop ever sees it, so the test was measuring the HTTP client and reporting on the agent.

Follow that failure down through the layers and it is clear why nothing in the agent was ever exercised. The injected fault is a single failure on the first attempt. The transport wrapper from the first chapter retries a retryable status several times before giving up, so attempt two happens inside that wrapper, succeeds, and the wrapper returns a normal result. The loop is handed a successful tool result and never learns a failure occurred. No error reaches the model, no breaker counter increments, no fallback path is chosen — and the run completes, which is the only thing the test asserts. The test's verdict is therefore constant with respect to the property it claims to measure: delete every line of recovery logic in the agent and it stays green, because none of those lines was on the path the fault took.

A recovery test earns the name when the injected failure is permanent for the duration of the run, so that no amount of retrying the original action can succeed, and when the assertions describe the alternate path rather than merely the absence of an exception.

def test_drafts_reply_when_crm_is_down():
    h = Harness(fixture="ticket_88231", seed=7)
    h.fail_tool("crm_lookup", status=503, forever=True)

    result = h.run()

    # 1. The goal was reached.
    assert result["status"] == "completed"
    assert "4471" in result["draft"]

    # 2. By a different path.
    assert "order_history_db" in h.tools_called

    # 3. Without grinding on the dead tool.
    assert h.call_count("crm_lookup") <= 5          # breaker opened
    assert result["stop_reason"] is None

    # 4. And the degradation is visible to the human approver.
    assert result["degraded"] == ["crm_lookup"]
    assert "could not be confirmed" in result["draft"]

    # 5. At a cost the fallback path can sustain.
    assert result["budgets"]["cost_usd"]["used"] < 0.09   # p95 is 0.062

forever=True is the load-bearing argument. It removes the escape the green test left open, because a fault that never heals cannot be absorbed by any retry at any layer, which forces whatever happens next to be a decision the agent made rather than a repeat the transport wrapper won.

The five assertions are worth reading as five different things being ruled out, since each one on its own admits a failure the next one catches. The first rules out a run that stopped: it reached a completed status and the draft names the order. The second rules out a run that reached that status by the original route — the goal was met, and a named alternate tool was used to meet it, which is what distinguishes recovery from the fault never having landed. The third rules out a run that got there by grinding: without a bound on how many times the dead tool was called, an agent that eventually succeeds after eighteen attempts satisfies the first two assertions and is a runaway in production, and the bound is set at the breaker's own threshold so the assertion fails unless the breaker actually opened. The fourth rules out a silent substitution, which is the failure the whole chapter is about. The fifth rules out a recovery that works and cannot be afforded.

Assertion 5's bound is chosen against a specific runaway rather than picked as a round number. The prose worry is a fallback path costing three times a normal run, and three times the run's own cost at the ninety-fifth percentile is comfortably above the ceiling the assertion sets — so a fallback that expensive fails the assertion, while ordinary variation around the usual cost does not. A bound loose enough to admit the thing it was written to catch is a bound that will pass forever, which is the same defect as the green test one level up.

Assertion 3 is the one that connects this back to the breaker: without a call-count bound, an agent that eventually succeeds after 18 attempts passes assertions 1 and 2 and is a runaway in production. Assertion 4 is the one that would have prevented the incident at the top of this chapter, because a draft that says "your refund could not be confirmed against our billing system" gets a different eleven seconds of human attention than one that says the refund completed on Tuesday. Assertion 5 exists because a recovery path costing three times the normal run is a capacity problem that only shows up when the outage is fleet-wide and every run takes the expensive path at once.

The fault matrix is the deliverable#

One test is a start; what makes the recovery behaviour reviewable is a table of injected faults against expected terminal states, run in CI, regenerated whenever the tool surface changes.

The first two rows are the pair that matters most, and they are adjacent on purpose. They inject the same status code and differ only in whether it heals, and the expected outcomes are opposites: one run must show a fallback and a notice, the other must show neither. A suite containing only the second row is the green test that shipped. A suite containing only the first row cannot tell whether the agent has become so eager to fall back that it abandons a dependency that would have answered on the next attempt. The pair is what pins the behaviour, and the same reading applies down the table — each row is chosen so that a plausible mistake in the machinery changes its outcome.

That last row is in the table on purpose. Every other fault produces a signal somewhere in the machinery, and a dependency that lies produces none, which is why sampled grounding accuracy is a separate line of defence rather than a nice-to-have. Writing the row down is also the cheapest way to stop a review conversation from concluding that the matrix is complete.

It is the only row whose expected outcome is that no instrument in this entire topic moves. Trace a wrong-but-well-formed response through everything built so far: the transport wrapper sees a success and does not retry, the semantic classifier sees no rejection, the breaker records a success and resets its counter, every budget is spent normally, the argument hash sees a call that was made once, and the progress detector sees a result digest that changed. A run built on that answer completes, is not degraded by any definition the harness can compute, and discloses nothing, because there is nothing it knows to disclose. The instruments in this topic all measure the shape of the interaction, and the shape of a lie is the shape of the truth.

Assertions that survive a model version bump#

Non-determinism makes these tests different from the ones around them, and the honest handling is to assert invariants rather than sequences. Asserting that step 6 is a call to order_history_db produces a test that fails on a model version bump for no reason anyone can act on; asserting that order_history_db appears somewhere in the run and crm_lookup appears at most five times survives it. Where a behaviour is genuinely probabilistic, run the case across a set of seeds — the fixed numbers that make the model's sampling repeat itself run to run, as seed=7 does above — and require a threshold — eighteen of twenty on this suite — and record the two that failed rather than deleting them, because a recovery path that works 90% of the time is a fact about the system that the team should hold rather than a flake to be re-run until green. Nishant Gupta's talk Deterministic Infra for Non-Deterministic AI Agents is a good companion here, since the argument runs the same way: the loop is stochastic and the guardrails around it do not have to be.

The threshold and the rate in that paragraph are the same statement twice, which is easy to read past. Requiring eighteen passes out of twenty seeds is a declaration that a recovery path failing one seed in ten is acceptable, and it is worth deciding that deliberately rather than inheriting it from whatever number made the suite go green. That is a design decision and not a requirement: nothing about the mechanism fixes it, and an agent whose fallback path touches money should be held to a different one than an agent drafting replies a human approves.

The counterpart test — inject a failure that heals after one attempt and assert the run completes without any fallback — is worth keeping as well, and it belongs in a separate case with a name that says what it tests. Both behaviours are real, they are the two halves of the first chapter's distinction, and a single test that conflates them tells you which one fired only by accident.

Two further limits are worth naming before this is treated as settled. A fault matrix tests the faults somebody thought of, and its last row is an admission that the list is open rather than a decoration; the honest expectation is that the row below it gets written after the next incident. And sampled grounding is a sample, so it bounds the error rate with the precision a sample of that size allows and no better — a fortnight where the sampled figure moves by a point or two has not necessarily told you anything, and treating sampling noise as a regression is how a weekly check turns into a weekly argument.

The controls in this topic compose into a small amount of code around the loop from topic 10.3 — a classifier that separates transport failures from semantic ones, a breaker keyed per tool, four budgets checked at the top of the loop and pushed down into every call, a hash for repeated actions with a progress counter behind it, and a terminal result with an effects ledger under it. None of it is difficult individually, and all of it is the kind of code that gets written after the incident rather than before, because a loop with none of it works perfectly on a good day.

What changes once it is in place is what your failures look like. A run that cannot succeed stops in seconds instead of minutes, names the reason it stopped, and hands back what it did. A dependency that goes down costs one run's worth of discovery rather than every concurrent run's. And the dashboard tells you a smaller, truer story than 612,340 successes, which is the trade the last thirty pages have been arguing for: an agent that sometimes says it could not do the task is worth considerably more than one that always says it did.