A guardrail is a limit declared before the winner is known

The readout for the new assistant configuration comes back on a Thursday, with 27,150 exposed conversations per arm, and it reads like this. Latency is reported at the 95th percentile, written p95, which is the response time that only one request in twenty exceeds and is where slowness is felt before it shows up in an average.

The meeting that follows is about how to weigh these against each other. Someone points out that 4 points of resolution rate is worth a lot more to the business than $0.007 a conversation. Someone else notes the flag rate is still under half a percent. A third person suggests shipping and opening tickets for the cost and the latency. Every one of those is a reasonable-sounding sentence, and the meeting is already lost, because the weighing is happening after the numbers exist and everyone in the room knows which way it needs to come out.

The thing declaring a threshold in advance actually buys#

A guardrail metric is one where a threshold and a consequence are fixed before launch, so that afterwards the only question is whether the number is on the right side of the line. That is the entire mechanism. It does not make anyone wiser about how much latency a point of resolution rate is worth; it moves that judgement to a moment when nobody knows which variant it will help.

The alternative is not a worse judgement so much as a different activity. Once the readout exists, everyone in the room has an implicit position — the person who wrote the prompt, the person whose quarterly goal is resolution rate, the person who owns the inference budget — and the discussion becomes a negotiation with a known payoff structure. The literature on this is old and comes from clinical trials, where the practice of registering primary endpoints, secondary endpoints and stopping rules before enrolment exists because outcome-dependent reasoning was demonstrably producing published effects that did not replicate. The failure mode transfers directly. A metric that becomes a trade the moment it is inconvenient was never a limit.

So the guardrail set for this experiment should have been written like configuration, because that is what it is:

experiment: support-sysprompt-v4
primary:
  metric: resolution_rate
  direction: increase
  minimum_detectable_effect: 2.0pp

guardrails:
  - metric: spend_per_conversation
    limit: "+15% relative"
    check: upper_bound_of_95_ci
    owner: platform-eng-lead
    on_breach: no_ship
  - metric: latency_p95
    limit: "+250ms absolute"
    check: upper_bound_of_95_ci
    owner: platform-eng-lead
    on_breach: no_ship
  - metric: safety_flag_rate
    limit: "+0.05pp absolute or +25% relative, whichever is smaller"
    check: upper_bound_of_95_ci
    owner: trust-and-safety-lead
    on_breach: abort_immediately
  - metric: escalation_to_human
    limit: "+1.0pp absolute"
    check: upper_bound_of_95_ci
    owner: support-ops-lead
    on_breach: no_ship

secondary:  # reported, no decision authority
  - turns_per_conversation
  - first_response_time
  - thumbs_down_rate

Three fields do the work that a shared understanding does not. The owner is a named person, not a team, because "the team decides" resolves to nobody at 6pm on a Thursday and because the owner of a guardrail should have no stake in the primary metric moving. The on_breach action distinguishes a variant that must not ship from one whose serving must stop today, which matters for safety and almost nothing else. And check says which statistic is compared to the limit, which is where most guardrail schemes quietly fail.

Why "not statistically significant" is not a pass#

A guardrail asks the opposite question from a primary metric. The primary metric starts from the null hypothesis, the assumption that the two configurations differ in nothing, and demands evidence of improvement against it. The guardrail needs evidence of absence of harm, and a test that fails to reject the null has produced no such evidence — it has produced silence, which an underpowered test produces regardless of what is true.

Suppose the safety flag rate had come back at 0.20% in control and 0.31% in treatment on a smaller test with 6,000 conversations per arm.

from math import sqrt

p1, n1 = 0.0020, 6000
p2, n2 = 0.0031, 6000
diff = p2 - p1
se = sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
lo, hi = diff - 1.96 * se, diff + 1.96 * se
print(f"diff {diff*100:.3f}pp  95% CI [{lo*100:.3f}, {hi*100:.3f}]pp")
# diff 0.110pp  95% CI [-0.062, 0.282]pp   → p ≈ 0.21, "no significant difference"

The interval reaches to +0.28 pp, which is more than double the declared limit of +0.05 pp. Reading that as a pass means concluding "no harm detected" from a test that could not have detected a harm four times larger than the one you said you would refuse to accept. Comparing the upper bound of the interval to the limit — the non-inferiority framing — is what makes the guardrail mean what it says. It also means an underpowered guardrail blocks rather than waves through, which is the correct default and is occasionally expensive.

The actual readout does not have this problem. At 27,150 per arm the interval on the flag rate is [0.15, 0.35] pp, the whole of it above the limit, and the relative increase is 2.25× with an interval around it of roughly [1.6, 3.1]. In counts, control flagged 54 conversations and treatment flagged 122. There is no small-numbers escape here — the smallness of 0.45% is a reason to check whether the difference is an artifact of 54 events, and the check comes back saying it is not.

Choosing the four or five, and choosing the numbers#

Picking the guardrail set is a failure-mode inventory rather than a metrics exercise. The question to put to the room before launch is what would make everyone regret shipping this even if resolution rate went up, and the answers tend to fall into four families: the change costs more than it earns, the change makes the product slow, the change produces output the organisation would not defend, and the change pushes work onto humans elsewhere in the system. Those map onto spend, tail latency, safety classifier rate and escalation, which is why that set recurs across so many teams. A fifth candidate specific to this product is complaint or refund volume, which lags by days and is worth tracking even where it cannot be resolved inside the window.

Setting the limit is where the discipline usually collapses, because the tempting method is to pick whatever the test can detect. A limit derived from statistical detectability has no relationship to what anyone would stop for, and it produces both directions of error: a 0.4% detectable change in escalation becomes a guardrail nobody would actually enforce, while a genuinely intolerable 12% cost rise on a low-traffic feature is undetectable and therefore unguarded. The workable method is to ask what number would make the owner say no, independently of this change — the platform lead knows the inference budget has 15% of headroom this quarter, the support operations lead knows a 1-point rise in escalation is 240 extra tickets a week and roughly two headcount — and then to check whether the test can detect that number. When it cannot, the honest response is to say so in the predeclaration and to plan a longer window or a post-launch holdback for that specific metric, rather than to lower the limit until the maths agrees.

Some harms cannot be observed in a fourteen-day window at all. Retention, churn, trust in the assistant after a bad experience and the slow migration of users to the human channel all move over months, and no guardrail evaluated at the horizon will catch them. The instrument for those is a long-run holdback, keeping a small share of users on the old configuration for a quarter after the ship, with the same threshold and the same owner attached. That converts an unmeasurable guardrail into a measurable one at the cost of running two configurations for three months, and it is the only honest way to hold a limit on a quantity that does not move inside a test.

The three proposals in the meeting#

Ship because resolution rate is what the business cares about. This treats the guardrail as another term in a weighted sum, which is the specific trade guardrails are declared to prevent. If the organisation genuinely believes 4 points of resolution rate is worth a 2.25× increase in flagged responses, that belief should be written into the limit and defended in advance, where it would have to be defended to the trust-and-safety lead rather than asserted to a room that wants to ship.

Ship and open tickets for cost and latency. A ticket is a promise to reduce a harm that is being incurred at full volume from the moment of the ship. The spend guardrail is breached by more than double its limit: at 1.2 million conversations a month, $0.021 to $0.028 is $8,400 a month, arriving immediately, against a fix that has no owner, no estimate and no place in the sprint. The latency breach is worse in kind, because p95 going from 2.3 s to 3.2 s changes user behaviour on this surface — abandonment on this assistant rises about 1.5 points per additional second at the tail — which means the latency regression is partly eating the resolution-rate win the ship is being justified by. That interaction is invisible in a table that reads each row separately.

The flag rate is under half a percent in absolute terms. Absolute smallness is an argument about statistical reliability, and it has already been answered. It is not an argument about acceptability, because the rate is a property of the system's behaviour, and a change from 54 to 122 flagged responses per 27,150 conversations means the new configuration is producing a different distribution of outputs in a category the organisation has decided to police.

Before anything can be said about that number, three things have to be established. The classifier must be identical in both arms and not differentially sensitive to the treatment's output style — a prompt that produces longer, more explicit answers can trip a keyword-sensitive classifier more often without producing more harm, which is a measurement-validity question and not a safety question. The 122 flagged conversations must be read, or a stratified sample of them, and broken down by category and severity, since 122 self-harm-adjacent responses and 122 mild-profanity responses are not the same finding. And it has to be known whether those outputs were shown to users or intercepted, which the previous topic's rollout mechanics determine.

Who decides is the part teams get wrong most often. It is not the experiment owner, whose incentive runs the other way, and it is not the room. It is the person named in owner for that guardrail, with the authority to say no and no obligation to justify it in terms of the primary metric.

What a guardrail is not#

It is not an alert. Alerts fire on the running system against absolute levels and page whoever is on call; guardrails compare two arms against each other over an analysis window and gate a decision. The thresholds often derive from the same service level objectives, the latency and availability targets the system is operated against, and they are still different objects, because an alert that never fires tells you the system is within its envelope while a guardrail that never fires tells you nothing about whether this change degraded anything.

It is not the rollback plan either. The previous topic's ramp and kill switch determine how fast a bad change can be withdrawn; the guardrail determines whether it should be released at all. A team with excellent rollback tooling is tempted to treat every guardrail as recoverable and ship on the primary metric, which works for latency and cost and does not work for a class of harm where the damage is done at the moment of serving.

And it is not free. Every guardrail is another comparison, and a set of eight guardrails on a null change will breach at least one about a third of the time at conventional thresholds, which produces enough false blocks that the mechanism gets ignored. The way out is to keep the set small — four or five, each tied to a failure mode someone can name — and to set limits at a size that matters operationally rather than at whatever is statistically detectable. A guardrail on a metric nobody would actually stop a ship for is a guardrail that will be argued away the first time it fires, and it takes the credibility of the others with it.

All of this assumes the numbers in the table are the ones the experiment was designed to produce, read once, at the end of the declared window. The Thursday readout was not the first time anyone looked at this dashboard.