Constraints travel worse than goals across a handoff

The customer's first message was one sentence: "refund my order, and do not email me about it — I'll check the app." The refund was correct, £68.40 against order 41182, issued nine seconds after the conversation started. Fourteen seconds later the customer received an email confirming it.

Reading the three payloads in order shows where the instruction stopped travelling, and there is nothing in any of them that a reviewer would flag.

// hop 1  triage -> refunds
{"objective": "Refund order 41182 as requested; customer does not want email.",
 "remaining": ["Check eligibility.", "Issue refund.", "Confirm outcome."],
 "constraints": [{"text": "do not email the customer about this",
                  "scope": "run", "stated_by": "customer"}]}

// hop 2  refunds -> notifications
{"objective": "Confirm refund RF-90412 (£68.40, order 41182) to the customer.",
 "remaining": ["Send the customer confirmation of the refund."],
 "constraints": []}

The refunds agent did its job and forwarded what was left of it. Its own task — check eligibility, issue the refund — carried no email component, so when it composed the next task, the do-not-email instruction was not part of the thing being described. Nothing in its instructions said to enumerate the constraints it had received and pass them on, and the field it would have put them in was one it had never been asked to populate. The notifications agent then received a task that was complete, unambiguous, and stripped of the one thing that would have changed its behaviour.

Why goals survive the hop and constraints do not#

A goal is the payload's subject. Every sender is composing a description of what the receiver should do, so the goal is the thing they are actively writing, and an agent that forgets to state the objective produces a payload that is visibly broken. A constraint is a property of the request rather than of the sub-task being handed on, and its relationship to the current hop is often indirect: "do not email me" constrains an action that refunds is not taking, about a channel refunds does not use, in service of a preference the customer expressed to a different agent two hops ago.

That asymmetry has a consequence for how the failure gets reviewed. Take each agent on its own and read its input and its output, and all three are correct. Triage extracted the constraint and passed it. Refunds issued the right refund and described the right next step. Notifications sent an accurate confirmation of a real refund through its only channel. The defect exists in the difference between what refunds held and what refunds sent, which is a relation between two states rather than a property of either one, so no amount of per-agent evaluation will surface it. This is the same reason that testing two services in isolation never catches a field the producer stopped sending and the consumer still needs; the check has to live at the boundary, on the pair.

The fix that reinstates the failure#

The reflex, once the incident is understood, is to stop sending task payloads and give every agent the whole conversation. The instruction is in the transcript, in plain English, in the customer's first message, and an agent reading it would surely have honoured it.

It would have to notice it first, which is the same inference that just failed, made over more material. The notifications agent's job would become: read 4,100 tokens of conversation, decide which parts of it constrain an action nobody has explicitly told you about, and act. That judgement is harder at hop three than at hop one, because by then the customer's original sentence is buried under an eligibility check, a policy quotation and a refund confirmation, all of which are more recent and more topically related to the task at hand.

The second cost is mechanical. Handing the full conversation forward means the context grows at every hop, and on this workload the measured growth is 6.4k tokens at the first transfer to 21k at the fifth, so the fifth hop costs 3.3× the first for the same reasoning. Chapter 1's cost table is that effect at conversation scale.

The third cost is the one that matters most a month later. A payload with a constraints list is something you can assert on: it is present or it is not, it matches the sender's list or it does not, and the check is a set comparison that runs in microseconds. A transcript gives you nothing to assert on, because "the instruction was somewhere in those 4,100 tokens" is true of both the run that worked and the run that did not. Choosing the transcript trades a testable interface for an untestable one, at the exact moment you have learned why you need the test.

Scoring the gap field by field#

The measurable version of this incident is the difference between what a sender held and what its receiver got, computed at every transfer, across a whole run set. Five classes of information are worth scoring separately, because they are lost at different rates and for different reasons.

CLASSES = ["established", "tool_results", "constraints", "confidence", "stop_conditions"]

def score_handoff(sender: AgentState, h: Handoff) -> dict[str, tuple[int, int]]:
    """For each class: (carried, held). Exact where the sender's state is structured."""
    held = {
        "established":     {f.key for f in sender.facts},
        "tool_results":    {c.id for c in sender.tool_calls if c.succeeded},
        "constraints":     {c.text for c in sender.constraints},
        "confidence":      {"self_reported"} if sender.confidence is not None else set(),
        "stop_conditions": set(sender.stop_conditions),
    }
    carried = {
        "established":     {f.key for f in h.established},
        "tool_results":    {f.evidence for f in h.established if f.source == "tool_result"},
        "constraints":     {c.text for c in h.constraints},
        "confidence":      {"self_reported"} if h.confidence is not None else set(),
        "stop_conditions": set(h.stop_conditions),
    }
    return {k: (len(held[k] & carried[k]), len(held[k])) for k in CLASSES}

Four of the five classes compare exactly, because the sender's state is structured and the payload is typed, so nothing here needs a model. Constraints need one wrinkle: an agent may legitimately reword a constraint, so exact string matching under-reports, and the version in production normalises with an embedding match at 0.92 cosine, which scores how close two pieces of text are in meaning on a scale where 1.0 is identical, before falling back to a judge — a separate model call asked to rule on whether the two wordings say the same thing — on the 3% that land between 0.80 and 0.92. That judge runs offline on a sample and costs about $0.002 per handoff scored.

Run over 214 recorded conversations and 1,043 transfers, the picture stopped being an anecdote.

Three of those rows changed a decision. Constraints at 71.9% turned the email incident from one customer complaint into a known rate of roughly one dropped constraint every 45 transfers, with more than half concentrated on a single edge, which made the first fix obvious: refunds' payload composition step now copies constraints forward by default and requires an explicit discharged_by entry to drop one. Stop conditions at 41% explained a different symptom entirely, because agents were overwriting the run's termination test with their own sub-task's completion, which is how a run ends when a specialist finishes rather than when the customer's problem does. Confidence at zero was not a loss at all; it was a field nobody had built, and the discovery that no receiver had ever been told how sure its sender was is the reason a partial answer arrives looking exactly like a complete one.

A constraint can arrive intact and mean less than it did#

The 71.9% carry rate counts a constraint as carried if the receiver's payload contains something close enough to it, and closeness is doing more work in that sentence than it should. Reading 40 of the 59 carried constraints by hand found seven that had been reworded into something weaker on the way. "Do not email the customer about this" arrived at notifications as "customer prefers app notifications", which scored 0.94 cosine against the original and was counted as a clean carry. The notifications agent then did what an agent does with a preference: it weighed the preference against its instruction to confirm every refund in writing, decided that a refund confirmation was important enough to warrant an exception, and sent the email. Every field was populated and the measurement said the hop was fine.

What was lost is the constraint's modality, meaning whether the customer forbade the thing, required it, or merely expressed a preference about it. A prohibition, a requirement and a preference are three different instructions about the same subject, and a similarity score cannot distinguish them, because they are lexically almost identical and semantically opposite in the only respect that matters. Modality is a small closed set, so it can be a field and compared exactly, which takes that part of the check out of the model's hands.

class Constraint(BaseModel):
    text: str
    modality: Literal["prohibition", "requirement", "preference"]
    scope: Literal["run", "task"]
    stated_by: Literal["customer", "policy"]
    discharged_by: str | None = None      # agent name + hop, or None while live

def modality_preserved(held: list[Constraint], sent: list[Constraint]) -> list[str]:
    sent_by_text = {c.text: c for c in sent}
    faults = []
    for h in held:
        s = sent_by_text.get(h.text) or nearest(h, sent, threshold=0.92)
        if s is None:
            faults.append(f"dropped: {h.text!r}")
        elif s.modality != h.modality:
            faults.append(f"weakened: {h.text!r} {h.modality} -> {s.modality}")
        elif s.scope != h.scope:
            faults.append(f"narrowed: {h.text!r} {h.scope} -> {s.scope}")
    return faults

Scope moves the same way and for the same reason. A constraint marked run applies to everything that happens until the run ends; the same text marked task applies to the hop it was attached to and evaporates at the next transfer. Senders relabelled run as task on four of the forty sampled handoffs, always in the direction that made the constraint the sender's problem rather than the run's, which is what a model does when it is summarising its own completed work. With modality and scope compared exactly, the weakening rate on the next 200 sampled handoffs was one, and the one was a genuine ambiguity in what the customer had written.

A constraint therefore has a lifecycle rather than a value, and the states are worth naming because the transitions are where the checks go.

The transition worth defending is live → discharged, because it is the only legitimate way a constraint leaves a run and it is the one agents will reach for once they know it exists. Requiring discharged_by to name both an agent and a hop makes discharge an auditable act rather than an omission, and it separates the two situations that look identical in a payload with an empty constraints list: the constraint was satisfied and closed, or nobody carried it. The distinction between dropped and violated matters for what you page on, since a dropped constraint is a defect that has not yet cost anything and a violated one is a customer who has already received the email.

Catching the next one before a customer does#

Scoring a run set tells you the rate. Detecting an instance needs an invariant that runs on every conversation, and the one that works here is a conservation property: a constraint extracted from the customer's turns must appear in every subsequent payload of that run until something explicitly discharges it.

def check_constraint_conservation(run: RunTrace) -> list[Violation]:
    live: dict[str, int] = {}          # constraint text -> hop first seen
    out = []
    for hop in run.hops:
        for c in extract_customer_constraints(hop.new_user_messages):
            live.setdefault(c, hop.seq)
        for d in hop.payload.discharged:
            live.pop(d.text, None)     # requires who discharged it and why
        missing = [c for c in live if c not in {x.text for x in hop.payload.constraints}]
        for c in missing:
            out.append(Violation(run.id, hop.seq, hop.from_agent, hop.to_agent, c))
    return out

extract_customer_constraints is a small model call over the new user turns only, so it runs on 100% of conversations for about $0.0009 each, which on 2,400 conversations a day is under $70 a month. Violations page nobody individually; the alert is on the rate, at any edge whose constraint carry rate falls below 98% over a rolling window of 50 transfers, which was chosen because 71.9% was the unmonitored baseline and anything near it should be loud.

What none of this measures is whether the run did the right thing. A handoff can carry all five classes at 100% and still send the conversation to the wrong agent, and a carry rate is a statement about information availability rather than about decisions. It is a lower bound: an agent cannot honour a constraint it never received, and once carry rates are high, a violated constraint becomes a reasoning failure you can investigate rather than a transmission failure you can only guess at.

Investigating it needs something this chapter has quietly assumed. Every computation here pairs a sender's state with its receiver's payload under one run, in order, with hop numbers attached — and the logs the team actually had were per-agent, keyed by agent and timestamp, with no record anywhere of the transfer itself.