Nobody holds the goal after a peer transfer

Several language models can be pointed at one customer conversation in two quite different arrangements. In the first, a manager component invokes a specialist, reads what comes back, and decides what happens next; that shape is the subject of topic 11.9 and it is called orchestrator–worker. In the second, the specialists address each other directly and the manager is deleted. This topic is about the second arrangement, and this chapter is about the single property that separates them: whether anything is still running that knows what the conversation was for.

The second arrangement has a name. A swarm is several agents — each one a language model given its own instructions and its own set of tools — that share a single conversation by passing it directly between themselves, with no manager component sitting above them. The move that passes it is a handoff, which transfers control rather than asking for help: the agent performing it stops running, the agent it names takes over the same conversation under its own instructions and toolset, and that agent may hand on again to a third. Each such transfer is a hop, and one customer's conversation, from their first message to the system's last, is a run. You meet the pattern wherever a system has been split into specialists and shipped: a support assistant that moves a customer from triage to a refunds specialist, a coding assistant that passes a finished patch to a reviewer agent, a research tool whose planner hands a subtask to a retriever.

Two decisions that arrive as one#

Swarms exist because one agent holding every policy and every tool for refunds, invoices, deliveries and warranty claims answers worse and costs more per call than several narrow agents do, and because a conversation that has already started with one specialist still has to be able to reach another. That is a requirement, and it is a good one. Splitting the work into narrow agents is the response to it.

Deleting the component that routes between those agents is a second and separate decision, and it is the one that usually arrives unexamined, because handoffs are a first-class construct in the frameworks teams start from and the shape lands in a codebase before anybody has asked what happens to the run as a whole. It is worth holding the two apart. A support assistant can have five narrow specialists and a router; it can have five narrow specialists and no router; the arguments for the first split say nothing about the second.

One team made the second decision deliberately. An earlier version of their support assistant used the orchestrator–worker shape from topic 11.9: one component held the objective, invoked a specialist, read the result, and decided what happened next. The team replaced it with peer transfers, on the reasoning that the orchestrator was an extra model call in the middle of every step and the specialists could hand off to each other directly. What they did not price in was that the orchestrator was also the only place in the system where the sentence "this conversation is about a refund on order 41182 and it is not finished" existed as state rather than as an inference over a transcript.

What a transfer actually does#

A handoff is a tool call whose return value is an agent. The agent that calls it stops running, the returned agent starts running, and the conversation carries on with a different set of instructions and a different toolset attached to it. Implementations differ in what they attach to the transfer, but the mechanism is always the same reassignment, and it is short enough to read in full.

def transfer_to_refunds() -> Agent:
    """Hand the conversation to the refunds agent."""
    return refunds_agent

triage_agent = Agent(
    name="triage",
    instructions="Identify the customer's issue and transfer to the right specialist.",
    tools=[transfer_to_refunds, transfer_to_billing, transfer_to_orders],
)

def run(agent: Agent, messages: list[dict]) -> list[dict]:
    while True:
        response = model.call(agent.instructions, messages, tools=agent.tools)
        messages += response.messages
        if not response.tool_calls:
            return messages                    # whoever holds control has decided to stop
        for call in response.tool_calls:
            result = execute(call)
            if isinstance(result, Agent):
                agent = result                 # the whole of the handoff mechanism
                break

Two lines in that loop carry the entire design. agent = result moves control, and return messages ends the run. Both are decisions made by whichever agent happens to be executing at the time, from inside its own turn, on the basis of the conversation in front of it. There is no expression anywhere in the loop for "the objective", "the steps completed so far", or "the condition under which this is finished", and there is nowhere for an outside caller to intervene between one agent and the next.

It is worth walking the consequence of that in steps, because it is the step people skip. The variable agent is a local of run, rebound from inside the loop by a value a model produced. Nothing outside the loop can read it, so no external component can know which agent currently holds the conversation. The loop exits on exactly one condition, not response.tool_calls, which is a property of one model response. Therefore the decision to end the run is taken by an agent that has been shown one turn's worth of context and never told what the run was for. And because the loop takes no other exit, there is no branch for an outside caller to reach — the absence of a stop button in the operator's dashboard is not a missing feature above this loop, it is this loop having no place to put one.

The difference in shape from the orchestrator design is worth seeing rather than describing, because the property that matters is topological.

In the left-hand shape every path returns to the centre, so there is a moment after each step at which one component has seen everything that has happened and is deciding what happens next. In the right-hand shape there are no return edges at all. Control is a token that moves forward, and the only component that ever sees the whole run is the transcript, which is a record rather than an actor.

An analogy helps here as long as its limit is stated with it. A relay race also passes a token forward and never back, and the runner holding the baton is the only one moving. Where the picture breaks is the part that matters: a relay has a finish line fixed by the rules of the race and a judge at it, and neither of those is a runner. In a swarm the finish line is a judgement made by whoever is holding the baton at the time, out of the same context they are using to decide everything else, which is why the two designs differ in what they can guarantee rather than in how fast they move.

Four things the orchestrator was holding#

The objective is the first. In the orchestrator design it exists as a field that was written once, at the top of the run, and is passed unchanged into each routing decision. In the swarm it exists only as the customer's opening message, somewhere in a transcript that is now several thousand tokens long, competing for attention with everything said since.

The completion test is the second. An orchestrator can express "this run is done when a refund exists and the customer has been told about it" as a condition it evaluates after each result. A swarm has to hope that the agent currently holding control both knows that condition and recognises that it has been met, and the agent currently holding control is often a specialist that was handed a narrow task and has no idea what the run as a whole was for.

The stop is the third. You can kill the process, and in a system that has already caused side effects that is not the same as stopping the run. In run_8f2c1a, a support conversation that had already passed through several agents, the refund was issued at the third hop; killing the process at the sixth would have left a refunded customer with no notification and no component anywhere that knew the run had been abandoned mid-way. An orchestrator has an obvious place to record abandoned_after_refund and to schedule the compensating notification, because it has a component whose job is to know what stage the run is at.

The retry is the fourth. When the notifications provider returns a 503, an orchestrator catches it, still holds the compact task description, and can re-invoke the worker with the same input or route to a fallback. In a swarm the 503 surfaces inside the notifications agent's own turn, and that agent has exactly two moves available to it: try again, or transfer. Transferring a failure is how a run acquires a second agent that will fail at the same thing, and it is the most common way a swarm turns a transient provider error into a loop.

Those four are one property seen from four sides. Each of them is a decision that has to be taken between two agent turns, using information from more than one of them, and a design whose only transition is agent = result has no place between two turns for anything to happen.

The hop that was supposed to be cheaper#

The argument for peer transfer that circulates most widely is that it removes an inference call from every step. The orchestrator has to think between each pair of specialists, and if the specialists can address each other directly, that thinking disappears and the system gets cheaper and faster for the same work. The first half of that is true and the conclusion does not follow, because the transfer decision is itself an inference call, and it is made by an agent whose context is the whole conversation rather than a compact routing state.

Before any figures, it is worth writing down what a conversation costs, because the shape of the expression is what the argument turns on. A conversation makes some number of routing calls and some number of specialist calls. Its input bill is the count of routing calls times the input tokens of one, plus the count of specialist calls times the input tokens of one, all multiplied by the input price. Its output bill has the same shape at the output price. Deleting the orchestrator sets the first count to zero, which is a real saving. It also changes the input tokens of a specialist call, and that term is multiplied by the larger count.

The numbers below come from replaying the same 200 recorded support conversations through both designs, at a list price of $5 per million input tokens and $25 per million output tokens, and reporting the median conversation. Where a percentile appears, p50 is the value half the conversations come in under and p99 the value all but the worst one in a hundred stay under.

Read the first two rows into the expression above and the third row falls out: five routing calls at 1.4k in and twelve specialist calls at 6.8k in are what account for design A's 88.6k of input, and thirteen specialist calls at 17.9k in are the whole of design B's 232.7k, because its first count is zero. The output rows work the same way from the 60 and 400 and 430 figures. Multiplying each total by its price and adding gives the $0.57 and the $1.30, and nothing else is in those numbers.

The orchestrator's routing calls are real and they are cheap: 1.4k tokens of compact state in, sixty tokens of destination and reason out, about $0.009 each and roughly 0.7 s of serial latency. Design B deletes five of those and pays for the deletion in two places. Every agent's prompt now carries the tool schemas for the peers it can reach — a tool schema being the JSON description of a tool's name, arguments and purpose that the model has to be shown before it can call it — which is about 780 tokens added to every call of every agent, and every receiving agent inherits the full conversation because the transcript is what the transfer passes by default. Those two effects are what take a specialist call from 6.8k of input to 17.9k, which is 2.6× the input on every call, and there are more calls to spend it on.

The place to read that table is the gaps rather than the levels. The gap in the routing-calls row is a saving of five cheap calls. The gap in the specialist-input figure is a surcharge on thirteen expensive ones. A design argument that quotes the first gap and not the second is not wrong about the first; it has compared the two designs on one term of an expression with two.

Latency behaves differently from cost, which is why the two often get argued about together and should not be. Design B does remove serial round trips, and on a conversation that goes triage → refunds → done, it is genuinely faster by about a second. The p50 gap of 1.6 s in the other direction comes from prefill, which is the pass a model makes over the whole prompt before it can emit its first token and therefore takes longer on the larger prompts design B produces, and the p99 gap comes from the hop distribution: single-issue conversations settle in 4.1 hops in design B, while conversations that raise a second issue mid-way run to 9.3 hops at p50 and 24 at p99.

That the two behave differently is the useful part. Cost is the product of a count and a per-hop price, so it grows with both. Latency is a sum over hops that a shorter prompt shortens and a longer chain lengthens, and on a two-hop conversation the two effects point in opposite directions. A design cannot be argued for on "faster and cheaper" as though those were one property, because the arithmetic underneath them is not the same arithmetic.

Second issues do not settle the choice#

The workload that makes this decision interesting is one where 15% of conversations raise a second, unrelated issue part-way through — a refund request that turns into a billing dispute, a delivery question that becomes a returns question. It is tempting to treat that as an argument for the swarm, because a peer transfer looks like the natural way to say "this is now someone else's problem". Design A handles second issues perfectly well: the orchestrator reads the specialist's result, sees an unresolved new request, and routes again. The routing logic does not care whether the new issue is the first or the third.

What the 15% actually changes is the shape of the run's tail. Multi-issue conversations are the long ones, the ones that revisit agents, and the ones where "is this finished?" stops having an obvious answer, so they are precisely the conversations on which the four capabilities above earn their keep. On that workload, and with nothing else in place, design A is the one to pick, because the failures that workload produces are stop-and-report failures and design A is the only one of the two that has a component able to stop and report.

The table that produced these figures has a limit worth stating with it, because it is the kind that survives into every comparison later in this topic. Both columns come from replaying the same 200 recorded conversations, which means both designs were shown a fixed set of customer turns. A real deployment does not work that way: what a customer says next depends on what the system just said, so a design that answers badly at hop two produces a different conversation from hop three onwards. A replay holds the conversation still and measures what each design does to it, which is the right way to isolate the routing machinery and the wrong way to estimate what either design costs in production. The figures are a comparison of mechanisms, and they are a floor on the divergence rather than an estimate of it.

There is also a neighbouring pattern the handoff gets confused with, and separating them removes a class of argument that goes nowhere. A sub-agent invoked as a tool returns a value to its caller, and the caller stays alive. Retries and timeouts around it are ordinary engineering, because there is a frame on the stack waiting for a result and that frame can catch an exception. A handoff ends the caller's turn. Every instinct that relies on a caller still being there to catch something has to be rebuilt, and the four capabilities above are that rebuild, listed.

The single fact underneath all of this is that agent = result has no inverse. An orchestrator holds the objective, the completion test, the stop and the retry because it is still on the stack when a specialist returns; a swarm holds none of them because nothing is. Every difference in the cost table follows from the same absence: with no compact routing state to hand forward, the transcript is what travels, and a growing transcript is what makes a specialist call in design B carry 2.6× the input of the same call in design A.

That is a verdict about the swarm as it usually arrives, rather than about peer transfer as an idea. The rest of this topic is the work that makes design B defensible: giving the transfer a payload rather than a transcript, deciding which agents may reach which, making routing decisions visible after the fact, and attaching budgets and termination conditions to the run rather than to each agent's turn. Control has moved, and the next question is what moved with it, where the default answer — the conversation so far — is the first class of failure.