A fixed order of operations is what a chain is for

The nightly job reads 3,000 supplier contracts and reconciles them against the billing system. Two designs were on the whiteboard. Design A is a two-stage chain: stage one extracts eleven structured fields from the contract, stage two takes those fields and validates them against the supplier record — payment terms, currency, rate card version, notice period. Design B is a single tool-using loop, given an extraction tool and a lookup tool, told to reconcile the contract and left to work out how. The senior engineer arguing for B made the case that B can adapt, that it can go back and re-extract when the lookup disagrees, and that A is rigid.

Design A is the right answer here, and the property that decides it is that the operations and their order are known before you see any contract. You cannot validate fields you have not extracted, and there is no contract in the set for which some other operation should happen first, or instead. Handed the same task, the loop's first decision is to call the extraction tool, its second is to call the lookup tool, and its third is to stop. It is choosing from a set of one, three thousand times a night, and paying model tokens for the privilege.

The test that separates the two shapes#

The question is not whether the task is complicated. It is whether the sequence is a function of the data or a constant. Three checks make that concrete, and any one of them failing is a genuine signal for a loop.

Can you write the list of operations down before you see the input? For the reconciliation job you can: extract, then validate. For "diagnose why this test suite started failing" you cannot, because the second action depends entirely on what the first one printed.

Does any step's choice depend on content, or only its result? The reconciliation lookup depends on the extracted supplier ID, which is a result flowing down the pipe. Nothing about the contract's content changes which operation runs next. Contrast a support workflow where a refund request, a bug report and a password reset take three different paths that only become knowable once the message has been read — although note that a small, enumerable set of paths is still a chain with a branch in it, which is the next point.

Is the number of steps bounded and known? Two, always, for the reconciliation. For "follow the references in this contract until you have identified every party", the number of hops depends on the document, and a pipeline with a fixed stage count cannot express it without a loop somewhere.

The second check is the one that gets misread, and the distinction it turns on is between a value flowing forward and a decision being taken. Every pipeline passes results between stages; that is what a pipeline is. The supplier ID extracted in stage one determines what stage two looks up, and it does not determine whether stage two runs or what stage two is. The code would call the lookup on every contract whatever the extraction returned. A step chosen from a result is different in kind: nothing in the code says what happens next, and the answer exists only once the previous step has produced something for a model to read.

None of this is a new class of design. Backend engineers have been building fixed-order pipelines over unreliable steps for decades, in the ETL DAGs an orchestrator runs — a DAG being a directed acyclic graph of steps, each waiting on the ones it depends on — in Unix pipes, in every batch job that extracts, transforms and loads in that order and no other. What is new is only that the transform is a model call, which brings a failure mode those older pipelines did not have — a step that returns something well-formed and wrong — and that failure mode is why the joints need contracts rather than just types. The instinct to reach for the older shape is correct; the instinct to trust each step the way you trust sort is not.

A chain is allowed to branch. A stage that classifies a document into one of four types, followed by code that dispatches to one of four downstream stages, is still a fixed pipeline in the sense that matters, because the set of paths is enumerable, drawable, and testable end to end. What distinguishes a chain from a loop is not the absence of branching, it is that the branch points and their options are chosen by you in code, over validated fields, rather than by the model over its own reasoning at run time.

Diagram

The two diagrams are the same task. The difference is where the arrows are decided: in A they are in your repository, in B they are produced at run time by a model that has to re-derive them for every one of the 3,000 contracts.

What the second design costs, measured#

Both designs were built and run over the same 500-contract sample, on the same model, and the numbers were not close. Wall clock is reported as percentiles, so the p50 row is the time half the contracts came in under and the p95 row is the time nineteen contracts in twenty came in under, which is the figure that matters when something downstream has a timeout.

The token gap is mostly conversation history: every tool result in B is appended to the messages array and resent on the next turn, so the input side grows with each iteration and a 4.7-call mean carries far more than 2.35 times a 2-call chain's input.

That "far more" has a shape worth knowing, because it is what makes loop costs surprising rather than merely higher. On turn one the model is sent the prompt. On turn two it is sent the prompt plus the first tool result. On turn three, the prompt plus the first two results, and so on, so the tokens billed on each turn are roughly the tokens billed on the previous turn plus one more result. Adding those up across a run means the total input is not the per-turn size times the number of turns; it is closer to the size of the last turn times half the turn count, which grows with the square of the number of turns. Doubling a run's length therefore roughly quadruples its input bill. The residual is that only the growing part behaves that way — a long fixed system prompt is resent unchanged every turn and grows linearly, and a provider that serves a repeated prefix from cache charges less for exactly that part.

The accuracy difference is inside the noise of a 500-item sample, which is a topic chapter 6 takes up properly.

What is worth attention is the last two rows, because they are not about money. In A, a contract that fails produces stage=extract contract=SupplierFields error=payment_terms: required, which points at one prompt and one input. In B, a contract that fails produces a transcript, and someone has to read it to find out whether the model extracted badly, looked up the wrong supplier, or looked up the right supplier and misread the response. That reading takes minutes rather than seconds, and it does not aggregate — you cannot count trace-reading across 40 failures the way you can count gate names.

The reason it does not aggregate is mechanical. A gate name is a categorical value the pipeline emits at the moment of failure, so forty failures arrive already labelled and grouping them is one query. A transcript carries no such label; the category exists only after a person has read the transcript and decided what it was. So the cost of categorising forty chain failures is one query, and the cost of categorising forty loop failures is forty readings, which is also why the first thing teams build on top of a loop is a model that reads the traces — a second system, with its own accuracy question.

The last row is where the argument for adaptivity meets its own evidence. A loop that has run out of ideas does not stop and say so; it produces a plausible wrong answer after eleven calls, which is the same outcome as a confident hallucination and costs five times as much. The reassuring belief is that a design free to choose its next step will notice when something is wrong, and what that freedom actually holds constant is the production of an answer. What varies is whether the answer was grounded in a successful lookup, and nothing in the loop's contract with its caller reports that quantity.

Frameworks encode this distinction explicitly. Google's ADK has a SequentialAgent whose whole contract is that its sub-agents run in the order you list them, passing state forward, with no model call spent deciding what runs next; its documentation on sequential agents is a compact statement of what a chain guarantees. In LangGraph the same shape is a graph with static edges, and the moment you add a conditional edge whose condition is a model output rather than a validated field, you have started building B.

Where the argument for B usually goes wrong#

The claim that decides most of these design reviews is that a loop can retry the extraction when validation fails and a fixed chain cannot recover. The first half is true and the second half is false, and chapter 2 already put the recovery in thirteen lines: the while in run_stage re-invokes the stage with the validation error appended, up to a budget, and this is ordinary control flow written at a boundary.

The confusion underneath it is a conflation of two independent questions. One is where the recovery lives — in your code at a gate, or in the model's next-action choice. The other is whether the sequence of steps is fixed or discovered. Those vary separately, and all four combinations exist: a chain with no recovery at all, a chain with per-stage retry budgets, a loop with no retry logic that simply gives up, and a loop that retries within its own reasoning. Arguing "B, because A cannot retry" picks a value on the second axis using an argument about the first.

A second argument that carries more weight than it should is that the loop will handle future requirements without a rewrite. That is true in a narrow sense and misleading in practice, because a requirement you can state — reconcile against a second billing system, flag contracts whose notice period exceeds 90 days — becomes a stage or a branch in design A in an afternoon, with fixtures and a gate, and becomes a sentence added to a system prompt in design B with no way to tell whether it took effect except by re-running the evaluation. Extensibility is not the same as extensibility you can verify, and on a job that writes to a billing system the second one is what has value.

There is a related version that is subtler and also worth naming. People say a chain cannot use the result of stage two to change what stage one did, which is true within a single pass, and then conclude that feedback requires an agent. Feedback within a chain is a loop you write: run the validation stage, and if it reports a specific class of mismatch, re-run extraction with that mismatch in the prompt, at most twice. The reason to be careful about doing this too freely is not that it is impossible, it is that each such edge is a path through the pipeline that has to be tested, and the count of paths is what keeps a chain understandable.

When a fixed shape stops being fixed#

The two designs are also not exclusive, and treating them as a binary is what makes these reviews longer than they need to be. A common and sensible arrangement is a chain whose stages are fixed while one stage is internally a loop, because that one operation genuinely has an unbounded step count — resolving a chain of corporate parents until you reach the ultimate owner, say, or reading through a document's cross-references. The pipeline keeps its fixed order, its gates and its per-stage evaluation; the unbounded work is contained inside a single stage with its own iteration cap, and the stage's contract at the boundary is the same declared structure any other stage would emit. Reserving adaptivity for the part of the task that needs it, rather than making the whole pipeline adaptive because one step is, is usually the design that survives review.

That arrangement passes the three checks at two different levels, which is why it is coherent rather than a compromise. At the level of the pipeline, the step list is known and the count is bounded, so the chain's checks hold. Inside the one stage, the step count is not bounded and the next action is chosen from what the last returned, so the loop's checks hold there. What keeps the two from contaminating each other is the contract at the stage boundary: whatever the loop did inside, what leaves the stage is the declared structure, validated before anything downstream runs.

The signal that the choice should be revisited is not a bad day, it is a trend in the shape of the work. If the chain accumulates a fifth and sixth conditional branch because the input distribution keeps producing document types nobody anticipated, the enumerable set of paths has stopped being enumerable, and the code is now a hand-written approximation of a policy that a model could produce more cheaply. The measurement that shows this is the first of the three checks, run again on a fresh sample rather than on the sample that justified the original design: ask someone to write the step list for thirty recent inputs before running them, and see how often they can.

Choosing the chain settles the architecture and hands you a different problem. Design A is two prompts, and the reconciliation suite scores it as one number — the fraction of contracts reconciled correctly. When that number moves, it tells you the pipeline changed, and it says nothing at all about which of the two prompts is responsible.