A chain's edges are fixed in code; an agent picks at runtime

A design document that calls something an agent has already told its readers what to be worried about, and when the word is wrong it has told them to be worried about the wrong things. That is the practical reason to want a criterion sharp enough to settle the question inside a review, rather than a definition everybody nods at and nobody applies. A misapplied label does not merely offend precision. It points reviewers away from the defects the system actually has, and it fixes the arithmetic they will use to plan capacity for as long as the document is the thing people read.

The criterion turns on control flow, which is the part of a program that decides what runs next. Any program can be drawn as a graph, in which a node is a step and an edge is a link saying which step follows which. A chain — also called a pipeline or a workflow — is a program whose edges are all written down in the source, so they can be traced with a finger before anything executes. An agent is a program in which a large language model, "the model" from here on, is given a fixed set of actions, names one of them on each pass of a loop, and is shown the result so that it can name the next one. The set of available actions is fixed in both designs. What differs is whether the order of them is.

The boundary is where the edges are decided#

A specimen is the fastest way to see what the criterion is asking. At 02:00 a cron job — a task the machine runs on a fixed schedule — selects the tickets created in the last 24 hours, roughly 800 of them; for each ticket it sends the subject and body to the model with a prompt, the standing instruction text that travels with every request, listing eleven categories; it parses the returned category and writes it to tickets.auto_category. The job takes about 22 minutes, mostly waiting on the model, and costs a little under two dollars a night. There is a model in it, it takes several steps, and it does something useful, which is enough for the question to come up.

Draw it as a graph. Every node is there, every edge is there, and both were there when the file was saved:

Diagram

Nothing about that graph depends on what the model returns. The reason is worth walking rather than asserting, because it is the whole criterion in five checkable steps. The model hands back a category string, and that string is data flowing along an edge — it decides the value written to a column, and it is assigned to a variable on the way there. Data can only change which node runs next if some piece of code branches on it, and nothing here branches on the category — the parse step checks the string against the eleven names, and the write step follows whatever the parse step concluded. So the sequence of nodes is identical on every ticket, whatever comes back. If the model returned a category nobody expected, the parse step would reject it and the write step would still be the next thing to happen. You can read the source and enumerate every path through the system before running it, and the number of model calls is 800 whatever the model says.

The last two links are where the leverage is. Enumerability follows from the absence of a branch on model output, and the call count follows from enumerability. Both are properties you can establish by reading, and neither requires observing a single run.

Now draw an agent over the same domain:

Diagram

The solid edges are the loop, and they are fixed in code exactly as they were in chapter 1, where an agent is a model, a set of tools, a loop that feeds every result back, and a condition that ends it. What is not fixed is which of the dotted branches runs on any given iteration, or how many times the cycle goes round, or whether the exit at the right is reached on iteration one or iteration nine. Those are outcomes of the model's output on each pass, and they cannot be determined by reading the source. That is the distinction, and it is the only one that survives contact with real systems: in a chain the edges exist before runtime; in an agent the model selects the next action, from a fixed set, while it is running.

So the nightly job is a chain. It has a loop, but the loop iterates over rows of data rather than over decisions, and no output of the model ever changes what the program does next. Calling it a batch classification pipeline describes it exactly and costs no accuracy at all.

The belief that gets this wrong is not carelessness, and it deserves stating in the form people actually hold it: if it uses a language model and does something useful, then it is an agent. That is tempting for a good reason. The model is the newest and most conspicuous component, it is the part that appears to be making a judgement, and the vendors' own documentation uses the word freely. What is actually true is that the criterion never asks whether a model is present. It asks who owns the edges, and a model can be present without owning a single one.

Two things the criterion deliberately does not care about are worth stating, because both get offered as counter-arguments. It does not care whether a model is involved: a shell script that pipes curl into jq and branches on the result has fixed edges too, and a system with four model calls arranged in a fixed sequence is still a chain. It does not care about complexity or step count either — a twelve-stage pipeline with retries, fan-out and conditional branches is a chain if every one of those branches is a condition you wrote, and a three-line loop is an agent if the model picks the tool.

Those two sentences share a shape worth naming, because it is what makes the mistake survive review. Across all four of those cases the quantity a reviewer is most likely to be reading — how big and how sophisticated the code looks — stays roughly constant, while the property that actually matters varies from one case to the next. The shell script and the twelve-stage pipeline are worlds apart in size and identical in ownership of their edges. The three-line loop is the smallest thing on the list and the only one that has given the exit away. Anybody sorting these systems by apparent sophistication will get every one of them wrong, in both directions.

The genuinely borderline case is a chain that branches on a model's output — a router that asks the model to classify an incoming message as billing, technical or sales, then dispatches to one of three fixed sub-pipelines. The model is influencing control flow, so the criterion has to be applied carefully. What separates it from an agent is that the branch is taken once, the set of destinations is a match statement in your code rather than a set of actions the model names, and there is no loop feeding the outcome back for a second decision. If the router runs again after the sub-pipeline finishes, using its result to choose the next destination, it has become an agent with three tools and a very short toolset — which is a fine thing to build, and should be documented as one so that somebody asks how many times round it can go.

The loop-back is the property doing the work there, and it is worth seeing why it alone is enough. A single branch produces one selection, and one selection can be listed exhaustively: three destinations, three paths, all visible in the match. Feed the outcome back and the second selection depends on what the first destination produced, so the pair of choices is a fact about the run rather than about the source. Add a third pass and the count of possible sequences grows again. Nothing about the destination set changed; the ability to revisit it did.

The same task, built both ways#

Take a task that genuinely admits both designs: a customer writes "why was I charged twice for AB-40912?" and the system must produce an answer.

As a chain, you write the steps because you know them. Fetch the order by the identifier extracted from the message. Fetch its charges. If there is more than one charge, fetch the refund records. Send everything to the model once, with a prompt asking it to write an explanation for the customer. Four steps, three of them ordinary code, one model call, and the graph is in the source.

As an agent, you write the tools and let the model sequence them. get_order, list_charges, list_refunds, search_docs go into TOOLS, the loop from chapter 1 runs, and the model calls get_order, sees two charges, calls list_charges and list_refunds in the same turn, reads that one charge was voided, and answers. The choosing happens in exactly one place: at the top of each iteration, when the response comes back and the loop reads stop_reason and the tool-use blocks. That is the line to mark when someone asks you where the agent decides — it is the return value of client.messages.create, and everything downstream of it in your code is a for loop over what it said.

The useful way to read that table is across rather than down, looking at the gap in each row rather than at which side sounds better. Every gap has the same shape. On the left of the model-call row sits a number; on the right sits a range. On the left of the latency row sits a point estimate; on the right sits a distribution. On the left of the testing row sits an assertion about one output; on the right sits an assertion about a population of runs. The left column holds quantities you can write into a capacity plan, and the right column holds quantities you can only estimate from a sample of runs you have actually observed. Every one of those gaps is a consequence of the first row, which is the only row that states a cause; the other seven are what that cause does downstream.

The chain wins on everything in the bottom half of that table and loses on one row: the case you did not foresee. The customer who writes "why was I charged twice, and can you cancel the second one?" gets a correct explanation and no cancellation from the chain, because there is no node for cancelling; the agent has void_charge in its toolset and will reach for it. Whether that row is worth the rest of the column is the actual design decision, and it depends entirely on how wide the space of incoming requests is. Barry Zhang's talk How We Build Effective Agents makes the case for the chain more forcefully than most agent proponents would like — the recommendation from the team that wrote Anthropic's agent guidance is to not build an agent until a chain has demonstrably failed you.

Why the word in the document matters#

None of this would be worth arguing about if the label had no consequences, and it has three.

It changes what reviewers look for. Reading "agent", the reasonable questions are: what is the iteration bound, what does the loop return when it trips, what is the worst-case cost of one request, what happens when a tool returns nothing forever. Reading "batch classification pipeline", those questions are irrelevant and different ones matter: what happens to the 40 tickets in the batch when the model rate-limits at ticket 612, is the job idempotent on re-run (does a second run leave the same result rather than doubling or overwriting anything), what does the pipeline write when the model returns a twelfth category that is not in the list.

It changes what gets built next. A document that says "our classification agent" establishes agents as the house pattern, and the next feature is designed as one whether or not it needs to be. The eleven-file pull request from chapter 1, the one that was deleted down to about seventy lines without losing any behaviour, exists because someone read a document like this one and inferred an architecture from a noun. The reverse error is rarer and worse: calling a real agent a pipeline hides an unbounded, model-controlled exit behind a word that implies fixed edges, and nobody asks the bounding question because the name says there is nothing to bound.

That reverse case is the same stumble seen from the other side, and it is worth holding both in view at once. In the first direction the code looks bigger than it is and attracts questions it cannot answer. In the second the code looks smaller and safer than it is, and the questions that would have found the unbounded exit are never asked, because nobody interrogates a pipeline about its iteration cap. What stayed constant in both cases was how the system looked on the page. What varied was who owned the exit, and that is the property neither reader was checking.

It changes the cost model people carry in their heads. That third consequence has an identity behind it rather than a warning, and the identity is worth writing out. A chain's bill is a product of three factors: the number of runs, the input size of one run, and the price per unit of input. Every factor is settled before the program starts. The run count is a row count. The input size is the prompt plus one ticket. The price is a published rate. Nothing is left over — there is no term standing for how many times the program went round, because it goes round exactly once per row by construction. That is why the figure can be settled with a multiplication in the design review: 800 tickets times about 700 input tokens times the input rate, which is where the two dollars a night came from.

What that product licenses is prediction without observation. Because no factor depends on what the model returns, no amount of variation in the model's output can move the total, and the same multiplication answers the question at any volume: double the tickets and the bill doubles, and you know that before running anything. One factor is worth naming honestly, because the review wrote down only two of the three. The price per unit of input is never stated — it is recoverable by dividing the nightly figure by the token count, and it is also the factor most likely to change without anybody editing the design document.

An agent's bill has one more factor, and that factor is not a constant. It is the number of iterations, chosen per run by the model and bounded above only by the iteration cap you set. The per-iteration input is not fixed either: every iteration re-sends the whole conversation so far, which is the message list from the previous chapter, where that list turns out to be the entire state carried between turns. So the total is a sum of a growing quantity over a count that varies, rather than a multiple of a fixed one. An agent's cost is a distribution whose tail is set by your iteration cap, and the only honest number is a range with a worst case. Planning with the median understates it, because a right-hand tail pulls the mean above the median, and teams that inherit the wrong word plan capacity with the wrong arithmetic and are surprised in the same direction every time.

Design decisions the requirement did not force#

Separating a requirement from a design decision is what would have found that defect faster, and the separation is easy to make on this system. The requirement is that tickets arrive with a category on them. Everything else is a decision somebody took: that the categorising happens overnight rather than on arrival, that the selection is by creation time within a fixed window, that there are eleven categories rather than some other number, and that the result lands in tickets.auto_category rather than in a new row alongside whatever a human wrote. A requirement can be unmet. A design decision can be wrong in a way nobody notices, because it was never written down as a decision at all.

The overwrite follows from two of those decisions meeting. Rows are selected by when the ticket was created, so a re-run inside the same window selects the same rows a second time. The write had no guard on it — the defect proves that, since a write that checked whether the column already held a human-set value could not have overwritten one. Put those together and a partial failure followed by a re-run is guaranteed to revisit rows that already have a value and to replace it. Neither decision is unreasonable on its own, and the fault is only visible where they meet, which is exactly the kind of fault a review finds by listing decisions rather than by reading code.

The criterion has a limit that is worth being blunt about. It answers one question — who owns the edges — and it does not answer whether the design is good. It says nothing about whether this job should exist, whether eleven categories is the right number, or whether the chain or the agent is the better answer for the charge-explanation task above. Those are separate arguments, and the one that decides them is how wide the space of incoming requests is rather than anything in the definition.

The label is also not a permanent property of a system. It is a property of the source as it stands today, and the router case shows how cheaply it changes: one edge added from a handler back to the classifier converts a chain into an agent, in a diff small enough to pass review on a Friday, with the design document still saying whatever it said last year. That makes the label something to re-check whenever an edge is added, and it makes the document worth updating on the same commit.

The word is worth this much attention because it is load-bearing in three separate places at once. It sets the questions a reviewer asks, and asking the agent set about a chain leaves ordinary pipeline defects — re-run semantics, partial failure, an unexpected value from the model — entirely unexamined. It sets the pattern the next feature is built to. And it sets the arithmetic people use to plan, which is a clean product for one shape and a distribution with a tail for the other. Chapter 7 takes up how to describe the thing to someone outside engineering, where the same word does even more damage.

There is one more consequence of letting the model pick the next action, and it is the one that produces incidents rather than confused meetings. If the model chooses what happens on each iteration, then it is also choosing, on every iteration, whether there is a next iteration at all.