Planning upfront and reasoning between steps differ mechanically

The ops team at a payments company runs a single agent for internal data work, and one Tuesday it picks up two tickets an hour apart.

The first: reconcile March's invoice from the card processor against the internal ledger, and issue account credits wherever the processor overcharged. The processor's export has 61,400 charge lines. Somewhere in the middle of that task the agent will call issue_credit, and every one of those calls moves real money and sends a customer an email.

The second: last night's aggregation job wrote 41% fewer rows than the night before. Find out why.

Both tickets go into the same loop — the one built from scratch in topic 10.3, where a model call returns tool calls, the harness executes them, the results come back as messages, and the loop turns again. What differs is whether the agent writes down the whole route before it takes the first step. On the reconciliation ticket, writing it down first is clearly the right call. On the row-count investigation, it is close to useless. The reason has nothing to do with which one sounds more thorough, and everything to do with what each approach does to the message array.

What the agent writes down when it plans#

The thing one ticket gets and the other does not has a name. Planning, in an agent, is the model producing the sequence of actions it intends to take as an explicit output — a list of steps, each naming a tool and the arguments that tool will be called with — before the first of those actions is executed, and then keeping that list where the later turns of the loop can read it. The list itself is the plan, and it has no special standing inside the model: it is text the model wrote a moment ago, and the only reason the agent follows it is that the harness either keeps showing it to the model or walks the steps itself. The two arrangements this chapter compares sit at the ends of that idea. Upfront planning produces the whole list before any tool runs. Interleaved reasoning produces no list at all, and decides each action from what the previous action returned.

Planning exists because the loop by itself has no way to say what it is about to do. A turn reads the transcript, chooses one tool call, and the harness executes it, so until the call has been issued the agent's intention lives inside the model and nowhere else, which means a log, a policy check or the engineer on call learns of each action at the moment it takes effect. On a run whose actions are read-only queries that arrangement is fine, because a wrong query costs a few seconds of warehouse time. On a run that ends in issue_credit, it means the first observable evidence of a mistake is the mistake. Writing the route down turns the agent's intention into an object that exists while the side effects are still in the future, and a smaller benefit comes with it on long tasks, because a route the model re-reads on every turn holds it to an ordering that a forty-message transcript tends to blur.

Each link in that argument is checkable on its own, and it is worth walking rather than accepting. A model call returns tool calls and nothing else — there is no field in the response that says what the model intends to do after this one. The harness therefore holds a single call and no description of the route it belongs to. So the only account of that route anywhere in the system is the one inside the model, and text that has not been emitted cannot be read by a log, a policy check or a person. What follows is not an extra assumption but a restatement: the earliest instant at which anything outside the model can learn of an action is the instant the harness dispatches it, and dispatching it is what makes it happen. Announcing an intention and carrying it out are the same event, and a plan is the machinery that pulls them apart.

Most readers have met the result from the outside. A coding assistant that prints a numbered to-do list before it edits any files, and ticks items off as it works, is running a plan; the approval screen that shows a proposed set of changes and waits before touching the repository is the same mechanism with a person standing in the middle of it. In frameworks the arrangement is called plan-and-execute — a planner call that writes the steps, an executor that walks them, and a replanner that revises the list when a step fails — and inside a product it usually appears as a tool named something like write_plan or update_todos that the model is required to call before it is allowed to touch anything else.

Two different things are running together in all of that, and prising them apart decides what you are actually choosing between. The requirement is that something outside the model can refuse an action before it takes effect. The design decision is to meet that requirement by producing the entire route as one object at the start of the run. Those are not the same thing, and the requirement has at least one other answer: hold each destructive call for approval at the moment the model proposes it, one step ahead, with no plan existing anywhere. That answer buys a narrower window — a reviewer sees one action rather than a route — and it costs nothing per turn, because there is no extra object to generate or carry. Upfront planning is the decision to buy a wider window at a fixed, prepaid price.

What the two shapes look like in the transcript#

An interleaved agent has no artifact called a plan. Its first model call sees the system prompt, the tool schemas and the user's request, and returns one or two tool calls. The harness runs them, appends the results, and calls the model again. Whatever route the agent takes emerges one step at a time, and each decision is made with everything the previous steps returned already in context.

# Interleaved: the first model call goes straight to a tool.
messages = [
    {"role": "user", "content": "Nightly agg job wrote 41% fewer rows. Find out why."},
]

# turn 1  -> tool_use: run_sql("select count(*), src from staging.events "
#                              "where day = current_date - 1 group by src")
# turn 2  <- observation: 3 sources present, one missing entirely (src='mobile-sdk')
#         -> tool_use: read_logs(job="nightly_agg", since="24h", grep="mobile-sdk")
# turn 3  <- observation: 0 matching lines; the loader never ran
#         -> tool_use: describe_dag_run(dag="mobile_ingest", date="2026-03-11")

Turn 2 could not have been written before turn 1 returned, because the whole point of turn 1 was to find out which source was missing. There are perhaps a dozen plausible second steps and the observation picks one.

It is worth making "could not have been written" precise, because that one sentence carries the whole mechanical difference and everything later in the chapter leans on it. Call the set of things the agent has available when it decides step k the information at that decision. Under interleaved reasoning, step k is decided after steps one through k minus one have run and returned, so its information is the request plus every observation so far. Under upfront planning, every step is decided during the planning call, so the information behind each of them is the request and nothing else. Subtract one from the other and the shortfall for step k is exactly the observations produced by the steps before it — no more, and nothing that depends on the model, the task or the tools.

Set k to one and that shortfall is empty. Under either shape the first action is chosen with the request and nothing else, because at that instant no observation exists under either shape. Planning cannot improve the first action and cannot damage it. Everything planning does to the quality of the agent's choices therefore begins at the second action, which licenses a narrow and useful claim: on a task with one real action, planning changes the run's cost and changes nothing else at all. It is a cancellation with no residual, and the cancellation is the finding.

An upfront-planning agent inserts one extra model call before any tool runs. That call produces a plan, the plan gets appended to the message array as an assistant turn (or written through a write_plan tool, which chapter 2 takes up), and every subsequent model call in the run sees it.

# Upfront plan: one model call before the first tool call.
messages = [
    {"role": "user", "content": "Reconcile March processor invoice; credit overcharges."},
    {"role": "assistant", "content": PLAN},   # ~1,200 tokens, produced in ~3.1s
]

# every later turn re-sends: user turn + PLAN + all observations so far

PLAN is the load-bearing part, and it is worth seeing what 1,200 tokens of it actually contains rather than imagining it.

{
  "goal": "Reconcile 2026-03 processor invoice against internal ledger; credit overcharges.",
  "steps": [
    {"n": 1, "tool": "s3.get", "args": {"key": "vendor/exports/2026-03-invoice.csv"}},
    {"n": 2, "tool": "warehouse.load", "args": {"target": "staging.vendor_invoice_202603"}},
    {"n": 3, "tool": "run_sql", "args": {"file": "sql/reconcile_march.sql"}},
    {"n": 4, "tool": "write_report", "args": {"path": "reports/2026-03-recon.md"}},
    {"n": 5, "tool": "issue_credit", "args": {"from": "step_3.overcharges", "cap_usd": 500}}
  ]
}

Step 5 is the reason the plan exists. It is the step that spends money, and it is the last one, which means there is a window — everything from the moment the plan is emitted to the moment step 5 executes — during which a human or an automated check can read the intended action and stop it. On the reconciliation ticket, an agent that discovers at step 5 that it misread the invoice's sign convention and credited 61,400 customers has caused an incident that takes a week to unwind. The plan's value is that the mistake is legible before the money moves.

Notice what fixes the length of that window, because it is a property of the plan's shape rather than of anybody's diligence. The window opens when the plan is emitted and closes when the money-spending step is dispatched, so its length is the wall-clock time taken by every step in front of that one. Put the expensive action last and the window is the whole run. Put it first and the window is whatever the gap between emitting the plan and starting is, which on an unattended run is nothing. A plan does not create review time by existing; it creates review time in proportion to how much cheap work stands between the plan and the irreversible action, and an agent whose first action is the dangerous one gets a written record and no window at all.

The two costs, in the units you actually pay them in#

An upfront plan costs latency once and tokens repeatedly. The latency is the plan-generation call: on this agent, 1,200 output tokens at roughly 3.1 seconds. The token cost is not paid once, because the plan sits in the message array and is re-sent with every later request. A four-step task involves five model calls after planning, so those 1,200 tokens are billed as input five times over — 6,000 token-turns of prompt that the interleaved version never pays. Prompt caching — where the provider stores the unchanged prefix of a request and bills a fraction of the usual rate when the next request repeats it — flattens most of the price of that, but caching does not give back the context window the plan occupies, and it does not stop the model from reading the plan and following it.

The multiplier in that paragraph is worth deriving rather than taking on trust, because the general form says something the single case hides. A run of n steps needs one model call to choose each step, which is n calls, and one more at the end to write the answer once the last observation has come back. So it makes n plus one model calls after the planning call. The plan is in the message array for all of them, since nothing removes it. A plan of some size is therefore generated once and re-sent n plus one times, and the prompt cost it adds over the run is its own size multiplied by n plus one. The four-step case above is that formula with n set to four: five later calls, and five resends of the same twelve hundred tokens.

Reading the formula as a shape rather than as a number is where it pays. The plan's cost rises with the length of the run it governs, one resend per additional step, without limit. Its benefit does not rise at all. A plan buys one review window per run — one object, produced once, read once, before the first side effect — and that is the same single window whether the run has three steps or thirty. A cost linear in the number of steps against a benefit flat in it means there is always some task length past which the arithmetic turns over, and where that length falls is a fact about a particular agent rather than about planning. This is why the honest form of the decision is a routing rule per task class, which chapter 7 measures, and why the plan's own size is never the deciding quantity on its own.

That last point about caching is the one that bites. The plan was written from what the agent knew before it had seen anything, which means it is a set of predictions about the world. On the reconciliation ticket the predictions are safe: the export is where it always is, the staging table has the same shape it had in February, the reconciliation SQL is checked into the repo. On the row-count ticket, a plan written at t=0 predicts what the logs will say, and it will be wrong, and the model will carry the wrong prediction in context for the rest of the run.

The table is more useful read across than read down, because every row is one difference and the differences are not independent. The first row is the cause and the six under it are consequences of it. Deciding the route once, before any tool runs, is what forces the extra model call, which is what buys the latency and the resident tokens; it is also what puts the whole path on paper, which is what makes it reviewable; and it is what leaves an object in context that no observation updates, which is what lets the route go stale. One design decision, taken at the top, pays out as three costs and two benefits, and there is no row that can be had without the first row. That is why the last row is a routing question rather than a preference: you are choosing the cause, and the consequences arrive as a set.

The row-count investigation lands on the wrong side of nearly every row. Its first action is a read-only query, which costs a few cents of warehouse time and can be run again five different ways at no risk. Its second action is genuinely undecidable in advance. And the branching factor is high: a missing source, a schema change, a partition that landed late, an upstream vendor outage, and a bug in the aggregation SQL all present the same symptom on the first observation and diverge immediately after. Writing a five-step plan for that task produces five steps that describe one of those five worlds, and four times out of five the agent then spends the run reconciling its observations against a document that was fiction from the start.

The fraction at the end of that paragraph is arithmetic on one case, and the algebra behind it settles every case. Suppose the symptom is consistent with n distinct explanations, and the plan commits to one of them at a moment when no evidence separates them. The plan then describes the real world in one case out of n, and describes a world that does not exist in all the rest — every case but one, out of n. With five candidate explanations that is four cases in five, which is the figure above; with more explanations the fraction climbs, approaching certainty and never arriving there. So the penalty for planning a branching task grows with the branching, and it stays a tax rather than turning into a guarantee of failure, which matches what the transcripts show.

One assumption is doing work in that derivation and hiding it would be dishonest. It treats the n explanations as equally likely, and they are not — a partition landing late is commoner than an upstream vendor outage, and a planner that has seen this queue before will lean toward the common ones, which lifts its hit rate above one in n. The improvement has a ceiling that is easy to state: it cannot exceed the probability of the single commonest explanation, because always naming the favourite is the best a guess with no evidence can do. Where that probability is itself low, which is precisely what "the branching factor is high" describes, the correction is small and the conclusion is unchanged.

The reconciliation ticket lands on the other side just as cleanly. The path is a pipeline that ops has run eleven times. There is essentially one route through it. And the terminal action is irreversible in the specific sense that matters: undoing 61,400 credits is not a rollback, it is a project.

Those two words, "irreversible" and "expensive", are doing separable jobs and it is worth keeping them apart. An expensive wrong action can be undone at a price you can quote. An irreversible one cannot be undone at any price, and the reconciliation case is the second kind wearing the first kind's clothes: a credit can be clawed back in the ledger, and the email that told a customer their money was coming back cannot be unsent. When a step's effects reach outside systems you control, reversibility stops being a property of your database and becomes a property of the world, and a plan is the only artefact in the run that exists before that boundary is crossed.

Chain-of-thought, workflows, and the middle ground#

Upfront planning is not chain-of-thought, which is the model working through the problem in prose inside one response before it answers. Reasoning tokens — the part of that response the harness treats as thinking rather than as the answer — improve that call's output and then, in most harnesses, are dropped or summarised before the next turn. A plan is a durable object that persists across turns and constrains later ones. Thinking harder about step 1 and writing down all five steps are different mechanisms with different costs, and an agent can do either, both, or neither.

The property that separates them is lifetime, and it is the one to hold onto. Chain-of-thought is consumed by the call that produced it and then, in the common configuration, stops existing. A plan is produced by one call and read by every call after it. So chain-of-thought can only affect the decision it was generated for, while a plan affects every decision downstream of it — which is why chain-of-thought cannot go stale and a plan can. Staleness needs something to persist unchanged while the world moves, and reasoning that is discarded at the end of its turn has nothing to persist.

It is also not a workflow. When an engineer draws the reconciliation pipeline as a five-node graph in code, that graph is authored once, reviewed in a pull request, covered by tests, and versioned. A model-generated plan is authored fresh at runtime for this one request and reviewed by nobody unless somebody built the review. The two look identical on a whiteboard and have almost nothing in common operationally, which is the subject of chapter 4.

The two are also not exclusive, which the framing so far has understated for the sake of naming the mechanism cleanly. Most production agents that plan also reason between steps: the plan sets the route, and each turn still sees the last observation and can deviate. That middle ground is where the rest of this topic lives, and it introduces its own problem, because an agent holding both a plan and a contradicting observation has to decide which one wins. The two shapes described above are the ends of that spectrum, and knowing what each end costs is what makes the middle designable.

And it does not make the model better at the task. There is a persistent intuition that an agent which plans first "knows what it's doing" and will therefore make better choices — the same intuition that says a person who writes an outline writes a better essay. The mechanism does not transfer. The agent's step 3 decision under interleaved reasoning is made with strictly more information than the same decision made at planning time, because by then the results of steps 1 and 2 are in context. A plan trades information for commitment. That trade is worth making when commitment buys you something — a review window, a stable order for a pipeline with real dependencies, a budget you can approve before it is spent — and it is a straight loss when it buys you nothing.

The outline is an analogy and it should be labelled as one, because it is carrying persuasive weight in an argument about mechanism. It breaks at a single joint, and the joint is where the information sits. An essayist writing an outline already holds everything they are going to know; the argument is in their head, and the outline reorders material that is entirely available at the moment of writing. An agent writing a plan holds almost none of what it will need, and the missing part arrives from the very tool calls the plan is scheduling. Outlining is a reordering problem and planning here is a prediction problem, and no property of the first carries over to the second.

There is a sharper way to say why that intuition survives contact with the evidence, and the shape recurs throughout this topic. The quantities a team looks at when it decides whether to plan are the plan's size and the plan's latency. Both are fixed at the moment the plan is generated and neither moves again for the rest of the run. The quantity that actually decides whether the plan was worth generating is a different one: how far the world has drifted from what the plan predicted. That one starts at nothing and grows with every observation, and it varies enormously between task classes and between two runs of the same class. A cost that is constant and a benefit that varies is exactly the arrangement in which the constant gets mistaken for the whole story, because the constant is the part that shows up in a dashboard.

Where the drift is visible from is what makes it stick. It is not visible from inside the plan, and it cannot be, since a plan has no field that could record it and no input that could produce one — nothing in the plan-generation call's inputs is a function of anything that happens after that call. A plan is therefore a statement about what the model believed before it looked. It certifies exactly that, and it certifies nothing whatever about the world it describes. The row-count run in the scenario is what that looks like from the inside: every step matched the document, no check anywhere would have complained, and the document was fiction from turn 0.

Everything above frames the plan as a cost centre with one benefit: somebody can read it before anything happens. That framing only holds up if the plan is actually readable — if it exists somewhere a log aggregator, a policy check or a person on call can get at it, and if its contents are specific enough that reading it tells you something. The plan shown earlier names tools and arguments; a plan that said "gather the data, then reconcile it, then handle any discrepancies" would be a paragraph of prose that no check can evaluate and no reviewer can disagree with.

So the two shapes differ in one mechanical fact and everything else follows from it. Upfront planning decides every action at a moment when the information behind those decisions is the request alone, and it then keeps that decision in context, at a resend per step, for the whole run. Interleaved reasoning decides each action with everything the run has learned so far, and keeps nothing. Planning therefore cannot help the first action, cannot help the model reason, and can only earn its price through the window it opens — a window whose width is the cheap work standing in front of the expensive step, and whose value is whatever a reviewer or a program does with it. The first real design question is not whether to plan. It is where the plan goes and what it has to contain before the review window is worth anything at all.