Model-controlled iteration fits inside one node of a fixed graph

A branch that a rule cannot handle is a narrow finding, and it licenses a narrow change. The instrument that produces it is a fallback counter — the count of inputs a fixed graph could not process, filed with a reason code — and the promise attached to building one is that the number will change the design when it comes back high. Honouring that promise means knowing exactly how much of the design the number touches, because the tempting reading is that a high fallback rate condemns the graph, and the honest reading is that it condemns one edge of it.

Three terms carry this chapter. A node is one step of a fixed graph: a function with a declared input type and a declared output type, which the graph calls and whose result the graph routes on. A budget is a limit the node enforces on itself — how many times round, how many tokens, how long — checked in code rather than requested in a prompt. A refusal is a value the node returns to say it did not produce a result, carrying the reason it stopped. The argument below is that those three together let a model-controlled loop sit inside a pipeline without the pipeline losing the properties that made it testable, and that a pipeline is worth defending precisely because those properties were expensive to get.

What a high fallback rate licenses, and what it does not#

Take the case that motivates the whole move. A receipt-processing pipeline reported a fallback rate of 11.4%. That is 2,700 receipts a week landing in the human queue, at a measured 70 seconds of review each, which is 52 hours of somebody's week. Worse for the argument, the reason codes did not cluster the way the invoice job's did. The pipeline had seen 3,100 distinct merchants in a quarter; the fallbacks were photographs of crumpled thermal paper, receipts in Portuguese and Turkish, handwritten tip lines added after printing, three receipts stapled together and photographed as one, and a long list of formats occurring twice a month each. There was no fourth template to write, because there was no fourth anything — the tail was the shape chapter 4 described, and no amount of further enumeration was going to bring the rate down.

This is the situation the instrumentation from chapter 3 was supposed to produce, and the change it justifies is narrow. Work out what the measurement actually ranged over. The counter fired on inputs reaching one branch, the branch taken when format detection matched nothing. It did not fire on the other branch, so the 88.6% of receipts that match a known merchant format still match, and nothing in the measurement is evidence about them. The counter said nothing about what happens after extraction either, so the ledger write is still not idempotent and the £25,000 approval threshold still applies. A measurement licenses conclusions about the population it was taken over, and this one was taken over the unmatched branch alone. That is the branch where the loop goes.

The double-bordered node is the only place in the graph where a model chooses what happens next. Everything upstream and downstream is unchanged code with unchanged tests, and the node has the same contract as the template extractor it sits beside: it takes OCR text and returns either a Fields object or a refusal. It is a drop-in for a function, which is what makes the rest of the graph's enumeration survive. The reasoning behind that last clause is worth walking, because it is the load-bearing step of the design. The graph's enumeration is a claim about the set of values that can flow along its edges. An edge carries whatever the node upstream returns. If the new node's return type is exactly the return type of the node it replaced, the set of values on that edge is unchanged. Therefore every downstream branch that was exhaustive before is still exhaustive, and no downstream test has to be rewritten.

The node, with every budget written down#

An unbounded loop inside a bounded graph would give back everything the graph was protecting, so the budgets are the substance of the implementation rather than an afterthought:

PARSE_TOOLS = [crop_and_reocr, lookup_merchant, convert_currency]   # all read-only

def parse_unmatched(text: str, image_id: str) -> Fields | Refusal:
    deadline = time.monotonic() + 25.0
    messages = [{"role": "user", "content": PARSE_PROMPT.format(text=text)}]
    tokens_used = 0

    for step in range(6):
        if time.monotonic() > deadline or tokens_used > 40_000:
            return Refusal(reason="budget_exhausted", steps=step, tokens=tokens_used)

        reply = model.call(messages, tools=PARSE_TOOLS, timeout=deadline - time.monotonic())
        tokens_used += reply.usage.input_tokens + reply.usage.output_tokens
        messages.append(reply.as_message())

        if reply.final_json is not None:
            fields = Fields.model_validate(reply.final_json)   # raises -> caught by caller
            return fields

        for call in reply.tool_calls:                          # names checked against PARSE_TOOLS
            messages.append(run_tool(call, image_id, deadline))

    return Refusal(reason="step_cap", steps=6, tokens=tokens_used)

Read that table across rather than down. The interesting quantity is not the size of any budget; it is the gap between the right-hand column and the set of states the caller already handles, and that gap is zero on every row. A step cap breach becomes a refusal, which routes to the human queue the graph already had. A token or clock breach becomes the same refusal. An unknown tool name becomes a hard error, which the caller was already catching. A schema violation becomes a validation error, which routes to the same queue. Five ways for the node to fail, five landings the graph already knew about, and no sixth landing anywhere. That zero is the design, and it is why validate and write_ledger did not have to change.

The five budget values themselves are design decisions, and they should read as decisions rather than as facts about the world. The requirement is that unmatched receipts must not silently degrade the ledger. Six steps, forty thousand tokens, twenty-five seconds and three read-only tools are one way of meeting it; other numbers would meet it too, and the chapter's own evidence later says which of them had slack in it. Keeping the two apart matters because a requirement is argued with the business and a budget is argued with a measurement, and a design review that confuses them ends up negotiating the wrong one.

The reachable state space of the system is still enumerated: known-format success, unmatched-and-parsed, unmatched-and-refused, validation failure, ledger write. Five outcomes, same as before, and one of them is now reached by a path of variable length whose length is capped at six.

The read-only constraint on the tool set is the load-bearing one. crop_and_reocr re-runs OCR on a region, lookup_merchant reads a merchants table, convert_currency hits a rates cache, and none of them mutate anything the business cares about. The write to the ledger happens two nodes later, in code, after validation, at the same threshold as always. If the loop is wrong, the cost is a bad Fields object that validation may or may not catch, which is bounded and recoverable. The moment somebody adds write_ledger to PARSE_TOOLS because it saves a hop, the enumeration argument for the whole downstream graph is void, and that change looks like a two-line diff.

The contract that makes it a drop-in#

Being a drop-in for a function is a property that has to be enforced rather than intended, and three parts of the signature carry it.

The return type is a union of success and an explicit refusal, so the caller is forced by the type checker to handle both. A node that signalled failure by raising, or by returning Fields with empty strings in it, would push the refusal handling into a try block or into a downstream null check, and either of those is a place where a refusal can be accidentally treated as a result. Refusal carries the reason, the step count and the token count, which is what makes the node's own behaviour measurable without reading traces.

The tool names are checked against PARSE_TOOLS rather than dispatched dynamically, for the reason chapter 1 gave about closed sets. A model emitting a call to write_ledger — which it has never been shown, and which occasionally happens anyway when a prompt mentions the ledger in passing — produces a hard error rather than a lookup miss that silently does nothing.

And the node is testable without the model. Most of what can go wrong in it is budget arithmetic, and budget arithmetic is deterministic:

def test_step_cap_refuses_rather_than_looping():
    with stub_model(always_calls="crop_and_reocr"):
        out = parse_unmatched(SAMPLE_TEXT, "img-1")
    assert isinstance(out, Refusal) and out.reason == "step_cap" and out.steps == 6

def test_deadline_is_propagated_to_the_model_call():
    with stub_model(sleeps=30.0) as m:
        out = parse_unmatched(SAMPLE_TEXT, "img-2")
    assert isinstance(out, Refusal)
    assert m.last_timeout <= 25.0        # the node never outlives its own deadline

def test_unknown_tool_name_is_fatal():
    with stub_model(calls_tool="write_ledger"):
        with pytest.raises(UnknownTool):
            parse_unmatched(SAMPLE_TEXT, "img-3")

Those three run in under a second with no network. The parts that genuinely need a model — whether it extracts the right total from a Turkish receipt — are measured as an eval over stored fallback documents, scored on exact match per field, and that eval is the node's own quality number rather than the pipeline's. Separating the two matters because they move for different reasons: the eval moves when the prompt or model changes, and the pipeline number moves when the input mix changes.

Containment is a property of the output, and the boundary is not self-enforcing#

Scoping the iteration to one node is not a claim that the node is safe. It is a claim that the node's failures are contained to the node's output, and containment is only worth something if the output is checked; a parse_unmatched returning to a validate that trusts it would be an agent with extra steps. The 5% that produced invalid fields is the number that proves the check is doing work. The belief to watch for is the one that stops at the word contained — that putting a loop inside a node has made the loop harmless — when what has actually been achieved is that its damage now has to pass through one typed value that something downstream inspects.

The second belief is that the boundary holds by itself, and it does not: a node's boundary is not a stable equilibrium by default. The failure mode is scope creep with a plausible justification at every step: the node would parse better if it could see the employee's previous receipts, and then it would be more useful if it could correct an earlier row, and then it could handle the validation itself since it already knows the format. Each of those is defensible in isolation and their sum is the whole-system agent that the measurement did not justify. Notice what stays constant while that happens. Every one of those diffs leaves the architecture diagram unchanged — it is still one node, still one bounded loop, still one graph — and the diagram is what a review looks at. What varies is what the node may touch, and that is carried by three things nobody draws: the tool list, the budgets and the output type. The defence is that all three are in code and in a test, so widening them is a reviewable diff rather than a prompt edit. Three questions asked of any such diff catch most of it: does this add a tool that writes anything, does this let the node see data from outside the single receipt it was given, and does this raise a budget without a measurement attached to the raise.

The write case has a standard answer that is worth knowing, because "the node needs to write something" is a legitimate requirement often enough. The node returns a proposed action rather than performing one, and a deterministic node downstream validates and executes it. A parse_unmatched that wants to correct a previously extracted row returns Correction(row_id, field, new_value, evidence); code checks that the row belongs to the same claim, that the field is in a permitted set, that the delta is within a bound, and then applies it. The model's influence is preserved, the enumeration is preserved, and the tool set stays read-only. This propose-and-verify shape is the general way to give an agentic node effect on the world without giving it the control flow, and it costs one extra type and one extra node.

And it does not resolve the case where the numbers are genuinely close. Here the comparison was easy because the alternatives were 20 hours of human review against $410 a month, which is two quantities that are not in the same neighbourhood. The harder version arrives when someone has built both a workflow and an agent for the same work and puts two success rates on a slide, three percentage points apart, and asks which one ships. Success rate is the wrong single number for that question, and what replaces it is not another average.

The move this chapter describes is available because a fallback counter reports a rate for one branch rather than a verdict on a graph. When the rate comes back high and the reasons do not cluster, the branch is where iteration goes, and it goes in wearing the same return type as the function it replaces, with a step cap, a token cap, a deadline and a read-only tool set that make its worst case a value the caller already knows how to route. Everything else keeps its tests, because nothing else changed.

Those numbers only exist because the loop was confined to a node with a measurable contract. A whole-system rewrite would have produced one number — end-to-end success — with no way to attribute it, and the 88.6% of receipts that were already fine would have been paying the 4.1-second median and the $0.038 for nothing.