Forty identical calls, and the four fixes it might need

The commonest of the four is a tool that succeeded and returned nothing useful, and the loop it produces is worth understanding before the other three, because the bug is not where the symptom is. A signature, in this chapter, is a pattern visible in a stored run that narrows the cause — a shape the message list takes rather than a value any counter holds. The material to read it in is the trajectory: the accumulated message list for one run, in the order the next request would present it. Four signatures produce the same stop="max_steps" result and the same line on a chart, so a team that reasons from the chart will reach for the wrong remedy roughly three times in four.

This trajectory comes from the duplicate-call bucket of the previous chapter, from before the cap was tightened, when the loop was allowed 40 steps. Messages 1 through 78 are the same two blocks, alternating:

[ 1] assistant  tool_use  toolu_01Ab  search_orders {"customer_id":"cus_8123","status":"refunded"}
[ 2] user       tool_result toolu_01Ab  "[]"
[ 3] assistant  tool_use  toolu_01Cd  search_orders {"customer_id":"cus_8123","status":"refunded"}
[ 4] user       tool_result toolu_01Cd  "[]"
[ 5] assistant  tool_use  toolu_01Ef  search_orders {"customer_id":"cus_8123","status":"refunded"}
...
[79] assistant  tool_use  toolu_02Zx  search_orders {"customer_id":"cus_8123","status":"refunded"}
[80] user       tool_result toolu_02Zx  "[]"
                → RunResult(stop="max_steps", output=None, steps=40)

Forty model calls, forty tool calls, 96 seconds, no exceptions, every tool returning in about 92 milliseconds with a successful status. The arguments are not merely similar; they are byte-identical across all forty, which is the detail that narrows the diagnosis. A model varying its arguments is searching. A model repeating them exactly is stuck.

Two of the counts in that summary are the same fact seen twice, and lining them up is a habit worth acquiring for long traces. Each model call produces one assistant turn, and the loop answers each turn with one user message carrying its result, so forty model calls put eighty messages in the trajectory at indices one through eighty. That is why the listing can show the first five, skip to the last two, and describe everything between them as the same pair alternating. The step count and the message count are not independent observations of the run; either determines the other, and a trace where they disagree contains a message that no model call accounts for.

The timings decompose as cleanly, and the decomposition matters later. Forty tool calls at about 92 milliseconds each is under four seconds of tool time inside a run that took 96 seconds, which leaves the remaining ninety-odd seconds as model latency — a little over two seconds for each of the forty calls. That is in the same range as the per-step figure the previous chapter used to price a cap, and slightly below it for a reason visible in the listing: these requests are tiny, because an empty result adds almost nothing to the history being resent. Naming that residual now is worth the sentence, because it puts a ceiling on what any change to the tool alone could ever save.

That trace has one obvious cause and three plausible ones, and reaching for the wrong remedy is easy, because "the agent loops forever" is a symptom that four unrelated bugs share. The trace is what separates them.

Four signatures#

Read that table down its columns rather than across its rows and it says something more useful than four descriptions do. The first column is what you can see without opening anything. The second is a claim about cause. The third is where the change goes. The fourth is the observation that licenses moving from the first column to the second. What the arrangement exposes is that the first column does not separate the four on its own — all of them are a repeated call, or the conspicuous absence of one, inside a run that hit its cap — while every entry in the fourth column is a property of the results. That asymmetry is the reading rule for the whole chapter: the calls tell you the model is stuck, and only the results tell you why.

The confirming test in the right-hand column is the part that gets skipped, and skipping it is how a team ends up adding retry logic to a tool that never failed. All four signatures produce the same dashboard entry — a run with stop="max_steps" — so the dashboard cannot tell you which one you have. Thirty seconds with the stored messages list can, and every one of the four tests is a property of the trajectory: are the argument objects equal, are the results errors, are the error texts the same, is there a terminal-looking turn the loop rejected.

The empty result the model cannot see#

The first row is the most common and the one worth understanding in detail, because the bug is not really in the loop. search_orders executed correctly forty times. There were no orders with status refunded for that customer, and the function did the right thing and returned an empty list, which the loop serialised as the two-character string [].

Consider what that string says at the moment the model reads it. It does not say that the query was well-formed. It does not say that the customer exists, or that they have orders under other statuses, or that running the same query again will produce the same answer. It is compatible with "the search failed", "the search has not finished", and "you asked the wrong question", and the reading a model most often takes is the last one — so it tries again, and having no information about what a better question would look like, it asks the same one. The tool is reporting an outcome; the model needs a fact.

The step from that reading to a forty-step run is worth walking, because what it produces is a fixed point rather than a mistake. The model's next action is a function of the message list and of nothing else. The two characters [] are compatible with the three readings above, and nothing in them ranks those readings, so the model has no basis for preferring one. The cheapest resolution of an ambiguous result is to ask the question again. Asking it again produces the identical string, so the message list gains a turn and gains no information. The input to the next decision is therefore materially the input to the last one, and the same decision comes out. That is the same argument the error-handling chapter makes about a loop that snapshots and restores its message list around a failing tool (chapter 3, where a byte-identical request produces a byte-identical decision, which calls the same tool again). The property being reused is that a decision function given an unchanged input returns an unchanged output, and it makes no difference whether the input was unchanged because somebody restored it or because the new content said nothing.

The fix is in the tool, and it is a matter of a few lines:

def search_orders(customer_id: str, status: str | None = None) -> str:
    rows = db.orders.find(customer_id=customer_id, status=status, limit=20)
    if rows:
        return json.dumps([r.as_summary() for r in rows])

    others = db.orders.find(customer_id=customer_id, limit=5)
    if not others:
        return json.dumps({"matched": 0, "note": f"No orders exist for {customer_id}."})
    seen = sorted({r.status for r in others})
    return json.dumps({
        "matched": 0,
        "query": {"customer_id": customer_id, "status": status},
        "note": (
            f"{customer_id} has {len(others)} orders, with statuses {seen}. "
            f"None have status {status!r}. Repeating this query returns the same result."
        ),
    })

Run against the same conversation, that version terminates in two steps rather than forty. The model reads that the customer has three orders, none refunded, calls search_orders once more without the status filter, and answers the question. The sentence about repetition is doing real work: it is the only part of the context that says the search space has been exhausted rather than merely unproductive.

It is worth being precise about what changed, because it was not the model and it was not the loop. The database returned the same rows both times. What the second version added is context the tool had available and was throwing away: the fact that the customer exists, the fact that other orders exist under other statuses, and the fact that the query is deterministic. Every tool has some version of that surplus — the thing it knows while computing an answer that does not survive into the answer — and the empty case is where discarding it is most expensive, because an empty result has no other content to compensate.

Two of the choices inside that replacement are decisions rather than requirements, and separating them matters when the pattern is copied to a second tool. Sampling five other orders is a decision: the requirement is only that the model learn the customer exists and that other statuses do, which any bounded sample satisfies, while an unbounded one reintroduces the cost the error-text chapter warns about (chapter 3, where a rejection quoting a large payload back at the model has turned a diagnostic into a payload). Stating that repeating the query returns the same result is also a decision, and it is the one carrying termination — without that sentence the reply is richer, more accurate, and still silent on whether asking again could help.

The same principle generalises past empty lists. A tool that returns null for a missing optional field, {} for a record with no attachments, or "" for an empty document should say what it looked at and found nothing in, because the alternative is a two-character string that the model has to interpret.

The other three, in traces#

The remaining rows of the table are easier to recognise once you have seen what they look like next to the first one, because the differences are small on screen and large in consequence.

A validation loop reads like an argument:

[ 3] assistant  tool_use  search_orders {"customer_id":"cus_8123","status":"in transit"}
[ 4] user       tool_result is_error  "search_orders rejected: status is not one of
                            ['open','shipped','delivered','refunded','cancelled']."
[ 5] assistant  tool_use  search_orders {"customer_id":"cus_8123","status":"transit"}
[ 6] user       tool_result is_error  "search_orders rejected: status is not one of ..."
[ 7] assistant  tool_use  search_orders {"customer_id":"cus_8123","status":"shipping"}

The arguments change every time, the results are all is_error, and the error text originates from the gate rather than from a tool. The model is trying to express a concept — a package that is on its way — that your enum does not have a word for, and it will keep guessing synonyms. The fix is in the declaration: either the enum needs the value, or the description needs to say which of the five existing values covers in-transit shipments. Raising the cap here would buy nothing, because the twelfth guess fails the same way as the third.

That last sentence is the fixed-point argument again, arriving from a different direction, and it is worth noticing that it holds for a different reason. In the empty-result case the input to each decision was unchanged. Here the input changes every step — a new rejection, naming a new rejected value — and the run still does not converge, because the thing being searched for is not in the space being searched. Two conditions produce a non-terminating loop, then: an input that never changes, and an input that changes without ever getting closer. Only the first is fixed by giving the tool more to say.

A transient-failure loop reads differently again:

[ 5] assistant  tool_use  get_shipment {"order_id":"4471"}
[ 6] user       tool_result is_error  "get_shipment failed: upstream returned 503 (attempt took 2.1s)."
[ 7] assistant  tool_use  get_shipment {"order_id":"4471"}
[ 8] user       tool_result is_error  "get_shipment failed: connection reset (attempt took 5.0s)."

Identical arguments, like the first signature, but the results are errors and the errors are not the same error, and the step latencies are seconds rather than milliseconds. The model is behaving reasonably — retrying a failure that says nothing about being permanent is the correct reading — and the bug is that retrying is happening at the wrong layer. A retry with exponential backoff inside get_shipment, where each attempt waits roughly twice as long as the one before it so that a struggling upstream is given room to recover, costs one model call for three attempts; a retry driven by the model costs three model calls, three full-history requests, and three turns of context growth. After the tool's own attempts are exhausted, the result should say so, in the "do not retry this call" language from the error-handling chapter.

The arithmetic of that layer choice is the reason it is worth a code change rather than a note in a runbook. Three attempts inside the tool cost three upstream requests and one model call. Three attempts driven by the model cost the same three upstream requests, three model calls, and three resends of the entire conversation — and the resend is the expensive part, since the whole history rides in every request. The upstream sees identical load under both designs. Only your provider bill and your wall clock differ, which is what makes this a pure loss rather than a trade-off.

The fourth signature is the odd one out because it has no repetition to look at:

[ 9] assistant  text  "I've refunded the order. Let me confirm the details once more."
[10] assistant  tool_use  search_orders {"customer_id":"cus_8123"}
...
[15] assistant  text  "Everything looks correct. Let me do a final check."

The model has finished the task and keeps finding one more thing to verify. Nothing is failing, every result is useful, and the run still exhausts its cap. That is an exit-condition problem, and it is usually solved in the system prompt — an instruction to stop once the user's question has been answered rather than to re-verify — or by giving the model a declared way to say it is done, which is the trade-off priced two chapters ago.

Detection that terminates#

Fixing the tool fixes this tool. The loop still has no defence against the next tool whose empty case nobody thought about, and adding one is cheap.

def call_key(call) -> str:
    payload = json.dumps(call.input, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(f"{call.name}:{payload}".encode()).hexdigest()


seen: Counter[str] = Counter()

key = call_key(call)
seen[key] += 1
if seen[key] == 2:
    result["content"] += (
        "\n\nNote: this is the second identical call to this tool with these "
        "exact arguments. The result has not changed. Try different arguments "
        "or a different tool."
    )
elif seen[key] >= 3:
    return RunResult(stop="repeated_call", output=None,
                     partial=last_assistant_text(messages),
                     messages=messages, steps=step + 1, usage=usage)

Two thresholds rather than one, because a second identical call is sometimes legitimate — a model re-reading a file after asking another tool to write it, or polling a job status — and a note costs 30 tokens and often recovers the run. A third is not legitimate under any reading, and terminating there converts a 96-second, forty-step, $0.31 failure into a three-step failure that arrives in four seconds carrying a stop value naming its own cause. That last property is the point: stop="repeated_call" is a metric you can chart, alert on, and bucket, whereas stop="max_steps" is four bugs sharing a label.

The price of that failure is worth taking apart, because it explains why nothing caught the run earlier. At the list price the previous chapter used for input tokens, thirty-one cents buys something on the order of sixty thousand tokens, and spread across forty requests that averages a little over fifteen hundred each — which is roughly what the first request cost. The history barely grew, because each step appended one bare tool_use block and a two-character result. The previous chapter's cost model, where a request starts near fourteen hundred tokens and gains seven hundred a step, would put a forty-step run in the low single dollars, so the two figures are not comparable and no derivation should be built across them. What the difference licenses is the more useful statement: this spin is cheap per step for exactly the reason it is invisible. An empty result adds almost nothing to the prefix, so the cost curve stays flat, the context never approaches its ceiling, and neither of the two quantities an operator watches ever moves.

This is not a cache#

The natural-looking alternative is to memoise: hash the call, return the stored result for a repeat, skip the tool. It removes 92 milliseconds and one database query per repeat, and it makes the failure worse. A cached spin is a faster spin — forty steps still happen, still cost forty model calls, still consume the whole cap — and by removing the tool latency it also removes the wall-clock deadline's chance of firing first. Caching optimises the cost of the symptom while leaving the loop intact.

The reason that reads as a good idea is worth naming, because it is not carelessness. A call with an identical key returning an identical value is the textbook case for memoisation, and every instinct a programmer has says the recomputation is the waste. The quantity a cache holds down is the cost of one call, and it genuinely holds it down: the 92 milliseconds and the database query really do go away, and the arithmetic on that saving is correct. What varies underneath the constant is whether the loop terminates, and a cache does not touch it. It moves it the wrong way, because those 92 milliseconds a step were the only thing giving a wall-clock deadline (chapter 7, where a budget in elapsed real seconds runs beside the step cap) any chance of firing before the cap did. The two mechanisms share no machinery and land on the same trace from opposite ends: the cache leaves forty steps and removes the latency, the detector leaves three steps and keeps it. Memoisation answers "how expensive is this call". The question here is "why is there a next call", and no answer to the first is an answer to the second.

The detector has the opposite goal. It is not trying to make repeated calls cheap; it is trying to make them end, which is why its output is a terminal RunResult rather than a value. Both can coexist — cache for latency, detect for termination — but they are answering different questions, and a team that adds the cache and calls the problem solved will find the same trace in the logs a week later with the tool timings removed.

The keys, and the tools that have to be exempt#

Sorting keys before hashing matters more than it looks, since {"a":1,"b":2} and {"b":2,"a":1} are the same call and the model does not emit fields in a stable order. Tools with genuinely time-varying results — a clock, a queue depth, a polling endpoint — need exempting by name, and it is better to keep that list explicit and short than to weaken the threshold for everything.

Both halves of that are decisions with a cost on the other side, and the costs are worth being able to state. Exempting by name means a new time-varying tool is unprotected until somebody remembers to add it, and the failure is silent: the detector simply never fires on that tool, and nothing anywhere reports an omission. Weakening the threshold instead would cover new tools automatically and would stop covering the old ones properly, because a run permitted four identical calls before termination spends four steps on every stuck tool rather than three, on every tool, forever. The explicit list wins that trade because its failure mode is a missing entry, which a reviewer can see in a diff, rather than a number that is slightly wrong everywhere and looks fine.

The detector also does not catch the second and fourth signatures, and it should not be expected to. Arguments that differ every step never produce a repeated key, and a model that keeps finding one more useful thing to verify is not repeating anything. Those two need the declaration fixed and the exit condition fixed respectively, which is the point of having four signatures rather than one defence.

What you have now, and what to do with it#

The loop from the first chapter has picked up six things since: the assistant turn and one result block per call id, error results a model can read, three gates before dispatch, an exit condition that reads stop_reason rather than an absence, a cap derived from a distribution alongside a wall-clock deadline, and a duplicate detector that terminates. That is on the order of a hundred and fifty lines, and it is the complete set of protocol, safety and observability decisions that an agent runtime would otherwise have made silently.

That list is worth reading as six answers to six questions, because those questions are what the topic has actually been working through: what must be appended and in what order, what a failed tool should tell the model, what happens to a call that should never have reached a tool, how the loop knows it is finished, how long it is allowed to try, and how it notices it is going nowhere. One chapter contributed nothing to the list, and the omission is informative rather than a gap — reading a finished trajectory (chapter 6, where a run is walked message by message) adds a discipline rather than a component. It is also the one the other six depend on, since every fix above was chosen by looking at a message list and every one of the four signatures is confirmed there.

The useful next move is to run it against real tasks and read what comes back, because everything in this topic is measured on trajectories: the step distribution that sets the cap, the rejection rate that reveals a missing tool, the bucketing that turns 4% of cap-hits into three separate bug reports. The loop is now the instrument as much as the mechanism. And when you do adopt a framework — most teams should, for the retries, streaming, caching and concurrency that a hand-built loop does not give you — this list is the checklist you evaluate it against: which of these six does it do, which does it do differently, and which has it left for you.