The agent loop implemented from scratch

Read these first

• What an agent is

What you will know

• Write the loop by hand in under a hundred lines

  The exercise should expose ordering bugs and make the provider and tool protocol visible rather than hiding it in an SDK.

• Handle tool dispatch, message accumulation, and stop conditions

  A stale assistant turn, mismatched tool-call ID, or duplicated result can corrupt the next model request even when the tool itself succeeds.

• Cap iterations and explain the number

  Choose a bound against latency and spend, and make exhaustion produce a deliberate terminal result instead of silently dropping work.

• Debug an agent that never terminates

  Compare traces for repeated calls, malformed arguments, transient failures, and model stop signals; each points to a different fix.

• Explain every message in a completed trajectory

  The trace should reveal what data the model saw at each turn and why control transferred, including when multiple tool calls are involved.

Chapters

1. 1

   Under a hundred lines, with nothing hidden by an SDK

   The whole loop fits in less code than most people expect: call the provider, look for tool calls, dispatch them, append what came back, go round again. Writing it by hand puts the provider's request shape and your own tool protocol in front of you instead of behind a framework, and it exposes the ordering bugs an SDK would have silently corrected on your behalf.

   Read chapter
2. 2

   The assistant turn, then one result block per call id

   An assistant turn carrying three tool calls obliges the next request to contain that turn unchanged followed by three results, each matched to its call id. Get it wrong and the request breaks although every tool worked — the classic version appends the result and drops the turn that asked for it. Flattening results into a single plain-text user message passes today's tests and forfeits everything later built on the structure.

   Read chapter
3. 3

   A tool that raises should return an error the model can read

   An exception inside a tool is not a crash of the run. Catch it, put the message back as the result for that call id, and let the model decide whether to fix its arguments, try a different tool, or give up. This chapter covers what belongs in that error text, what must never be put into it, and why swallowing the exception and appending nothing is the version that hangs.

   Read chapter
4. 4

   Validate the call before the dispatch table sees it

   Between the provider's response and your dispatch table sits a check nobody writes the first time: the tool name may not exist, the arguments may not parse as JSON at all, and arguments that do parse may still violate the schema you declared. Each of those is a rejection the model can act on, so it goes back as a result block against that call id rather than up the stack as an exception. This is a different failure from the previous chapter's — that tool ran and this one never should — and it is one of the four causes of a run that will not terminate.

   Read chapter
5. 5

   A turn with no tool calls is the loop's normal exit

   The terminal turn is the one the model returns without requesting a tool, and what the loop hands its caller is that turn's content together with the accumulated message list. The alternatives are worth pricing before you choose: a declared finish tool buys a structured terminal payload, a way to distinguish finished from gave up, and immunity to trailing commentary after the last tool call, at the cost of an extra tool in every request's schema, a second exit path for when the model never calls it, and a stop condition that injected content can invoke. Settle the exit here, because the following chapters read and diagnose runs against it.

   Read chapter
6. 6

   Reading a finished trajectory one message at a time

   Asked to describe a run, most engineers describe intent: first it decides, then it searches, then it answers. The useful account names each message, who produced it, what the model could see at that point, and why control moved where it did. Practise on a four-turn trajectory that includes a turn with two tool calls, until the retelling is a walk through the message list rather than a story.

   Read chapter
7. 7

   Choosing the cap, and the stops a step count cannot express

   Pick the number from the p95 step count of successful runs plus headroom, checked against your latency and spend budgets. Treat the 4% of tasks that hit it as a detector rather than raising the cap to 25 and moving on, and make exhaustion return a deliberate terminal result carrying the partial work. A step cap cannot see a hung tool, so a wall-clock deadline belongs beside the normal exit condition you designed two chapters ago.

   Read chapter
8. 8

   Forty identical calls, and the four fixes it might need

   A trace showing the same search with byte-identical arguments forty times over does not have one cause. Repeated identical calls point at an empty result the model cannot see and at missing duplicate detection; malformed arguments point at a validation gap; transient failures point at the retry path; a missing stop signal points at the exit condition. Each is a different fix, and the trace is what distinguishes them.

   Read chapter