Under a hundred lines, with nothing hidden by an SDK

A language model takes text and produces text, and that is the whole of its interface. An agent loop is the piece of control flow that sits between such a model and the rest of a system: it sends the conversation so far to the model along with a list of tools the model is allowed to request, reads the reply, runs whichever tools were asked for, appends what they returned to the conversation, and sends the whole thing again. It goes round until the model answers without asking for a tool, or until a limit you set stops it. The problem it exists to solve is that a model produces text and nothing else. By itself it cannot read a row from a database, call an internal service or move money; the loop is what turns a request expressed in text into an action that actually happens, and then puts the outcome back where the model can read it before choosing what to do next.

You meet it whenever you use an assistant that edits files and reruns your tests, a support agent that looks up an order before answering, or a research tool that searches and then reads what it found, because underneath each of those is some version of these few lines. In production they are nearly always inside a framework or a provider's SDK — a software development kit, the vendor's client library for talking to the model — which is why one malformed request, in the failure this chapter closes on, sat nine frames away from the engineer who had to fix it. The four parts demonstrated earlier — a model that can request actions, a fixed toolset, a loop that feeds results back, and a condition that ends it — are the whole architecture, and the sketch that demonstrated them left two things as stubs: TOOLS and dispatch(). Nearly every decision that matters in practice lives inside those two stubs and in the exact shape of the list you send back. Writing the loop by hand is how you get those decisions in front of you, in code you can read in one sitting, before you adopt a framework that makes them for you.

Why the loop has to be a loop#

The four steps in that description are worth reading as links in a chain rather than as a list of features, because each one is checkable on its own and everything later in this chapter is a consequence of one of them. A model can only emit text, so a request for an action has to arrive as text in a shape your code recognises. Your code, and not the model, performs the action, which puts the outcome on your side of the boundary where the model has not seen it. The model's next decision depends on that outcome, so the outcome has to be written back into the conversation before the next request goes out. Writing it back makes the conversation longer than it was, so the next request is a different request — which is why this is a loop and not a function call with an extra step in the middle.

The third link is the one that gets skipped, and skipping it produces a loop that runs the tool and then asks the model the same question it asked before. A provider's endpoint answers each request from the bytes of that request alone. It is not holding your conversation open between calls, there is no session identifier in the request under which it could be held, and the model has no access to variables in your process. So a tool result that stays in a local variable is invisible to the thing making the decisions, and the only place it can be put where the next decision will be conditioned on it is the message list itself.

That gives the plain-words version of what the message list is, and it is worth having before any code is read. The list is not a log of the run kept alongside the run. It is the run's entire state, the whole of what the model knows, rebuilt and re-sent on every step. Every bug in this topic is a bug about what is in that list and in what order.

Ninety-one lines, and where each of them goes#

Here is a complete loop for an agent with two tools. It is deliberately the naive version: it has the ordering right and the error handling wrong, which is the state the next three chapters work from.

import json
from anthropic import Anthropic

client = Anthropic()
MODEL = "claude-opus-5"
MAX_STEPS = 10

SYSTEM = (
    "You are a support agent for an online store. Use the tools to look up "
    "real data; never state an order status you have not read from a tool."
)

TOOLS = [
    {
        "name": "search_orders",
        "description": (
            "Find orders belonging to one customer. Returns at most 20 orders, "
            "newest first. Returns an empty list if the customer has no matching orders."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string", "description": "e.g. cus_8123"},
                "status": {
                    "type": "string",
                    "enum": ["open", "shipped", "delivered", "refunded", "cancelled"],
                },
            },
            "required": ["customer_id"],
            "additionalProperties": False,
        },
    },
    {
        "name": "issue_refund",
        "description": "Refund an order in whole or in part. Amounts are in cents.",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount_cents": {"type": "integer", "minimum": 1},
            },
            "required": ["order_id", "amount_cents"],
            "additionalProperties": False,
        },
    },
]


def search_orders(customer_id: str, status: str | None = None) -> str:
    rows = db.orders.find(customer_id=customer_id, status=status, limit=20)
    return json.dumps([r.as_summary() for r in rows])


def issue_refund(order_id: str, amount_cents: int) -> str:
    receipt = payments.refund(order_id=order_id, amount_cents=amount_cents)
    return json.dumps({"refund_id": receipt.id, "amount_cents": receipt.amount})


DISPATCH = {"search_orders": search_orders, "issue_refund": issue_refund}


def text_of(response) -> str:
    return "".join(b.text for b in response.content if b.type == "text")


def run(user_input: str) -> dict:
    messages = [{"role": "user", "content": user_input}]

    for step in range(MAX_STEPS):
        response = client.messages.create(
            model=MODEL,
            max_tokens=2048,
            system=SYSTEM,
            tools=TOOLS,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": response.content})

        calls = [b for b in response.content if b.type == "tool_use"]
        if not calls:
            return {
                "stop": "end_turn",
                "output": text_of(response),
                "messages": messages,
                "steps": step + 1,
            }

        results = []
        for call in calls:
            output = DISPATCH[call.name](**call.input)
            results.append(
                {"type": "tool_result", "tool_use_id": call.id, "content": output}
            )
        messages.append({"role": "user", "content": results})

    return {"stop": "max_steps", "output": None, "messages": messages, "steps": MAX_STEPS}

Two-thirds of that listing is tool declarations, which is the first thing the exercise teaches: the schemas are the bulk of the artefact, and they are the part a framework's decorator syntax makes look small.

Counting those lines is worth doing rather than taking on trust, because the count is the argument and the obvious count gives the wrong answer. From import json to the final return the listing is ninety-six lines as printed, seventeen of them blank. Neither ninety-six nor the seventy-nine non-blank lines is ninety-one. The reconciliation is a convention rather than a fudge: Python's style guide puts two blank lines between top-level definitions, and there are exactly five places in this listing where that rule fires. Count each of those separations once rather than twice — which is what anyone reading down a screen does — and ninety-six becomes ninety-one.

The same convention splits the file cleanly in two, and the split is why the proportions can be quoted as fact. Everything above def run — the imports, the model and step constants, the system prompt, the two tool declarations, their implementations, the dispatch dictionary and the text helper — is sixty lines under that count, and all five of the double separations fall inside it. def run and its body are the remaining thirty-one, with no double separations in them at all. Sixty plus thirty-one is ninety-one exactly, with nothing left over, which is what makes "two-thirds" a property of the listing rather than a rounding chosen because it sounds good.

The fourteen lines quoted for run are a different count of the same function, and the gap between fourteen and thirty-one is worth naming rather than leaving as an apparent inconsistency. Thirty-one counts physical lines; fourteen counts statements. The def, the message list, the for, the create call, the assistant append, the calls comprehension, the if, the early return, the empty results list, the inner for, the dispatch, the result append, the results append and the final return come to fourteen. The residual is seventeen lines, and it decomposes exactly: thirteen continuation lines belonging to multi-line calls that have already been counted once, and four blank lines. So there is no third category hiding in there, and when this chapter calls the loop fourteen lines it is counting decisions rather than newlines.

Every one of those fourteen corresponds to a decision, and the correspondence is what the rest of the topic works through. messages.append on the assistant turn, before any dispatch happens, is the ordering rule that the staging failure below violated. The list comprehension that pulls out tool_use blocks is the stop condition, expressed as its absence. The inner loop that builds results is the tool protocol: one result object per call, carrying the id the model assigned. The for step in range(MAX_STEPS) is the bound, and the return after it is the admission that the bound can be reached.

Two-thirds of the file is the part that decides behaviour#

The proportions of that listing are the most useful thing about it. Sixty of the ninety-one lines are tool declarations, and they are the lines that determine what the agent does, far more than the fourteen lines of control flow. A decorator-based framework compresses those sixty lines into three-line function signatures with docstrings, which is a genuine ergonomic win and also the reason people underestimate how much design work is sitting there.

Read the search_orders declaration as the model reads it, which is as a sentence and a shape. The description says the tool returns at most 20 orders, newest first, and returns an empty list when there are none — three facts that shape the model's behaviour on the next step, because a model that does not know the result is truncated at 20 will treat 20 results as the complete set, and a model that does not know an empty list is possible will treat one as a failure. The enum on status is not decoration either: it is the difference between the model sending "in transit" and the model sending "shipped", and the enum values are the vocabulary you are teaching it. additionalProperties: false says that a field you did not declare is an error rather than something to ignore quietly.

Each of those three facts earns its place by a chain that ends in a different next request. A truncated list that does not announce itself is indistinguishable from a complete one, so the model has no reason to narrow its query, so it answers from a partial set and states the answer with the confidence of a complete one. An empty list with no explanation is compatible with a failed search, so the model has a reason to retry, and the retry is byte-identical because nothing in the result told it what a better question would look like. A missing enum leaves the model choosing vocabulary from the task rather than from your schema, and the words a customer uses are not the words in your database. The description is not documentation for a human reader who has wandered in. It is the only thing standing between the model's guess and your data model.

The costs are on the same page as the design. Each of those two schemas is roughly 180 tokens once serialised, and the schemas ride in every request of the run, so a six-tool agent pays about 1,100 tokens per step for its menu — around 11,000 tokens across a ten-step run, before the conversation itself. That is the arithmetic that makes people want to trim descriptions, and trimming them is usually the wrong economy, because the description is what prevents the extra round trip that costs several thousand tokens on its own.

That arithmetic is worth doing slowly, because the same quantity is used again later and it is easy to mistake for a one-off charge. One schema is the figure just quoted; six of them is a little over one thousand, which is where the per-step number comes from. The word doing the work is step. The schemas are not sent once and remembered, so the ten-step figure is the per-step cost multiplied by the number of steps rather than amortised across them, and a run that takes twice as long pays twice as much for a menu that has not changed. Hold that per-step schema cost, because the next section spends it a second time on a different question.

The last thing the declarations decide is the blast radius. issue_refund moves money, and nothing in the loop above distinguishes it from search_orders; both are entries in the same dictionary and both are called the same way. Whatever policy you have about which actions require a check, a confirmation, or an audit record has to live either in the tool implementation or in a wrapper around the dispatch table, because the loop does not have a concept of a dangerous tool and the model does not have a concept of your authorisation model.

What goes over the wire on step two#

The listing above hides one thing worth looking at directly, because it is the thing the model actually reads. On the second iteration of a run where the model asked for one order search, the body of POST /v1/messages looks like this:

{
  "model": "claude-opus-5",
  "max_tokens": 2048,
  "system": "You are a support agent for an online store. ...",
  "tools": [ { "name": "search_orders", "...": "..." },
             { "name": "issue_refund", "...": "..." } ],
  "messages": [
    { "role": "user", "content": "Did my refund for order 4471 go through?" },
    { "role": "assistant", "content": [
        { "type": "text", "text": "Let me look up your orders." },
        { "type": "tool_use", "id": "toolu_01WkP9Qm",
          "name": "search_orders",
          "input": { "customer_id": "cus_8123" } }
    ]},
    { "role": "user", "content": [
        { "type": "tool_result", "tool_use_id": "toolu_01WkP9Qm",
          "content": "[{\"id\": \"4471\", \"status\": \"refunded\", ...}]" }
    ]}
  ]
}

Nothing in that body refers to a previous request, and that absence is the property the rest of this chapter rests on. There is no conversation identifier, no continuation token, nothing that would let the provider look up what was said before. So the tools are declared again, the system prompt is sent again, and every message is sent again — including the assistant turn the model itself produced on the previous step. The cost growth, the ordering problem in the next chapter and the value of prompt caching are all consequences of that one property, which is why it is worth stating once here rather than three times later.

Three properties of that body are worth naming because they drive everything downstream. The whole conversation is resent on every step, so the request grows: with six tools declared at roughly 180 tokens of schema each, a run starts at about 1,400 input tokens before the user has said anything, and a six-step run's final request commonly lands between 6,000 and 9,000 input tokens. The tool result is a string inside a structured block rather than free prose, which is what lets the model tell "the tool returned an empty list" apart from "the assistant is speculating". And the assistant turn is echoed back verbatim, including the text block that preceded the tool call, because the model's next decision is conditioned on what it already said.

The opening figure in that paragraph is the per-step schema cost being spent a second time, and the difference between the two numbers is worth naming rather than leaving as slack. A six-tool menu is a little over one thousand tokens; the request before the user has typed anything is about four hundred tokens more than that. The residual is the system prompt and the request scaffolding — the field names, the model identifier, the token limit — and it is small enough to leave out of a cost model and large enough to matter when a long run is approaching a context limit. Naming it also stops the two figures reading as a contradiction, which is what an unexplained gap between two numbers about the same request always does.

What a framework was doing on your behalf#

Hand-rolling is not a claim that frameworks are wasteful. It is a way of finding out what they were doing, so that when you adopt one you know which of its behaviours you are depending on.

The rows in the second half of that table are the ones people are surprised to find they were relying on. A framework that silently returns "unknown tool: serch_orders" to the model instead of raising is making a policy decision — that malformed calls are the model's problem to recover from rather than an operator error — and it is usually the right decision, but it is a decision, and it changes what your traces look like and how your runs terminate.

The stumble that table exists to prevent is not a misreading of any one row. It is the belief that adopting a framework leaves the loop's behaviour where you left it, and the belief is tempting because the quantities you can see really do stay constant: the run still terminates, the request still validates, the tests you already had still pass. What varies underneath those constants is which of the eight rows is being decided for you, and how. A framework that answers a hallucinated tool name with a result block rather than an exception has changed how your runs end, how long a broken run takes, and what the trace of a failure contains — while every observable you were watching held steady. The check that catches it is not running the agent and seeing what happens. It is reading the eight rows one at a time and asking, for each, what your own code would have done, because a difference in behaviour that never shows up as a failure will never announce itself.

The bugs this exercise exists to surface#

The value of writing it yourself is concentrated in a handful of failures that are easy to introduce and hard to reason about afterwards, and the loop above is one careless edit away from each of them. Appending the tool result without the assistant turn that requested it produces a request the API rejects even though the tool worked perfectly. Appending them in the wrong order does the same. Running two tools concurrently and appending their results as two separate user messages produces a request that is accepted, works today, and quietly changes how the model behaves on later turns. Catching an exception around the dispatch step and continuing produces a request byte-identical to the previous one, and therefore the same tool call again, until the cap fires. A KeyError on a hallucinated tool name takes down the worker rather than the run.

None of those are exotic. All of them have the same shape: the loop's correctness lives in what you append and when, and the API's error message points at the message index rather than at the line of code that produced it. That indirection is what makes them expensive to debug in a system where the appends happen inside library code. It is also what makes them cheap to debug here, because there are exactly two append calls to inspect and both are on screen at once.

The indirection has a cause worth stating, because it is otherwise read as the provider being unhelpful. The API validates the message list it received and has no access to the code that assembled it — the two are on opposite sides of an HTTP request, and the request carries no stack. So the only coordinate the error can name is a position in the structure it was given, which is why the message in the scenario below names a slot in the message list rather than a function in your repository. The distance between that coordinate and the line of code responsible is exactly the number of layers between your append and the wire, which is why the same bug costs seconds in the listing above and an afternoon inside a framework.

There is a related habit worth acquiring while the code is this small: print the message list. A ten-line function that walks messages and prints each entry's role, its block types, and the first sixty characters of each block turns every one of the failures above into something visible in one screen of output. Most teams write that function eventually, usually during an incident; writing it on the first afternoon is cheaper. Thorsten Ball's How to Build an Agent builds the same object in roughly 300 lines of Go and is worth reading alongside this, because seeing a second person's version of the same fourteen lines makes it clear which parts are essential and which are taste.

Where the hand-built loop stops being the right answer#

This is a teaching object and a starting point, not a claim that you should ship it. What it does not give you is substantial: retries with backoff on 429 and 529 responses, which are the provider saying that you are rate limited and that it is overloaded, streaming so a user sees output before the run finishes, token accounting across steps, prompt caching so that the resent prefix costs a tenth as much on step six as it did on step one, concurrent dispatch of parallel tool calls, subagents, and a persistence layer so a run can survive a process restart. Every one of those is real work, and a mature client library or agent runtime has already done it.

Two kinds of thing are mixed together in that list, and separating them is what turns "adopt a framework" from a default into a decision. Retries on the rate-limit and overloaded responses, a timeout on every tool, and some bound on the length of a run are requirements: an agent without them fails in production regardless of anybody's opinion about frameworks. Streaming, prompt caching, concurrent dispatch, subagents and durable runs are design decisions, each with a workload that justifies it and a workload that does not — streaming matters when a person is watching and is dead weight in a batch job, and caching pays only when the resent prefix is large and stable. A hand-built loop missing a requirement is broken. A hand-built loop missing a design decision is merely narrow, and knowing which of the two you are holding is the difference between choosing a runtime and inheriting one.

The argument for writing the loop anyway is narrower than "frameworks are bad": it is that protocol, safety, and observability decisions are cheapest to make while they are still visible, and a framework's job is to make them invisible. Once you have written the fourteen lines, adopting a runtime is a matter of checking which of your decisions it reproduces. Before you have written them, adopting a runtime means inheriting decisions you cannot name. This is roughly the argument in Factor 8: Own your control flow, and it is why the exercise is worth an afternoon even on a team that will never ship a hand-rolled agent.

What the afternoon leaves you with is a proportion and a list. The proportion is that the control flow is the small part — sixty lines of declarations against fourteen statements of loop — and that the declarations are what decide whether the agent answers a question or issues a refund, because the model's behaviour on the next step is a function of what the description said and what the enum allowed. The list is the eight rows of the framework table, which is the set of decisions any runtime will make on your behalf whether or not you have an opinion about them. Neither is visible from inside a framework, and both are cheap to see once.

The listing above gets one thing right that the rest of this topic depends on, and it is the thing the staging failure got wrong. The assistant turn goes into the message list before the tools run, and the results go in after it, matched by id. That rule has more structure to it than a single append suggests, particularly once a single turn asks for three tools at once, and it is where the next chapter starts.