The run's state lives in the file, not in flight

At 09:41 a run of the contract-ingestion pipeline stopped producing output. It had forty-two PDFs to work through, three stages — parse, clause-extract, report — and one agent per stage. The parse agent had finished at 09:12. Nothing since. The on-call engineer has a run identifier, run_8f21, and a question that sounds simple: which stage is this sitting in, and did anything survive?

What she can answer depends entirely on a design decision made eighteen months earlier, when somebody chose how the three agents would coordinate. Topic 11.11 covered the shape where an agent finishes its turn and hands control to a named successor, carrying a structured payload with it. Call that design A: the parse agent finishes, emits a payload containing the forty-two parsed documents and a pointer to where it wrote them, invokes the clause-extract agent with that payload, and exits. There is a second design, and it is the subject of this topic. Call it design B: all three agents read and write one shared JSON document that holds the task list, who owns each item, and where each stage wrote its output. No agent invokes another. Each one reads the document, decides whether there is work in it that it can do, does that work, and writes back.

The second design has a name. A blackboard architecture, also called shared-artifact coordination, is one in which the participants never address each other: each reads a common structure, decides for itself whether that structure contains work it can do, writes what it produced back into it, and reads again. The name is the metaphor, a room of specialists around one board, each watching for an entry it can extend. It exists because a chain of handoffs keeps the run's state nowhere in particular — the state is whatever the currently running process happens to be holding — so finding out what a run has done means knowing which process to ask and hoping it is still there to answer. A board puts that state on disk where anything can open it, including an agent that was not running when the work was done. The pattern appears in ordinary tooling more often than the name does: a coding assistant with a plan file and a checklist its sub-agents tick off, a jobs table that a pool of workers polls for unclaimed rows, a review document several people edit in place. It also brings the questions any shared mutable store brings, about who may write which field, who owns an item while they work on it, and what happens when two writers arrive at once.

The interesting difference between the two designs is not throughput and not code volume. It is where the run's state physically sits at the moment a process dies.

In flight, or on disk#

In design A, the payload exists in three places over its lifetime: in the parse agent's memory as it is constructed, on the wire during the invocation, and in the clause-extract agent's memory while that agent works. If the clause-extract agent's container is OOM-killed at minute three of a four-minute turn — killed by the kernel for exceeding its memory limit, with no chance to save anything — every copy is gone. The parse agent has already exited and will not send it again. Whatever the clause agent had accumulated in that three minutes — thirty of the forty-two documents processed, say — is also gone, because it was going to be included in the payload it never got to emit.

The run does not fail loudly, because there is nobody left to fail. The parse agent exited normally, the clause agent's process is gone, and the report agent was never invoked and does not know it exists. What the on-call engineer sees at 09:41 is silence, and the only way to reconstruct where the run got to is to find the clause agent's logs, if it logged, and infer from them.

In design B, the run's state is in a file. The clause agent's container dying takes with it only the work that agent had not yet written back. Everything recorded up to that point is on disk, in a document anybody can open, including a replacement agent. The engineer at 09:41 opens run_8f21.json and reads it.

{
  "run_id": "run_8f21",
  "version": 47,
  "updated_at": "2026-03-11T09:14:22Z",
  "tasks": [
    {
      "id": "parse",
      "status": "done",
      "owner": null,
      "owned_since": null,
      "inputs": {"source": "s3://intake/run_8f21/raw/"},
      "output": {"uri": "s3://intake/run_8f21/parsed/", "count": 42},
      "completed_at": "2026-03-11T09:12:03Z"
    },
    {
      "id": "clause-extract",
      "status": "in_progress",
      "owner": "agent-clause-7c1",
      "owned_since": "2026-03-11T09:12:44Z",
      "inputs": {"from_task": "parse"},
      "output": {"uri": "s3://intake/run_8f21/clauses/", "count": 30},
      "completed_at": null
    },
    {
      "id": "report",
      "status": "blocked",
      "owner": null,
      "owned_since": null,
      "inputs": {"from_task": "clause-extract"},
      "output": null,
      "completed_at": null
    }
  ]
}

The document says the clause task is held by agent-clause-7c1, has been held since 09:12:44, and has written thirty of forty-two outputs. It is now 09:41, so that agent has held the task for twenty-eight minutes against a stage that normally takes four. That is enough to act on. A replacement can claim the task, read the thirty existing outputs, and continue from document thirty-one, because the pointer to the partial output is in the same document as the ownership record.

This is a familiar move rather than a new one. A web application that keeps session state in process memory loses it when the process restarts, and one that keeps session state in a database survives the restart, at the cost of every request now paying a round trip and every write needing a concurrency story. Design B is that trade applied to a multi-agent run: durability and third-party readability bought with a shared mutable store and everything a shared mutable store implies. The architecture has an older pedigree than agents — blackboard systems date to the Hearsay-II speech understanding work in the 1970s, where independent knowledge sources with no channel to each other contributed hypotheses to a common structure, and a recent line of work has revived the pattern for LLM agents specifically, including Exploring Advanced LLM Multi-Agent Systems Based on Blackboard Architecture, which is the cleanest statement of the version where agents self-select work off the board with no messaging at all.

A schema another agent can act on without guessing#

The document above is doing more work than it looks like. Each of its fields exists because some other agent, or some human, needs to reach a decision from the file alone.

owner and owned_since together say who holds an item and for how long, which is what turns "this looks stuck" into a decision with a threshold behind it. inputs names what the task consumes, so a replacement agent knows where to look without reading the code that produced the task. output is a pointer rather than the content, so the document stays a few kilobytes instead of growing to hold forty-two parsed contracts; run_8f21.json at version 47 is 4.1 KB, which matters because every agent reads the whole thing on every poll. status and completed_at say what phase the item is in and when it last moved.

The one field that is easy to leave out and expensive to add later is version. It is an integer that increments on every accepted write, and it exists so that a writer can say "apply this only if the document is still at the version I read". Chapter 3 is about what happens when it is missing. For now, note that it is present, and that its value — 47 after a run that has completed one of three stages — is already telling you that agents write this document far more often than they complete tasks.

A useful discipline when designing this schema is to write down the questions the file must answer, and keep only fields that answer one:

Fields that answer none of these are decoration, and decoration in a shared document is worse than in a private one, because every agent that reads it has to decide whether the field means anything.

The schema also has to be enough for an agent to select its own work, since in design B nobody tells an agent to run. The clause agent's loop is short: read the document, look for a task whose id matches one of the stages it can perform, whose status is pending, and whose inputs.from_task refers to a task that is done. If it finds one, it claims it by writing itself into owner. If it does not, it sleeps and reads again. That is the whole control mechanism, and it means the readiness of a task is computed from the document rather than asserted by whoever finished before it, which is why the report task above sits at blocked with no owner instead of being pushed anywhere by the clause agent.

def find_claimable(doc, capabilities):
    done = {t["id"] for t in doc["tasks"] if t["status"] == "done"}
    for task in doc["tasks"]:
        if task["id"] not in capabilities:
            continue
        if task["status"] != "pending" or task["owner"] is not None:
            continue
        dep = task["inputs"].get("from_task")
        if dep is None or dep in done:
            return task
    return None

The thing the board does not keep#

There is a claim about design B that sounds obviously true and is close to backwards, and it comes up almost every time somebody argues for this architecture: that the shared file gives you a full audit trail of everything that happened, while the handoff payload is transient.

A blackboard holds current state. That is precisely the property that makes it readable in one screen — the engineer at 09:41 does not have to reconstruct anything, because the document already says what is true now. The cost of that property is that each write overwrites the previous value. Version 47 of run_8f21.json does not tell you what version 46 said. If the clause agent claimed the task, released it, and reclaimed it, the file shows one owned_since and no evidence of the churn. If a task went in_progress → failed → in_progress, the failure is not in the document unless somebody deliberately appended it.

Design A, meanwhile, is a sequence of discrete messages, and a discrete message is a natural logging unit. If each invocation payload was written to a log or an event store on its way past — which is the default in most frameworks, because the payload is already serialised at that moment — then design A has an ordered record of every handoff, with contents, and design B has none. The transient thing is more loggable than the durable thing, because it was already a message.

You can have history in a blackboard, and most production ones do, by appending events alongside the current state rather than relying on the state to carry them. That is a deliberate addition with a cost, and chapter 5 works through what an append-only event log buys and what it charges. The point to hold onto here is that the audit-trail argument is not an argument for design B. Reaching for it usually means somebody has not built one, because ordering and history are exactly where the difficulty in these systems lives.

Which one for a stage that takes four minutes and crashes#

Set the two designs against the failures they each make cheap.

Now the concrete question. The clause-extract stage takes about four minutes per run and crashes roughly two runs in a hundred, which at forty runs a day is close to one lost run per day. In design A each of those is a total loss requiring a full re-run, at four minutes of stage-two compute plus the parse stage again if the parsed output was carried in the payload rather than written out. In design B each is a resumption from document thirty-one, costing about a minute. The failure mode that dominates this pipeline is the one design B makes cheap, so design B is the right choice here — with the qualification that you are choosing to give up cheap ordering questions and will need to buy some of that back deliberately.

None of this yet says anything about correctness of the file's contents. The schema in this chapter is one an agent can act on without guessing, which is a claim about structure. Whether an agent should believe what the fields say is a separate question, and the next chapter starts where that goes wrong: a done flag that four agents wrote, that no reader can verify, and that turns out to mean something different to each of them.