Hierarchical / supervisor trees

Read these first

• Orchestrator–worker

What you will know

• Build a two-level supervisor tree

  Use a concrete task with delegated workers, explicit return contracts, and traceable handoffs so lost context and duplicated work are visible.

• Calculate the compounding error across the depth

  If each delegation succeeds with probability p, d sequential handoffs succeed at roughly p^d; correlated failures can make this estimate falsely optimistic.

• Measure a single-loop baseline against two- and three-level trees on the same workload

  Compare completion quality, token spend, wall-clock latency, tool calls, and recovery rate before attributing any gain to hierarchy.

• Decide the maximum depth you would ship

  Set a numeric cap and removal criteria from error budgets, latency SLOs, cost limits, and the observability needed to debug each boundary.

• Flatten a hierarchy that was deeper than it needed to be

  Remove redundant routing and pass required context directly, then verify that task success, failure isolation, and trace coverage have not regressed.

Chapters

1. 1

   A second level buys a boundary, not a specialist

   Design A fans six research workers out of one orchestrator; design B routes them through two sector supervisors. Those supervisors are the same model with a different system prompt and do no research at all, so what the level actually adds is two delegation briefs going down and two return contracts coming up. You will build the two-level tree and name each boundary before arguing about roles.

   Read chapter
2. 2

   Two workers research the same company because the brief let them

   Duplicated findings in a final brief are usually written into the delegation rather than into the workers. When two mid-level supervisors receive scopes that overlap, both can legitimately claim the same company, and deduplicating at the merge step hides that instead of fixing it. Here each delegation carries an explicit, non-overlapping scope, and the traces make lost context and duplicated work visible.

   Read chapter
3. 3

   A clean boundary and a successful run are different events

   Every figure the rest of this topic runs on — 95% per boundary, 96% per boundary — measures something nobody has yet defined. A sector supervisor that returns a well-formed summary faithfully derived from a misread brief scores as a clean boundary and ruins the run; a boundary that hiccups and is retried scores as a failure while the run still succeeds. Choose what you count at each hop — schema-valid return, brief satisfied, output usable downstream — and instrument it, and the two rates stop being the same number: end-to-end lands below the product when failures share an origin and above it when a boundary recovers, which is the reconciliation, and it is also where your own compounding arithmetic gets its inputs.

   Read chapter
4. 4

   Failures at a shared origin arrive clustered, not spread evenly

   If each boundary completes cleanly 95% of the time, three of them read as 86% end to end, which sounds like it fits inside an error budget. The multiplication assumes independence, and an underspecified brief at the top is inherited by every layer beneath it, so real failures land as whole ruined runs on the same afternoon. Compute the naive figure, then instrument for the shared symptoms that mark correlation.

   Read chapter
5. 5

   Back-port the fixes before comparing the tree with the loop

   Most hierarchy wins evaporate once you notice that the prompt and tool repairs discovered while building the tree were never applied to the flat baseline. Run single-loop, two-level and three-level arms over the same task set with the same model and tool budget, and report completion quality, token spend, p95 wall-clock, tool-call count and recovery rate after an injected failure.

   Read chapter
6. 6

   The depth you can attribute is shallower than the depth you can afford

   Product allows a p95 of 20 seconds and each boundary adds roughly 2.5 seconds at the mean but 6 at the tail, so a mean-based budget approves a tree that misses the SLO by design. The constraint that binds first is usually different again: correlation IDs propagating one level below the orchestrator mean nobody can name the boundary a run failed at. You set a numeric cap and the criteria that would raise it.

   Read chapter
7. 7

   A level pays when routing a branch has become a full task

   Having priced the boundaries, you can state the positive condition rather than a preference. Inserting a supervisor between an orchestrator and its workers earns its extra inference only when one branch holds enough workers that dispatching and reconciling them is itself a job the top supervisor is measurably failing at. Anything short of that is a prompt change the flat design can have for free, which is also the answer to what would have to appear in the results before you added the middle layer at all.

   Read chapter
8. 8

   Unchanged success and 31% cheaper is not yet a flattening win

   You delete the middle layer, pass constraints straight to the workers, and the regression suite reports identical success with a third of the tokens gone. That suite scores the runs that succeed, while the deleted layer earned its keep on runs that fail, by containing a bad worker result before it reached the top. Check failure isolation and trace coverage, and design the test that shows whether the layer ever caught anything.

   Read chapter