Contradicting workers need a merge rule, not a bigger window

Scout is an internal competitive-research assistant at a company that sells analytics software to enterprises, and it is built on the orchestrator–worker arrangement. One model call, the orchestrator, reads a goal and emits smaller pieces of research called subtasks. Each subtask goes to a separate model call, a worker, which runs with its own tools in its own fresh context and returns findings rather than a finished answer. A synthesis step then folds every worker's findings into the single answer a person reads. Chapter 4 gave that parent a way to know which of its subtasks came back and which did not, so Scout now knows that all five workers returned.

It still has the problem chapter 1 opened on. Two of those workers assert different things about when a competitor called Vantage moved its enterprise pricing floor, and one of them adds a carve-out the other does not mention. Knowing the set is complete tells the parent nothing about which claim to write down. That choice has to be made by something, and the only question is what. The choice is between making it in code, where you can name the rule and change it, and leaving it to whatever the synthesis model does when it is handed two contradictory paragraphs in one prompt. A merge rule, throughout this chapter, is the policy that decides which of two incompatible findings the answer asserts.

Prose cannot collide, and claims can#

The first requirement for making the decision in code is that conflicts be detectable at all. That sounds like a property of the workers and it is really a property of the shape they return in. Two paragraphs are always compatible, in the narrow sense that a model can write a third paragraph containing both. The sentence "Vantage raised its floor in March" and the sentence "Vantage raised its floor in January, for new business only" can be joined by an "and" without producing anything a parser would object to. Nothing in the representation forces the two into contact, so nothing in the representation can report that they failed to agree.

Requiring each worker to emit atomic claims changes that, for the subset of findings that are factual. A claim carries a subject, a predicate and a value, which is enough structure to make two findings land in the same place when they are about the same thing. Two claims that share a subject and a predicate are answers to one question, and if their values differ the disagreement is now a comparison rather than a judgement. This turns a semantic problem into a grouping problem, which is the whole of the trick.

[
  {"claim_id": "c_12", "subtask_id": "st_1", "question_id": "q1",
   "subject": "vantage:enterprise", "predicate": "list_floor_usd_yr",
   "value": "60000", "as_of": "2025-03-01",
   "evidence": {"kind": "primary_quote", "source": "web.archive.org/vantage.com/pricing",
                "span": "Enterprise — from $60,000/yr", "retrieved_at": "2025-09-02"}},
  {"claim_id": "c_31", "subtask_id": "st_2", "question_id": "q3",
   "subject": "vantage:enterprise", "predicate": "list_floor_usd_yr",
   "value": "60000", "as_of": "2025-01-22",
   "evidence": {"kind": "reported_speech", "source": "crm:opp_118204",
                "span": "their AE confirmed 60k floor, new business only",
                "retrieved_at": "2025-09-02"}}
]

Work the grouping by hand on those two records, because the result is more interesting than "they disagree". Both carry the subject vantage:enterprise and the predicate list_floor_usd_yr, so they group together. Compare the fields inside the group one at a time. The value field is 60000 in both, so the two workers agree on what the floor became. The as_of field is 2025-03-01 in one and 2025-01-22 in the other, so they disagree on when it became that. What the paragraphs presented as a general contradiction is a conflict on exactly one field, and the field is the date.

That precision is what a schema buys, and the same pass exposes something the paragraphs hid. Claim c_31 carries a scope qualifier — the floor applied to new business only — and there is no field for it. It is not in value, which holds the amount, and it is not in as_of, which holds the date. A qualifier with nowhere to live is a schema gap rather than a conflict, and it is a different kind of defect from a disagreement: no rule for choosing between claims will ever recover it, because the machinery cannot see it. Gaps of this kind only surface at the moment somebody tries to represent claims rather than sentences, which is one of the better arguments for doing so early.

Emitting claims in this shape is a requirement of detecting conflicts at all. Everything after this point is a decision somebody makes and could unmake.

Four rules, and the conditions each one is right under#

Read that table down the last two columns rather than across the first. Each rule's failure mode is the exact inverse of the condition that makes it right, which is what tells you the four are not four levels of sophistication. Stronger evidence is right when one source is closer to the fact, and it fails when directness and accuracy come apart. Recency is right when the subject changes over time, and it fails when a re-crawl date is mistaken for a change date. The gap between the two columns is never large, and a rule sitting in the wrong workload lands in its failure mode immediately rather than gradually.

None of these is the default. The default, if nothing is written, is that the synthesis model resolves the conflict by whichever claim reads more fluently and appears earlier in the concatenation, which is the behaviour chapter 1 opened on. Choosing a rule is not adding sophistication to a system that was neutral; it is replacing an existing, unwritten, unreviewable rule with one you can point at.

EVIDENCE_RANK = {"primary_quote": 3, "reported_speech": 2, "inference": 1}

def resolve(group: list[Claim], policy: MergePolicy) -> Resolution:
    if len({c.value for c in group}) == 1 and len({c.as_of for c in group}) == 1:
        return Resolution(kind="agreed", claims=group)

    if policy.rule == "stronger_evidence":
        ranked = sorted(group, key=lambda c: EVIDENCE_RANK[c.evidence.kind])
        if EVIDENCE_RANK[ranked[-1].evidence.kind] > EVIDENCE_RANK[ranked[-2].evidence.kind]:
            return Resolution(kind="preferred", claims=[ranked[-1]],
                              dropped=ranked[:-1], reason="evidence_kind")
        return Resolution(kind="unresolved", claims=group)

    if policy.rule == "recency":
        newest = max(group, key=lambda c: c.as_of or c.evidence.retrieved_at)
        return Resolution(kind="preferred", claims=[newest],
                          dropped=[c for c in group if c is not newest],
                          reason="as_of")

    if policy.rule == "recheck" and policy.recheck_budget > 0:
        return dispatch_recheck(group)     # returns "preferred" or "unresolved"

    return Resolution(kind="conflicted", claims=group)

The first if is doing more work than it looks. A group whose claims all share one value and one as_of is not a conflict at all, and returning agreed there keeps the rest of the function off the fast path. It also makes the stronger-evidence branch safe: ranked[-2] would fail on a group of one, and a group of one always has a single value and a single as_of, so it never reaches that line.

EVIDENCE_RANK is an ordinal scale and nothing more, which decides what the branch above it can and cannot do. The three integers say that a primary quote outranks reported speech and that reported speech outranks an inference. They do not say by how much, and the gaps between 3, 2 and 1 carry no meaning that survives being written down — nothing about the sources makes the step from inference to reported speech the same size as the step from reported speech to a primary quote. That is why the branch compares the top two ranks and does nothing else with the numbers. It never sums them, never averages them, and never treats two reported-speech claims as jointly outweighing one quote, because none of those operations is defined on an ordering. A tie in an ordinal scale is a genuine absence of information rather than a near miss to be broken by a tiebreaker.

The unresolved return from the stronger-evidence branch is the part that keeps the rule honest. Two claims of equal evidence kind cannot be separated by that rule. A function that quietly picks the first one in that case is a coin flip wearing a policy's name, and it will be read as a decision by everything downstream. Falling through to conflicted, and rendering it, is the correct behaviour of a rule that does not apply.

Attaching one rule to the whole system is the tempting simplification and it is the wrong shape. Recency is close to always correct for a price and close to always wrong for a historical fact, where the more recent source is usually the more derivative one. Stronger-evidence ranking is right when the sources are documents and misleading when one of the sources is a person, because reported speech from a rep who was in the room outranks a quotation from a marketing page in every respect except the one the rank ordering measures. Attaching the rule to the predicate rather than to the system takes about twenty lines of configuration and removes both errors.

Not every finding fits the subject–predicate–value shape. "Vantage's messaging has shifted from cost savings towards compliance" is a claim, is contestable, and has no value field. For findings like these, conflict detection needs a model: cluster the narrative findings, then ask whether any pair is incompatible. Scout ran that as a single call over the finding list and measured it against 120 hand-labelled pairs, where it reached 0.86 precision and 0.71 recall.

Those two numbers describe different populations and it is worth being exact about which. Precision is a property of what the detector flagged: of the pairs it called incompatible, most were. Recall is a property of what existed: of the pairs that really were incompatible, it found under three quarters. The second number is the one that bounds what you can build on, because a detector that misses roughly three in ten conflicts cannot be a gate — a gate has to be trusted about the things it stays silent on. The practical split is to require structured claims for anything a reader might quote as a fact, since those are where a wrong resolution does damage, and accept model-detected clustering for the softer material with the knowledge that some conflicts there go unnoticed.

Where the context window actually binds#

Scout's five worker outputs come to roughly 14,000 tokens. Every model Scout might run fits that several times over, the model saw both pricing claims in the same prompt, and the conflict was resolved badly anyway. That single observation disposes of the idea that context size is what stands between the parent and a good merge, and it is worth being precise about why the idea is tempting.

The belief goes like this: a model that cannot see two facts at once obviously cannot compare them, so a model that can see them presumably can. The first half is true and the second half does not follow. Capacity to hold the inputs is a necessary condition for comparing them, and a necessary condition is not a sufficient one. The quantity that stayed constant here is the window, which was ample throughout; the quantity that varied is whether a policy existed, and it did not. Watching a fixed quantity while a different one moves is how a defence gets believed in, and it is the commonest way an engineering team spends a quarter on the wrong subsystem.

Context size does become a real constraint on aggregation once the fan-out is wide, and the arithmetic for that is straightforward. Cap each worker's output and the total volume of findings is the cap times the number of workers. A competitive review over 24 vendors, at a 1,200-token cap per worker, is 28,800 tokens of findings plus the goal and instructions — still within a window, and already past the point where position effects are measurable: material in the middle of a long context is recalled less reliably than material at either end, so the twelfth vendor's finding is at a structural disadvantage against the first and the twenty-fourth. At 60 workers it stops being a subtle effect and starts being a truncation.

The position effect deserves its causal chain, because it is the reason a comfortable fit is not a safe one. A model reads a prompt as one sequence and attends over the whole of it when producing each token. Attention is a finite resource spread across the sequence, so the share any one passage receives falls as the sequence lengthens. The opening and the closing of a prompt are structurally advantaged — the opening because it conditions everything that follows, the closing because it is nearest the point of generation — and whatever sits between them competes with the most material. A finding placed there is therefore less likely to be recalled than an identical finding placed at either end. Nothing about that depends on the window being full, which is why a run at a third of its capacity still loses claims from the middle.

Two bounded structures answer this, and they trade differently.

Both numbers in the tree's column are forced by its shape rather than chosen. Every merge consumes two items and produces one, so each merge reduces the number of live items by exactly one. Going from 24 worker outputs down to a single answer means the count has to fall by 23, and that takes 23 merges no matter which pairs are chosen or in what order. Serial depth asks a different question: how many merges must happen one after another before the last one can start. Pairing halves the population each round, so 24 becomes 12, then 6, then 3, then 2, then 1 — five rounds, which is the depth in the table below. And the largest context in that path is two worker outputs at the 1,200-token cap plus a merge instruction, which is where the table's figure for the tree comes from.

Staged batches replace halving with a fixed number of groups. Splitting 24 workers into 3 batches of 8, summarising each batch, and running one final call over the three summaries costs 3 summarisation calls plus 1 final synthesis. Serial depth is 2, because the three batch summaries have nothing to do with each other and can run at the same time; only the final call has to wait. The largest context is one batch of eight worker outputs at the same cap.

No column dominates. The tree has the smallest contexts by a wide margin and the worst conflict visibility; single-shot has perfect conflict visibility and the largest context; staged batches sit between them on both. Reading the table as three options to rank is the wrong reading, because the row that decides is the last one and it does not have to be traded against anything.

Two workers whose claims conflict may sit in different subtrees and never meet until level four, by which point each side's claim has been through three rounds of summarisation and the evidence spans have been paraphrased away. That is a real cost of the tree and it is also avoidable, because conflict detection does not have to happen where prose is combined. Grouping by (subject, predicate) is a pass over the whole claim list, it is indifferent to how the prose is later merged, and it costs no model call at all. Detect and resolve conflicts globally, then aggregate prose in whatever bounded structure the width demands, and the last row of the table stops being a constraint on the choice.

Staged batches is the structure Scout uses, with batches formed by question rather than arbitrarily, so that everything bearing on one sub-question is summarised together. It keeps serial depth at two, which matters because each level's latency is a tail latency, and 45,000 aggregation tokens against 31,000 is a 45% premium on the aggregation step and a much smaller one on the run as a whole, since the workers dominate.

A rule decides what the system asserts, not what is true#

A merge rule and the truth come apart in a specific way, and the way is systematic rather than occasional. A rule that prefers the archived pricing page will prefer it even when the page was stale and the CRM note was right, and it will do so on every run where that pair appears. An unwritten policy makes an occasional error, because a model handed the same two paragraphs twice may resolve them differently. A written one makes the same error every time, which converts noise into bias.

That is an argument for recording resolutions rather than for avoiding rules. A dropped list with a reason on every resolution makes the bias auditable, and 200 runs of resolutions is a dataset that can be reviewed to find out whether the ranking you chose reflects the sources you actually have. A bias you can count is a bias you can correct; a bias distributed across a model's preferences is not available for inspection at all.

The rules also only reach conflicts that exist. Two workers agreeing is not evidence, particularly when chapter 2's duplication put them on the same source, and a single worker's claim that nothing contradicts passes through every rule in the table untouched. Follow that through the resolve function and the reason is mechanical: a group of one satisfies the first condition, returns agreed, and is written into the answer as the settled view. The word agreed is accurate about the group and misleading about the world, since a group of one has nothing in it to agree with.

Everything in this chapter is downstream of the claim shape. Findings that can collide can be grouped; findings that are grouped can be compared field by field; comparisons that fail can be resolved by a rule you wrote, recorded with the claims the rule dropped, and audited later against the sources you turned out to have. None of those steps is available to a system whose workers return paragraphs, and none of them becomes available by making the paragraphs fit more comfortably in one prompt.

That is the gap the next chapter is about: a claim can be well-formed, correctly typed, carrying an evidence field, in conflict with nothing, and entirely invented.