Instruction conflicts

Prompts collect instructions because a sentence is the cheapest way to change what a model does. Training is out of reach for most teams, code cannot express "explain what is missing and why", and a rule appended to a system prompt — the standing block of text sent to the model ahead of every request — takes ten minutes and usually works, so every production surprise leaves one behind. After a year that prompt is a rulebook written by several people, none of whom has read all of it, charged against the context budget on every call. That accumulation is prompt bloat, and a prompt carrying so many rules that they begin to overlap and contradict each other is over-constrained. The cost arrives as tokens spent, as behaviour nobody can predict from reading the text, and as pairs of rules that no author ever considered together.

You meet the resulting failure wherever a prompt has been maintained by more than one person for more than a few months: document extraction services, support agents, ticket classifiers, the standing instructions of a coding assistant. The symptom is a behaviour that is right on most traffic and unstable on a subset, with no deploy to blame, nothing in the error logs upstream of whatever consumes the output, and a prompt in which every line reads as sensible on its own. This chapter is about the structure underneath that symptom, and the structure is narrow enough to be worth stating exactly: two rules, one input class where they issue different orders, and no statement of precedence between them.

Two rules and the input class they contest#

Here is a pair, taken from an invoice-extraction prompt of around 1,900 tokens that returns one JSON object per document. The two lines sit about eleven hundred tokens apart, in different sections, written by different people.

# line 4, in the output contract section
Always return valid JSON matching the schema below. Do not include
any prose, preamble, or explanation outside the JSON object.

...

# line 61, in the "handling difficult documents" section
If required information is missing from the document, explain what
is missing and why it could not be determined, so the operations
team can decide whether to request a corrected invoice.

Sort a day's traffic into two piles. In the first are the documents where every field the schema marks as required is present in the text — 235 of the 340 that arrive daily, about seventy percent. On those, both rules are satisfiable at once: the model emits JSON, the JSON is complete, and the second rule never fires because nothing is missing. The two lines coexist for months without producing a single incident.

In the second pile are the 105 documents where at least one required field is genuinely absent — a scanned fax with the tax ID cropped off, a proforma invoice with no invoice number yet, a hand-annotated PDF where the total is written in the margin and does not survive text extraction. On these, the two rules issue different orders about the same observable. The format rule says the response must be a JSON object and must contain no prose. The difficult-document rule says the response must contain an explanation of what is missing and why. The schema has no field for that explanation, so there is no output that satisfies both. About forty of those 105 come back as prose, which is the roughly thirty-eight percent that produces a twelve percent failure rate over the whole day's traffic.

Those proportions are worth reconstructing rather than reading past, because the same three numbers come back as the incident report, the ticket and the eventual success criterion, and people argue about this failure mostly by using different denominators without saying so. Of the three hundred and forty documents that arrive daily, two hundred and thirty-five carry every required field, which is a shade under seven in ten; the remaining hundred and five are the contested class. Of that hundred and five, about forty come back as prose, which is a shade under two in five — that is the rate at which the undeclared contest resolves toward the difficult-document rule. Divide the same forty by the whole day's three hundred and forty instead, and you have the failure rate the downstream consumer experiences, a little under one call in eight. One quantity, three denominators.

Nothing in the prompt tells the model which line wins. There is no ordering statement, no "in case of conflict", no marking of one as hard and the other as soft. The model is being asked to resolve a contradiction that the prompt's authors never resolved, and it resolves it once per call, independently, with no memory of what it decided last time.

Why the split is probabilistic rather than consistent#

The whole disagreement collapses onto the first token of the response. If the model emits {, the rest of the completion is effectively committed: it is now inside a JSON object, the schema constrains what comes next, and the explanation has nowhere to go except perhaps stuffed into a string field. If it emits The or I, it has started a sentence, and the JSON is gone. Autoregressive generation makes that first choice self-reinforcing, so a decision that could have gone either way at token one is locked in by token five.

"Self-reinforcing" is carrying a mechanism, and the mechanism is one property of the decoder: it emits tokens left to right and never revises one it has already emitted. Each new token is chosen conditioned on everything emitted before it. So the moment { is out, the model's own output has become part of its input, and the continuations carrying any weight are the ones that continue a JSON object. The difficult-document rule is still exerting pull at that point, but it has nowhere to pull to that would not first require unwriting a character the decoder has no mechanism for unwriting. The same holds in the other direction after The. That is why the two responses shown below are each internally coherent rather than a muddle of the two: the contest is settled once, early, and everything after it is consistent with whichever side won.

At the branch point the logits for those two candidates are close. In one instrumented run on a missing-tax-ID document, the top candidates carried log probabilities of about -0.42 for { and -1.09 for The, which after temperature scaling at 0.2 works out to roughly a two-in-three chance of JSON and one-in-three chance of prose. Those numbers move document by document, because the strength of each rule's pull depends on how loudly the document itself signals that something is missing. A fax with one cropped field pulls weakly toward prose; a proforma missing three required fields pulls hard.

Undoing the logarithm is the step that makes those two figures readable. A log probability is the natural logarithm of a probability, so raising e to each of them recovers the probability: e to the minus nought point four two is a little under two thirds, and e to the minus one point nought nine is a little over one third. The substantive fact is that the two add to essentially all of the probability available. No third candidate carries meaningful weight, so the model is not wandering among many options — it is choosing between exactly two, at about those odds, and that is what makes the output distribution bimodal rather than merely varied. Temperature is the dial that widens or narrows a gap of this kind: it divides the scores before the probabilities are computed, so a value below one stretches the distance between candidates and makes the leader more dominant, while a value above one compresses it. What temperature cannot do is invent a third option, or remove the disagreement that produced the second one.

This is the mechanical reason the failure looks like flakiness. It is not noise in the infrastructure and it is not a model that "sometimes ignores instructions" — it is a genuinely undetermined decision being made by a sampler. Setting temperature to 0 does not repair it. Greedy decoding freezes the coin, so each individual document becomes deterministic, but different documents still land on different sides according to where their logits fall, and the whole pattern reshuffles on the next model version because the margins are small enough that a point release moves them.

// same document, same prompt, same model, two calls
{"invoice_number":"INV-88213","vendor_tax_id":null,"total":1284.50,"currency":"EUR"}

I was unable to determine the vendor tax ID for this invoice. The document
header is cropped on the right-hand side and the registration number that
normally appears there is not present in the extracted text. All other
required fields were found: invoice number INV-88213, total 1284.50 EUR.

Both responses are obedient. They obey different lines.

Where conflicts hide#

Reading a long prompt looking for contradictory wording will not find this, because the wording is not contradictory. Always return valid JSON and explain what is missing share no vocabulary, sit in different sections, and were written in different registers. What they share is an input class and an observable. Conflicts are structural, so the search should be structural too, and there are four structures that account for most of them.

Exception clauses are the most common source in prompts that have been maintained for a while, because an exception is what you write when a specific case burned you. Somebody hits a bad path in production, writes a rule that starts with If, and appends it to whichever section seems closest. The rule is correct about the case it was written for and silent about everything else in the prompt that it now cuts across. Scope conflicts are the second most common and are usually created by a copied template, because templates come with confident global statements like "always respond in English" that made sense in the system they were copied from.

There is a mechanical check that finds all four, and it takes about twenty minutes on a thirty-rule prompt. For each numbered instruction, write down two things: the input class it applies to, expressed as a predicate you could actually evaluate on a request, and the observable it constrains — output shape, output length, tone, which tool the model asks the runtime to run on its behalf, whether a refusal is allowed. Two rules are in conflict when their input classes intersect and they constrain the same observable to different values. In the invoice case the predicates are every request and document is missing a required field, which obviously intersect, and both constrain output shape. Written down in that form it takes four seconds to see.

Ambiguity, redundancy and the fixes that only hide it#

A conflict, in the sense set out above, is two clear instructions that cannot both be obeyed on some input class. That is worth separating from ambiguity, which is one instruction that does not say enough — "keep the summary short" produces variance in length, but it produces a distribution rather than two modes, and the fix is to say what short means. If your outputs cluster into two distinct shapes, look for a conflict. If they spread smoothly, look for an underspecified rule.

It is also worth separating from redundancy. Three instructions about tone that all say roughly the same thing in different words cost tokens and make the prompt harder to maintain, and they are the subject of later chapters in this topic, but they do not produce bimodal output because they do not disagree. Not every pair of rules that mention the same topic is a conflict, and treating overlap as automatically pathological will have you deleting things that were load-bearing.

The invoice failure is also not a validation problem, although it will be tempting to treat it as one. Constrained decoding — JSON mode, a grammar, a structured-output parameter that forces the sampler to stay inside the schema — makes the parse errors stop immediately, because the token The becomes unreachable at position one. What it does not do is resolve the disagreement. The model still wants to report the missing tax ID, has nowhere to put that report, and now emits a well-formed object in which vendor_tax_id is null or, worse, a plausible-looking number reconstructed from context. The forty visible failures a day become zero visible failures and an unknown number of quietly wrong records.

"Unreachable" there is literal rather than figurative, and knowing why is what stops the fix being mistaken for a repair. A constrained decoder holds a description of the outputs that are legal — a schema, a grammar — and at each step works out which tokens could still lead to a legal output. Everything else is masked: its score is driven to nothing before the sampler is shown the candidate list at all, and the probability that would have gone to it is spread across the survivors. At position one, The cannot begin a JSON object, so it is masked, and the whole of the weight the difficult-document rule had put behind the prose branch is redistributed over tokens that continue an object. The rule's pull did not weaken. It was rerouted into the only channel still open, which is a field of the schema — and a field of the schema accepts a fabricated value exactly as readily as a real one.

That is not an argument against constrained decoding, which is a good tool. It is an observation about the shape of a whole class of fixes: the ones that make a conflict stop being visible without making it stop existing.

A conflict is a structural property of a pair of rules, not a property of their wording, and that is why it survives careful reading. Two instructions written in different registers, in different sections, months apart, by people solving different problems, will read as sensible individually for as long as anyone cares to look at them individually. What locates the pair is the pair of questions the mechanical check asks — which requests does this apply to, and what does it constrain — because a conflict is exactly an intersection in the first answer plus a disagreement in the second.

The most popular member of the class of fixes that hide rather than resolve is neither a validator nor a grammar. It is a single line appended to the end of the prompt declaring which rule wins, and it takes about eleven minutes to write.