Model judges as verifiers

An LLM judge is a second model call that reads an output, the request that produced it and whatever context it needs, and returns a verdict. It is the proposal that arrives in every design review once a deterministic check has been shipped and found wanting, and it arrives with a good argument attached: the deterministic check misses the semantic errors, and a model can see them.

That argument is correct, and the swap is not free in ways that do not show up in a demo. A checker has four properties worth naming separately — what it covers, whether its verdict is stable, whether it fails for the same reasons the generator does, and what it costs per candidate — and a judge moves all four at once, three of them in the wrong direction. This chapter measures each on the same batch of work, prices the loop that results, and sets out the three arrangements in which the coverage is worth having.

Four properties, and which survive the swap#

Take the analytics agent from earlier in this topic, which writes SQL against a warehouse and still ships wrong numbers at roughly the rate it always did, EXPLAIN having removed only the queries that failed to parse. The proposed replacement sends the generated SQL, the question that produced it and the relevant slice of the schema to a larger model with an instruction to reply ACCEPT or REJECT with a reason.

Run over 200 queries drawn from real traffic and hand-labelled beforehand — 160 correct, 40 semantically wrong — the judge rejected 23 of the 40 wrong queries and 19 of the 160 correct ones. That is a 57.5% catch rate on the errors that motivated the whole exercise, and an 11.9% false reject rate on work that was already right, each false reject buying a regeneration of a correct query. EXPLAIN, on the same 200, caught zero of the 40 and rejected none of the 160, because every query in the batch was executable.

The judge is unambiguously covering ground the deterministic check cannot reach, and it is worth being precise about what was paid for that ground. Both rates above are conditional on the label, which is what makes them comparable across batches with different error mixes: 23 out of the 40 known-wrong queries is 57.5%, and 19 out of the 160 known-correct ones is a bit under twelve in a hundred. Add the two rejection counts and the judge rejected 42 queries out of 200 — a fifth of the batch, of which slightly more than half were mistakes on its part. That single figure is the one the loop will actually pay, because a loop regenerates on a rejection without knowing whether the rejection was earned. Any calculation about cost in the next section runs on the 21%, and any calculation about quality runs on the 57.5% and the 11.9%, and mixing them up is how a judge gets priced as though only its correct rejections cost money.

Reproducibility is the first casualty and the one people expect least, because temperature 0 — the decoding setting that makes the model always take its single highest-scoring next token instead of sampling among the plausible ones — has a reputation for determinism it does not deserve. Greedy decoding fixes the sampling step and leaves everything else in play: floating-point addition is not associative, so a change in server-side batch size changes the last bits of a logit, which is the raw score the model assigns each candidate token before those scores are turned into probabilities, and a mixture-of-experts router, the component that sends each token through only a few of the network's many specialised sub-blocks, can send the same token down a different path depending on what else is in the batch. The observed consequence in this measurement was that 27 of the 200 queries received different verdicts on different runs of an identical request. A check whose answer depends on how busy the provider was cannot be the thing your deployment gate consults, and it makes the ordinary debugging move — rerun it and see — meaningless, because a rerun is a new sample rather than a reproduction.

Independence is the subtler loss, and the one that decides how much the coverage number is worth. A planner has no opinions about business logic; its correctness is unrelated to whether the generator understood the schema, so the two fail for uncorrelated reasons. A judge from the same model family has read the same documentation, formed the same impression that shipments joins cleanly to orders, and inherits the same blind spot. Of the 17 semantic errors the judge missed, 12 were errors of exactly the kind the generator makes most often, which is the pattern you would expect if the generator's mistakes and the judge's misses have a common cause. Running a similarly-sized judge from a different vendor over the same 40 confirmed it: it caught 26, and only 9 of its 14 misses overlapped with the first judge's 17. Two judges from different families ensembled would have caught 34 of 40, at twice the price, which is a real option and a real bill.

The quantity that decides whether the second judge is worth buying is not its own catch rate, and reaching for the catch rate is the natural mistake. A pair of checkers run together can only fail on the errors both of them miss, so what the second one adds is entirely determined by how much of its blind spot it shares with the first. Where the overlap between the two miss sets is small, the pair sees nearly everything either one can see and the second judge is buying new coverage. Where the overlap is large, the second judge is buying a copy of the first one's blind spot at full price, and its impressive individual catch rate is being spent on errors that were already caught. This is the same argument as independence from the generator, applied one level along: two checkers are worth having together in proportion to how differently they are wrong, not in proportion to how often they are right.

That framing also explains why the deterministic check is not made redundant by a judge that covers strictly more semantic ground. EXPLAIN and the judge have almost no overlap in what they miss, because one of them is answering a question about the catalogue and the other about meaning. The judge shares its blind spot with the generator, which is the pairing that matters and the one that is invisible on a coverage table.

What the rejection loop costs once the judge is in it#

Pricing this per accepted output changes the conversation more than the accuracy numbers do. The judge rejects 42 of 200 first attempts, a 21% per-attempt rejection rate, which under the same conditional-failure profile as the provisioning service works out to about 1.27 generations and 1.27 reviews per accepted output. Generation is $0.0125 and review is $0.072, so one accepted output costs roughly $0.107 against $0.0156 for the EXPLAIN loop. The check is now 85% of the pipeline's model spend, and the loop is spending five and a half times as much on judging a candidate as on producing it.

Every figure in that paragraph comes out of one multiplication and it is worth performing, because the structure of the answer is the argument. One attempt now costs a generation plus a review: $0.0125 and $0.072, which is a bit under nine cents. Multiply by the 1.27 attempts an accepted output consumes and the result is a bit under eleven cents. Divide the review's share into the attempt total and it is 85 in a hundred. Divide the review by the generation and it is five and a half. Nothing subtle is happening; the review is simply the large term, and every ratio in the paragraph is a restatement of that.

Two of those numbers are worth looking at sideways. The judge's $0.072 for 3,900 input and 180 output tokens implies a token rate several times the generator's, which is what "a larger model" means on a price list and is the whole reason the term dominates. And the $0.0156 comparison figure is the EXPLAIN loop from the earlier chapter, where the check contributed essentially nothing and the entire cost was one and a quarter generations. So the swap has not made checking more expensive by some percentage. It has moved the check from a term small enough to omit from the model to the term that sets the bill.

Latency moves the same way and lands where it always lands. A judge at 2.3 seconds p50 and 6.1 seconds p95 runs on every attempt including the rejected ones, so a three-attempt request carries three reviews, and its exposure to at least one 6.1-second review is 14.3%. End-to-end p95 for this configuration measured 11.8 seconds against 4.7 seconds for the EXPLAIN version. For an interactive analytics tool where somebody is watching a spinner, that is the difference between a tool people use and a tool people work around, and no amount of accuracy on the outputs that do arrive compensates for the ones nobody waited for.

The comparison that decides it is against the residual. A wrong analytics number that lands in a board deck can cost a great deal more than $0.09, and if the judge removes 57.5% of those, the arithmetic can easily favour it. That calculation has to be made with the severity number written down, because the version done in someone's head always assumes the errors are caught before they matter.

That $0.09 is the difference between the two loops — a bit under eleven cents against a bit over one and a half — so it is what the judge costs per accepted output. Setting it against the cost of one escaped wrong number needs one more step, and skipping the step is how this comparison usually gets made too favourably. The two quantities have different denominators. Nine cents is spent on every output, and an escaped error occurs on only some of them. On this batch the semantically wrong queries were a fifth of traffic and the judge removed a bit over half of those, so each accepted output buys somewhere around one-ninth of an avoided error rather than a whole one. The severity at which the full judge pays for itself is therefore roughly nine times the nine cents, not equal to it. The general form is the sentence to keep: a check pays for itself when its extra cost per output is less than the cost of an escaped error multiplied by the share of outputs on which it prevents one — and that share is a product of two rates, both of which are usually smaller than people carry in their heads.

Three arrangements that keep the coverage#

The first is triage rather than adjudication. Instead of regenerating on a REJECT, route the flagged query to the analyst with the judge's reason attached, and let the human decide. The judge's stochasticity stops mattering, because a wrong flag costs thirty seconds of attention instead of a regeneration and a coin flip; its false rejects become the price of a queue that is ordered usefully. At a 21% flag rate on this traffic that is roughly one query in five reaching a person, which the team could staff and which caught considerably more than 57.5% of the errors, since the analyst reading a flagged query with a specific hypothesis in front of them is a better detector than the judge alone.

The second is monitoring against a labelled set. A judge that agrees with human labels 88% of the time on a class you care about can watch every output continuously and alert on a shift, provided its agreement is re-measured against fresh labels on a schedule. The relationship between judges and labelled evaluation was set out when we measured false accepts; the point here is only that a judge is usable as a sensor at a quality that would be unacceptable in a gate, because a sensor's errors average out over thousands of observations and a gate's do not.

The third arrangement is the interesting one, because it restores the asymmetry rather than working around its absence. Ask the judge for a machine-checkable hypothesis instead of a verdict, then check the hypothesis deterministically.

class JoinFanoutHypothesis(BaseModel):
    suspected: bool
    join_table: str | None      # the table the judge thinks multiplies rows
    grain_key: str | None       # column that should be unique per fact row
    probe_sql: str | None       # must be a SELECT; validated before execution

def check_fanout(candidate_sql: str, h: JoinFanoutHypothesis, wh) -> Verdict:
    if not h.suspected:
        return Verdict(True, None)
    if not is_read_only(h.probe_sql):          # rejected before it reaches the warehouse
        return Verdict(True, None)
    rows = wh.execute(h.probe_sql)             # e.g. SELECT COUNT(*), COUNT(DISTINCT oi.id) ...
    total, distinct = rows[0]
    if total > distinct:
        return Verdict(
            False,
            f"{h.join_table} fans out {h.grain_key}: {total} rows for {distinct} distinct keys",
        )
    return Verdict(True, None)

The reason that probe decides anything is a counting argument, and it is the same one that made the revenue query wrong in the first place. A join that fans out does not corrupt any individual value; it repeats rows. So if the result set contains more rows than there are distinct fact keys in it, some fact key appears more than once, and any sum over those rows counts something twice. COUNT(*) returns how many rows there are and COUNT(DISTINCT oi.id) returns how many distinct item rows those correspond to, and the two are equal exactly when nothing has been duplicated. The comparison needs no knowledge of what the question meant, no opinion about business logic, and no model — which is why its verdict has the properties the judge's verdict does not.

What the split has done is give each component the job it is good at. Noticing that a fan-out might be present requires reading the query against the question, which is a semantic act and the thing a model is for. Deciding whether one is actually present requires counting rows, which is a mechanical act and the thing a warehouse is for. The judge's stochasticity now only affects which queries get probed, and a probe that runs unnecessarily costs 900 ms and returns accept.

The third argument of the hypothesis is a piece of SQL the model wrote, and the is_read_only guard is the line to notice. If the probe is not a read, the function returns Verdict(True, None) — it accepts. That is a deliberate choice and it is the same distinction as a fault versus a verdict: a malformed probe is a failure of the checker, and a checker that cannot decide must not resolve its own confusion into a rejection. Fail-closed here would let the judge cause regenerations by emitting garbage, which is a loop that spends money in proportion to how badly its own checker is behaving. Fail-open costs a missed catch, which is what the arrangement was already accepting from the judge's misses. The guard is also the only thing standing between a model's free-text output and the warehouse, which is a second reason it belongs before execute rather than after it.

The judge is still stochastic and still correlated with the generator, so it will miss fan-outs it should have suspected. What it can no longer do is reject a correct query on a hunch, because the verdict comes from a COUNT(*) against COUNT(DISTINCT) and that comparison is reproducible, cheap at about 900 ms of warehouse time, and independent of every model in the pipeline. Running this variant on the same 200 queries caught 21 of the 40 wrong ones with 2 false rejects, both of which were genuine one-to-many joins that the query intended. Coverage dropped by two errors relative to the raw judge and false rejects fell by seventeen, and the reason string is now something an analyst can check in ten seconds.

Both of those movements are subtractions against the earlier figures and both are worth stating in their own units, because they are not the same currency. Coverage went from 23 of the 40 wrong queries to 21, a loss of two errors that will now escape. False rejects went from 19 of the 160 correct queries to 2, which is seventeen regenerations of already-correct work no longer paid for. Trading two escapes for seventeen avoided regenerations is a good trade at almost any severity a team would tolerate, and the reason it was available at all is that the two error types were never symmetric: the judge's false rejects came from suspicion, and suspicion is exactly the part that a COUNT can adjudicate.

Grading its own homework#

A judge is not an evaluator, though the same model and the same prompt will happily serve as both, and conflating them produces a system that grades its own homework. An evaluation runs offline over a fixed set and reports a number that people compare across versions. A gate runs online and decides what ships. A judge used as a gate also becomes the target of any prompt tuning you do, which is Goodhart's law arriving on schedule, since the generator's prompt gets adjusted until the judge stops complaining.

It is also not the same thing as sampling the generator several times and taking the majority answer, which is the cheaper move that gets proposed in the same meeting. Self-consistency across five samples caught 8 of the 40 semantic errors here, because a model that misreads a schema misreads it consistently, and agreement among correlated samples is evidence of consistency rather than of correctness. Follow the mechanism and the low number stops being surprising. Majority voting works when each sample is an independent attempt whose errors scatter, so that wrong answers disagree with each other and the right one accumulates. Five samples from one model on one prompt are not independent attempts. They differ in wording and agree on interpretation, so a schema misreading appears in all five and wins its own vote unanimously. The procedure suppresses exactly the errors that come from sampling — a stray token, an inconsistent alias — and leaves untouched every error that comes from the model's understanding, which is the entire category anyone installed a checker for. That is the same correlation problem as the judge, without the compensating benefit of a different vantage point, and it costs five generations. Understanding Verification Dynamics in Large Language Models is worth reading on how the generate-versus-verify gap actually behaves across task types, because it is much smaller on some tasks than the folk model assumes and the folk model is what these proposals are built on.

A model judge buys coverage and sells reproducibility, independence and price, and the exchange rate is bad enough that installing one as a gate is almost never the right first move. Installed as a triage queue, as a monitor calibrated against human labels, or as a generator of hypotheses that something deterministic then decides, the same coverage arrives without the properties being given up — and the third of those is the only arrangement that puts the asymmetry back.

Whatever you install — a planner, a judge, a hypothesis-and-probe hybrid — the thing it produces is a pass rate, and a pass rate is a fact about the checker rather than about the work. The next chapter follows one of those numbers into a service that has been reporting it to stakeholders as its quality metric for two quarters, and works out what raising the retry ceiling can and cannot do to it.