Saying no by naming the adjacent thing you can measure

Legal operations wants a Slack bot that answers any question about any of the 14,000 contracts in the repository. The prototype takes three weeks and demonstrates well. Measured properly — 300 questions sampled from a year of the team's actual Slack threads, graded by two paralegals against a rubric — it comes back at 61%, with an exact interval of 55% to 67%. The wrong answers are not obviously wrong. They cite a real clause from a real contract and misread it, and they are written in the same confident register as the right ones, so a reader who did not already know the answer has no way to tell which kind they are holding.

Nothing in the previous chapters rescues this. The calibration arithmetic says 61% over 400 questions a week is around 150 wrong answers a week landing on a team of four. The criterion could be written properly and would then be failed properly. The disclosure machinery would report the number honestly. What none of that changes is that 61% with undetectable errors is not a product, and the decision in front of you is whether to say so.

The half-delivery is the expensive option#

The instinct is to ship it as a beta with a disclaimer, because a beta is not a promise and the disclaimer transfers the checking to the user. That reasoning fails on the mechanics of how the thing gets used.

An API that returns HTTP 422 on input it cannot handle is easy to build against, because the failure is at the boundary and the caller's code branches on it. An API that returns 200 with subtly wrong data on a third of calls is much worse than one that is down, since a caller cannot write a branch for a failure that is indistinguishable from success. The contract bot is the second kind. A disclaimer at the top of a Slack channel does not restore detectability, because the person asking the question is asking precisely because they do not know the answer, so they cannot audit the response without doing the work the bot was supposed to do.

What follows from that is a drift you can watch happen in the usage logs over about six months. Early on, people ask it easy questions they could have answered themselves, and it does well, so trust builds on the easy end. Then they start giving it the work they least want to do, which is the multi-contract comparison — does any of our top twenty MSAs, the master service agreements that govern the largest accounts, have a shorter notice period than ninety days — and that is the slice where measured accuracy is 41% rather than 61%. By month six, 38% of queries are of that kind, so the majority of the value being claimed sits in the part of the distribution that works worst, and somebody in finance has built a quarterly exposure summary that consumes its output. Turning the bot off at that point is an incident affecting a reporting process, which is a much larger event than never having shipped it, and the people most upset will be the ones who were told it was a beta.

That is the specific harm of the quiet half-delivery: users cannot see the rate, so they cannot avoid the tail, and the tail becomes load-bearing while nobody is looking.

Say the boundary, then say the number you can stand behind#

The refusal is one sentence and it names the rate rather than the difficulty: we measured 61% on 300 of your own questions, the errors are confident and hard to spot, and there is no version of this scope I would put in writing. Then, immediately, the alternatives, each with a measurement attached.

Two of the entries in that table need reading before the comparison works. The 12% abstention against option A is the share of questions the system declines to answer at all, handing back the passages it found and no conclusion, and that decline is what buys the accuracy on the questions it does answer. The recall@5 against option C is a retrieval measure rather than an answer measure: for 91% of the questions the correct passage appears somewhere in the top five results, and whether the reader recognises it when they see it is not a thing that number claims.

The table is what makes this a negotiation rather than a rejection. Legal operations came in with one option that had no number on it and they leave with three that do, and the choice between them is theirs, because the trade-offs are about their team's time and their tolerance for latency. In practice they usually take a combination — the bounded extraction for the clause types that come up weekly, the review queue for everything else — and the important property is that neither path leaves anybody depending on an unmeasured 41%.

Two responses come back reliably and both have short answers. The first is whether it can be improved, and the honest reply is that some of the gap is closable and the shape of the errors says how much: of the 117 wrong answers in the sample, 71 came from the retrieval step returning the wrong clause, which is an engineering problem with a known toolbox, and 46 came from misreading a clause that was correctly retrieved, which is not. Fixing all of the first category and none of the second gets you to about 85%, which is still not a number anyone should build a reporting process on. The second response is that somebody in another team has a working version, and that is the demo problem from earlier in this topic, answered with the same question about population and sample size.

Timing changes what a refusal costs by an order of magnitude. Saying no in week three, when the artefact at risk is a three-week prototype, is a conversation about disappointment. Saying the same thing in month nine, after the finance summary has been built on top of it and two teams have retired the manual process, is a migration with a deadline and an angry sponsor. That asymmetry is the argument for spending the week on the 300-question measurement before anything is announced internally, because the measurement is what allows the refusal to happen while it is still cheap. It is also why "let's ship it and see" is a more expensive experiment than it sounds; the thing you find out is not whether the feature works, since you could have measured that, but how hard it will be to remove.

Abstention is what makes the narrow version narrow#

Option A does not reach 94% by using a better model. It reaches it by declining to answer when the evidence is thin, which trades coverage for accuracy on the questions it does answer.

def answer(question: str, gate: float = 0.62) -> Response:
    hits = retrieve(question, k=8)
    if not hits or hits[0].score < gate:
        return Abstain(reason="no_supporting_passage", passages=hits[:3])

    draft = model.answer(question, hits)
    if not draft.citations or draft.support_score < gate:
        return Abstain(reason="claim_not_supported", passages=hits[:3])

    return Answer(text=draft.text, citations=draft.citations)

The gate value is a product decision with a measurable consequence, and the sweep looks like this on the same 300-question set.

Reading the last column against the first shows what the boundary actually buys: the same system, the same model, and a hundredfold difference in wrong answers reaching a person. It also shows the cost honestly, because the abstentions do not evaporate. At the 0.74 gate, two-thirds of the traffic gets "I could not find support for this, here are three passages that look relevant" and goes to a human, so the question for legal operations is whether that is a useful outcome or an insult. For contract questions it is generally useful, since the three passages are most of the work; for a customer-facing assistant answering billing questions it would be neither, and the honest conclusion there might be that no gate setting produces a shippable product.

Abstention only works when the abstention itself is honest, which is a measurement of its own. A gate tuned on 300 questions will look better on those 300 than on next quarter's, and the failure mode to watch is the confidently wrong answer that clears the gate, since those are the ones the gate was supposed to catch. Reporting accuracy on answered without reporting coverage next to it is the standard way this metric gets used to mislead, usually without anybody intending to.

Making the no revisable#

A boundary stated with no route past it invites people to route around you, which is how a shadow version of the tool ends up running on somebody's laptop. The version that holds is a no with a condition and a date attached: the multi-contract comparison case fails because the retrieval step cannot assemble the right set of clauses across documents, and if we get 800 labelled comparison questions from your team's history we can test whether a different retrieval design moves it, and we will know within three weeks. That is a genuine offer, and it also transfers a real cost to the requester, which is a reasonable test of how much they want it.

It matters that the boundary is about the measured rate rather than the difficulty of the work. "That is hard" invites a negotiation about effort and implies that more engineers would fix it. "We measured 61% and the errors are undetectable at point of use" is a fact about the current system on their data, and it either changes when something changes or it does not. Keeping the refusal factual is also what makes it survivable when a colleague builds a prototype that appears to do the thing; the response is the population and the number, and the demo has already been dealt with.

The weakest part of everything above is the review queue in option B, and it is worth saying so to the people who choose it. A reviewer who sees drafts that are correct 97% of the time after their own corrections will, by about week six, stop reading them closely, so the measured 97% quietly becomes something lower without anything in the system changing. The countermeasure is to seed the queue with known-bad drafts at a rate of about 2% and report the catch rate monthly; a catch rate that falls from 90% to 55% is the signal that the human step has become a rubber stamp, and it is far better to find that in a monthly metric than in a contract dispute. Any design that puts a person in the loop as its quality mechanism owes that person a measurement, otherwise the number you are quoting to stakeholders is the number the process had on its first day.

The next request of this kind will arrive before you have a measurement, and the useful discipline is to produce one before the conversations start rather than after. Three hundred questions sampled from what the team already asks each other, graded against a written rubric by two people, is usually under a week of work, and it is the artefact that decides whether the rest of this is a calibration exercise, a launch criterion, or a refusal with a narrower yes attached to it.