Lock-in is a decision with an exit trigger and a reversal cost

The product is contract review. A customer uploads an agreement, the system reads it against a 19,000-token policy prefix, and returns a marked-up list of clauses with risk ratings. Gross margin is 71%, and it is 71% because the policy prefix is cached: without caching the same workload runs at a margin of 38%, which is below the 55% floor the board wrote into the last plan.

In March a competing provider shipped a model that is clearly better at this task. On the company's own 240-case suite the rubric score goes from 0.86 to 0.91, and in blind pairwise comparison by two contract lawyers the new model's output was preferred in 62% of cases against 24% for the incumbent, with 14% ties. The new provider has no prompt caching. Running the same prefix through it at full input price would take the margin to 38% and change the company from one that is default-alive, meaning it reaches profitability on the money already in the bank, to one that needs a price rise or another funding round.

Both of the obvious positions are defensible, and both are wrong in the same way, which is that they treat the question as a single binary with no expiry date.

Both cases, with the numbers each one rests on#

The quality case is not sentimental. A bake-off is the evaluation a prospective customer runs before buying, putting their own contracts through two vendors' products and comparing what comes back, so a loss reason recorded there is as close to a controlled quality measurement as the sales pipeline produces. Three lost bake-offs out of eleven, with the loss reason recorded as clause-detection misses, is roughly $540k of annual contract value that a 0.05 rubric improvement plausibly addresses, and the four churned accounts are another $186k. Quality problems in a legal product also compound through references, and a reputation for missing clauses is expensive to shed once it exists in a market of a few hundred buyers who talk to each other.

The cost case is not timid. A margin of 38% at this company's burn is seven months of runway, and the fundraising environment does not reward a company that has just made its unit economics — what a single contract review costs to serve, set against what the customer pays for it — worse in order to improve a rubric score by five points. The 70% saving is also not a fluke of one vendor's pricing; it is the return on having built a 19,000-token policy prefix that is stable across every customer, which is an asset the company created and which caching is the mechanism for monetising.

The decomposition that both positions miss#

The argument is framed as a choice of provider, and the seam from chapter 2 means it does not have to be. Contract review is not one task. The breakdown by volume and by where quality actually decides anything looks like this.

risk-rate and answer-question are where the lawyers' preference concentrated and where the lost bake-offs were decided; together they are 17% of spend. Moving those two to the better model, keeping clause-extract and summarise-review on the cached incumbent, gives a blended cost per review of $0.23 and a margin of 61%, with the rubric on the two decisive tasks at 0.91. That option existed the whole time and appeared on nobody's slide, because the question had been posed as "which provider are we", and the eval evidence was per task while the decision was being made per company.

The split is not free. Two providers means two sets of credentials, two rate-limit budgets, two error taxonomies, and roughly a week of transport work — a cost the shared transport client from chapter 1 makes bounded and legible.

The levers the binary framing hides#

Before accepting either horn, it is worth listing what else moves the cost side, since "caching or 38% margin" is only true if caching is the sole lever. Four others existed here and three of them were used.

The policy prefix had grown to 19,000 tokens by accretion and had never been pruned; an afternoon of measurement found that 3,100 tokens of it were examples for clause types that had not appeared in production traffic in six months, and removing them cost 0.004 on the rubric. A batch endpoint was available on the better provider at half price for the overnight re-review job, which is 22% of clause-extract volume and does not need to be interactive. Routing was available in the other direction as well: 60% of uploaded agreements are one of four standard templates, and for those the cheaper sibling model in the incumbent family scores within 0.01 of the larger one. And list pricing had not moved in two years while the product had added two significant capabilities, which makes a 14% rise a conversation rather than an emergency.

None of those individually rescues the margin, and together they change the arithmetic enough that the choice stops being existential. That matters for the decision quality, because a trade framed as company-threatening gets decided by whoever is most senior in the room, and a trade framed as six points of margin against five points of rubric gets decided by evidence.

Write the record while you are making the decision#

What makes the remaining dependency rational is that its terms are written down at the moment of adoption, when the person who understands the trade is in the room. The artefact that holds them is an architecture decision record, usually shortened to ADR: a one-page file kept in the repository that states a decision, the alternatives weighed, and the reasoning, so that the reasoning survives the people who did it. The record is short and its fields are chosen so that a stranger can act on it two years later.

# ADR-118: Depend on vendor-a prompt caching for clause-extract

Status: accepted, 2026-04-02 · Owner: platform-ml · Decider: VP Eng
Review: 2026-10-02, and on any trigger below

## The dependency
clause-extract and summarise-review bind to vendor-a and use its cache
breakpoint after the 19k-token policy prefix. `render_prompt` for these two
tasks is vendor-a-specific and has no qualified counterpart elsewhere.

## Expected payoff, measured
Measured over 30 days on production traffic (dashboard: margin/by-task):
  cost per review        $0.51 -> $0.15   (-71%)
  time to first token    1,900ms -> 640ms
  gross margin           38% -> 71%
Re-measured monthly. If the saving falls below 45% for two consecutive
months, this record is void and must be re-argued.

## What we are giving up
Rubric 0.86 vs 0.91 available elsewhere on this task. Accepted because
clause-extract quality was not cited in any lost bake-off; risk-rate and
answer-question, which were, are bound to vendor-b as of ADR-119.

## Exit triggers (any one fires a review within 10 working days)
T1  clause-extract rubric gap to best available exceeds 0.08 on our suite
T2  clause quality cited as loss reason in >=2 bake-offs in one quarter
T3  vendor-a cache read pricing rises above $0.60/M, or TTL falls below 5 min
T4  a cached-equivalent capability ships at vendor-b and qualifies
T5  vendor-a announces sunset of the bound identifier (registry handles)

## Reversal cost, and how we know
3 engineer-weeks, rehearsed 2026-03-19 against vendor-b on 10% traffic:
  binding rewrite 4d · retune 5d · shadow+adjudication 4d · rollout 2d
Margin impact on reversal: -33pp at current pricing, requiring either a
14% list price rise or routing 60% of clause-extract to a smaller model.
Next rehearsal: 2026-09-19. If a rehearsal is skipped, treat the 3 weeks
as unverified and re-estimate before relying on it.

The reversal-cost section is the one that changes behaviour, because it is the only claim in the record that decays. An estimate of three weeks written in April and never tested is worth very little by the following spring, since the prompts will have moved, the vendor-b binding will have drifted out of qualification, and the person who wrote the estimate may have left. Keeping it true means keeping the alternative binding alive in the nightly compatibility run from chapter 6, which generalises from model dependencies to feature dependencies without any change to the mechanism: an unqualified escape route is a plan, and a qualified one is an option.

What a rehearsal consists of#

The March rehearsal named in the record took one engineer three days and is the reason the three-week figure is worth anything. It ran the alternative binding for clause-extract against 10% of production traffic for 48 hours, serving from the incumbent and comparing in shadow, with the explicit goal of finding out what the estimate had missed rather than of proving the estimate right.

It found three things. The alternative provider's rate limits were expressed per-minute rather than per-day, and the review job's bursty shape hit them at 10% traffic, which would have needed a queue at 100% and was two days of work nobody had counted. The 19,000-token prefix pushed each request past the point where the alternative's latency became visible in the UI, at 3.4 seconds to first token against 0.64, which turned a cost decision into a product decision about a spinner. And the eval suite's cost gate had been written as an absolute dollar figure that assumed cached pricing, so it failed on the alternative for a reason that had nothing to do with the alternative's quality, which meant the suite would have blocked a legitimate reversal.

All three were cheap to fix once known and none of them were guessable from a spreadsheet. The rehearsal's output was an amended estimate, from two weeks to three, and a rate-limit runbook. Running it twice a year is roughly six engineer-days annually, which is the standing cost of keeping the exit real rather than notional.

Triggers are commitments about attention, not automation#

None of T1 through T5 fires a migration. Each fires a review with a named decider and a ten-day clock, which is the realistic unit, because the decision to move will always depend on things the trigger cannot see — a fundraise in progress, a customer commitment, a competing project. What the trigger buys is that the question gets asked by the calendar rather than by an incident, and that it lands on a specific person rather than dissolving into a Slack thread.

The threshold values in the triggers should be set by someone who will be embarrassed if they are wrong in either direction. A rubric gap of 0.08 is a real number in this company's suite: it is roughly the gap at which the lawyers' blind preference exceeded 70/20, which is the level that correlated with the bake-off losses. A trigger set at a number nobody can justify is a trigger that will be argued with on the day it fires.

What this framework is not for#

It is not an argument for writing records about everything. A team that produces an ADR for each vendor-specific parameter will produce forty documents nobody reads, and the forty will hide the three that matter. The threshold worth applying is whether the dependency touches unit economics, sits in the critical path of the product's best feature, or has a reversal cost above about two engineer-weeks. Choosing one vendor's tool-call format for an internal admin tool clears none of those bars and should simply be done.

It is also not a way to make lock-in safe. The record does not stop the vendor from raising prices, changing cache TTLs, or retiring the model; it makes those events land on someone with a pre-computed cost and a rehearsed route out. A company that had all five triggers written and rehearsed would still have a bad quarter if T3 fired at the same moment as a fundraise, and the record's contribution is that the bad quarter would be a managed one.

And it does not resolve the case where the exit is genuinely closed. Some dependencies have no counterpart anywhere — a provider's server-side execution environment, a fine-tune on a base model with no equivalent, a data-residency guarantee that only one vendor holds in one region. For those, the honest record says the reversal cost is a rebuild rather than a migration, gives a number, and the decision to accept it is a real decision made in the open rather than a fact that accumulated.

The practical starting point is one file. Take whatever your system's largest provider-specific dependency is — most likely caching, structured output, or a tool-execution loop — and write the payoff you have measured, the trigger you would accept, and the reversal cost you have actually rehearsed. Where you cannot fill a field, that gap is the next piece of work: an unmeasured payoff means instrumenting the dashboard, an unrehearsed reversal means running a day against a second binding, and a missing owner means a name has to go in the registry. Then put the review date in a calendar, because a record with no date on it becomes, within a year, exactly the kind of undocumented fact it was written to prevent.