The size of the shared contract

The previous chapter's seam had one method per port, and adding a capability there means adding a port: a new interface, one implementation for the vendor that has the feature, and one line at the composition root. This chapter is about the other arrangement — one LLMClient that every vendor implements and every caller holds — and specifically about what each method added to it costs after it is added. The arithmetic is the whole subject, because the individual decisions are not where the failure lives.

Here is such an interface, at fourteen methods:

export interface LLMClient {
  chat(req: ChatRequest): Promise<ChatResponse>;
  chatStream(req: ChatRequest): AsyncIterable<ChatEvent>;
  chatStructured<T>(req: ChatRequest, schema: JsonSchema): Promise<T>;
  embed(texts: string[]): Promise<number[][]>;
  rerank(query: string, docs: string[]): Promise<Ranked[]>;
  countTokens(messages: Message[]): Promise<number>;
  listModels(): Promise<ModelInfo[]>;
  moderate(text: string): Promise<ModerationResult>;
  transcribe(audio: Blob): Promise<string>;
  describeImage(image: Blob, prompt: string): Promise<string>;
  createBatch(reqs: ChatRequest[]): Promise<BatchHandle>;
  getBatch(handle: BatchHandle): Promise<BatchStatus>;
  cachePrefix(messages: Message[]): Promise<CacheHandle>;
  estimateCost(req: ChatRequest): Promise<Cents>;
}

Nothing in that list is unreasonable on its own. Each method arrived attached to a real requirement: countTokens because a support tool had to reject oversized attachments before sending them, createBatch and getBatch because a nightly classification job halved its bill on asynchronous batch pricing, where requests are submitted as a job that completes within hours in exchange for roughly half the per-token rate, cachePrefix because a long system prompt was being re-billed on every turn, estimateCost because finance wanted per-request attribution. Prompt caching, the feature behind the fourth of those, has the provider store a prefix of your request and bill later calls that reuse it at a fraction of the input rate, with a cache breakpoint marking how far along the prompt the stored prefix should run. Every one of those was a good ticket. The interface got to fourteen methods without a single bad decision, which is what makes this failure mode worth studying — nobody chose it, and everybody built it.

The cost of a contract is multiplied, not added#

Adding a method to a per-capability port costs a file. Adding a method to LLMClient costs one signature, three method bodies, three sets of unit tests, one update to the in-memory fake, and whatever the forty tests using that fake need in order to keep compiling. One team measured this without meaning to: the last four methods added to their LLMClient averaged 590 lines of diff and nine days between opening and merge, against a median of 80 lines and one day for everything else in the repository. That ratio is the interface charging rent on every capability it has taken responsibility for.

Those two figures are worth setting side by side, because the ratio is the finding. Five hundred and ninety lines against a median of eighty is a little over seven times the diff, and nine days against one is nine times the elapsed time — and elapsed time grew faster than size, which is what happens when a change needs several people's attention rather than one person's afternoon. The counting behind it is mechanical. One signature, then a body in each of the three implementations, then a test suite for each of those bodies, then the fake, then whatever the forty tests holding the fake need to keep compiling. Nothing in that list is optional and nothing in it is difficult.

The uncomfortable part is which term dominates. The per-method cost is roughly the number of implementations times the work per implementation, so it rises with the count of vendors the contract covers. A contract with one implementation is nearly free to extend; the same contract with three costs three times as much per capability; a fourth vendor makes every future capability a third more expensive again. The interface therefore gets more expensive exactly as it succeeds at the thing it was built for, which is why the cost is invisible at the point the design is chosen — on the day it has one implementation, it is cheap, and every measurement anybody has of it says so.

The multiplication also runs the other way, over time. Once a method exists, it constrains the vendors you can add. A fourth provider must supply fourteen behaviours or fourteen exceptions, and evaluating it now means checking fourteen boxes rather than the two your application actually cannot live without. An interface that started as a way to make vendors interchangeable has become a barrier to entry that you enforce against yourself.

The subtler damage is at the call sites. A caller that has to learn, at runtime, whether the client it was handed can actually perform a capability ends up reading a configuration flag inside a request handler, and that flag describes a property of the vendor. Domain code has begun making decisions on vendor identity again, through an interface built to prevent exactly that, and it does so because the interface promised a capability only one implementation can keep.

That last sentence is a chain rather than a complaint, and each link is forced. The interface declares that capability, so every caller holding an LLMClient may call it. Two of the three implementations cannot do it, so their bodies throw. Something therefore has to decide whether calling it is safe, and that decision cannot be made inside the adapter, because by the time the adapter is running the call has already happened. It has to be made by the caller. The only thing that distinguishes a client that can search from one that cannot is which vendor is behind it, so the caller reads the vendor. The abstraction did not fail to hide vendor identity through carelessness. It made hiding it impossible, by promising something that depends on it.

The belief this attacks is the reason the whole design gets built, and it is worth stating in its most persuasive form: a shared interface makes providers interchangeable. It is tempting because it is true of every shared interface most engineers have used — swapping Postgres for MySQL behind JDBC really does leave the calling code alone. What it misses is the direction of the dependency. An interface can only make interchangeable the things the implementations already agree about; where they disagree it has exactly two options, and both give up part of the claim. It can promise the weakest behaviour any vendor offers, in which case the vendors are interchangeable and the good one has been levelled down to the poor one. Or it can promise the strongest, in which case the callers must check who they are talking to, and the vendors were never interchangeable in the first place. Interchangeability is a property of the vendors. The interface can only report it.

Standards that worked were ratified, not invented#

The shape here is old and has a good precedent. JDBC and ODBC — Java's and Microsoft's standard interfaces for talking to any relational database — along with the POSIX file API, the set of calls Unix systems agreed on for reading and writing anything that behaves like a file, all did what LLMClient is trying to do: present many heterogeneous backends through one contract. They largely succeeded, and the circumstances of that success are specific.

They were written after the fact. SQL databases had existed for over a decade and the query-execute-fetch cycle was common across all of them before anybody standardised it, so the shared shape described observed agreement rather than predicting it. An internal LLMClient is written in the opposite direction — one team guessing, in the middle of a period where vendors are shipping genuinely novel capabilities every few months, which of those capabilities will converge.

Being written afterwards is not a point about process maturity. It is what makes the methods honest. A shared operation that describes something every backend already does is a report of a fact, checkable by looking at the backends, and its cost to a new implementation is nil, because that implementation was going to do it anyway. A shared operation that predicts convergence is a bet, and a losing bet does not fail loudly — it produces exactly the method-that-throws above, in the implementations that lost. So the same interface shape can be cheap or ruinous depending only on when it was written relative to the agreement it describes, which is why "we will standardise our own client" is a much riskier sentence in this decade than the JDBC precedent makes it sound.

They also shipped escape hatches on purpose, an escape hatch being a documented, sanctioned way to reach past the shared contract and hold the underlying vendor's own object when the contract does not cover what you need. JDBC has Connection.unwrap(), which hands you the vendor's own object and admits that the standard does not cover everything; drivers advertise capability through DatabaseMetaData rather than pretending uniformity; and every serious application eventually writes some vendor-specific SQL and knows it has. POSIX has ioctl, which is an admission in the shape of a function.

The third difference is governance. JDBC has a version number, a compatibility policy, and a change process slow enough that driver authors can follow it. An internal interface has none of those, so a change to it is simultaneously a change to the standard and to every implementation, landed in one pull request by whoever drew the short straw. That is why each new method feels like a breaking change: it is one, and there is no mechanism designed to absorb it. A contract that has taken on the reach of a standard while carrying none of the ratification, versioning or documented ways out that make a standard workable is a pseudo-standard.

Which capabilities actually belong in the shared shape#

Two questions decide it. Does the calling code branch on the capability — would your application logic look different depending on whether it is available? And can every vendor behind this contract implement it with the same meaning, not merely with a method of the same name? A capability needs both. If the caller does not branch on it, it belongs inside one adapter. If vendors cannot agree on its meaning, a shared method will paper over a difference that then shows up somewhere else.

Three rows are worth dwelling on. estimateCost is in the interface because it needed the model name and the token count, both of which the client had, so it got attached to the nearest object rather than to the concept it belongs to; pricing is a table you maintain, and a function over (model, input_tokens, output_tokens) needs no vendor client at all. cachePrefix is there because caching is a real feature with real savings, and the caller does not branch on it — a cached call and an uncached call return the same content and differ only in the bill, which makes caching an adapter's private business, expressed as configuration on the adapter rather than as a method every implementation must answer for.

The token-counting row shows what "shared, with a documented margin" costs, and it is worth pricing rather than waving at. The purpose of the count is to reject an oversized attachment before sending it, so the guard has to hold whatever the true count turns out to be. Tokenizers disagree by between two and eight percent, so a guard that must never let an oversized request through has to budget for the worst disagreement, which means reserving that much of the window and refusing requests that would in fact have fitted. The method is genuinely shared and the disagreement did not disappear; it was converted into headroom somebody has to decide on and write down. That conversion is the best available outcome for a capability the caller really does branch on, and it is still a loss, which is why the two-question filter puts most capabilities somewhere else.

What survives that filter is small. For most applications it is chat, embeddings, and possibly structured output, which is roughly where the previous chapter's two ports started. The narrow version is not less capable, since the vendor features are all still available; they are reachable through the adapter that has them rather than through a contract that pretends they are universal.

When normalizing everything is the right call#

There is a case where a fourteen-method contract is correct, and it is worth naming so the rule does not become a superstition. If normalization is your product — if you are building a gateway that other teams call, and the value you sell is that they do not have to know which vendor served the request — then the shared contract is the deliverable and maintaining it is the job rather than an overhead. Products like LiteLLM and OpenRouter exist precisely to absorb this cost on behalf of many users, and the openrouter.ai piece LLM Gateway: What It Is and How to Choose One is a clear statement of the job description: normalization, routing, failover, observability. The economics work at that scale because the cost is paid once and amortised over everyone who would otherwise pay it themselves.

The distinction is not about size or seriousness, and it is not a licence granted to companies above a certain headcount. It is about who the contract is for. When you normalize for yourself, the multiplied cost measured earlier falls on a team that gains nothing from the fourteenth method except the ability to have written it. When normalization is the product, that same multiplied cost is the thing customers are paying to avoid, so the term that was pure overhead becomes revenue. Same interface, same arithmetic, opposite sign — which is why reading a gateway vendor's engineering blog as advice for your own internal client is a reliable way to build the wrong thing.

The other honest exception is a genuine intersection. If your application really uses only the subset that all your vendors implement identically, a shared contract of that subset is not a pseudo-standard; it is an accurate description of what you do. The trouble is that this state is unstable, because the next requirement will not respect the intersection, and the interface will grow one reasonable method at a time in exactly the way this chapter opened with.

The discipline this asks for is unglamorous: when a capability arrives, decide whether the caller branches on it and whether every vendor means the same thing by it, and let the answer put it in the contract, in a separate port, inside one adapter, or behind a hatch. Applied once it saves a few hundred lines. Applied for a year it is the difference between an interface a fourth vendor can be evaluated against in an afternoon and one that has to be argued about for a quarter.

Keeping the contract narrow removes one whole category of pain, and it does not remove the other. A two-method port can still lose things, because the loss happens inside the types the contract does have rather than in the methods it lacks. A ChatResponse carrying text, tool calls and usage is a small honest-looking shape, and the next chapter follows what falls out of it on the way through — none of it raising an error, and all of it visible only as a metric moving in the wrong direction.