The model abstraction boundary

The shared core and per-model adapters of the previous chapter invite an obvious extension: if adapters can absorb markup, turn placement and control parameters, why not push until no model-specific text exists anywhere and application code never branches on a model identifier? That is the shape of the platform-team proposal that follows a painful migration, and most of it is right. The task, the policy, the domain vocabulary and the output contract should exist once, and application code should not be branching on a model name.

The part that breaks is the claim that every difference is a difference in how you say things. Some are differences in what the model can be made to do, and those cannot be translated — only papered over. Knowing which is which is the whole of this chapter, and the test is a single question: does the difference change what the system can guarantee?

Two kinds of difference, and only one of them translates#

Markup does not change what can be guaranteed: a correctly rendered core says the same thing in both dialects, and the authority measurement from earlier in this topic is what tells you the rendering is faithful. A schema-constrained decoder does, because with it there is a class of output that cannot occur and without it there is not, and no amount of adapter code manufactures that class boundary.

The split is not a matter of taste, and the reason comes from what an interface is. An interface is a contract every implementation behind it has to satisfy. A guarantee that only some implementations can make therefore cannot be part of that contract, because the rest would be unable to implement it — so the guarantees a uniform interface offers are the intersection of the guarantees its members offer, never the union. Two things follow immediately. Adding a model can only shrink that set and never grow it. And the shrinking is silent, because nothing about the interface changes when a member with fewer capabilities joins: the same function signatures, the same return types, a smaller guarantee behind them.

The capability worth refusing to hide#

Native tool calling is the one worth naming, because the pressure to unify it is strong and the cost of unifying it is measurable.

A uniform interface has to serve the weakest member, so if one of three models has no native tool API, the interface either drops tools or expresses them as a text protocol that every model gets:

When you need to use a tool, output exactly:

ACTION: lookup_order
INPUT: {"order_id": "..."}

Then stop and wait. Do not output anything else on those lines.

Running that protocol on two models that had native tool calling, against the same frozen 600-conversation set:

Nearly six points of tool-selection accuracy and a 2.3% parse failure rate, in exchange for a call site that does not know which model it is talking to. That trade is not obviously wrong for every product, and it is obviously wrong for an agent whose value is in calling the right tool. What makes it dangerous is that nothing in the codebase records it: six months later the protocol is just how tools work here, the 93.4% is the baseline, and an engineer investigating tool-selection complaints has no reason to suspect the abstraction layer.

The top two rows come from one difference, and it is the same difference the chapter on decoding guarantees turned on. On the native path the tool name is drawn from a closed set the provider knows about and the arguments are produced under the tool's schema, so a name outside that set and a payload that does not fit the schema are not outputs the model can reach — which is why the parse row reads 100% and is annotated as enforced rather than measured. On the text protocol the tool name and the JSON are ordinary output tokens like any others. Nothing stands between the model and lookup_orders where lookup_order was defined, or a stray sentence between the two lines, or a trailing comma. The protocol asks the model to be reliable. The native path removes the alternatives.

The same difference accounts for the selection row, which surprises people more, since choosing a tool feels like a reasoning task rather than a formatting one. Choosing from a native definition is a selection among named options presented as options. Choosing from a protocol described in prose is a text-generation task that happens to have a right answer, and text generation offers many ways to be nearly right. A nearly right tool name is a wrong tool call.

The 85 extra output tokens per call are worth converting into money before anyone calls them a rounding error. On a support agent's 1.4 million tool-using turns a month, that is 119 million output tokens, which at $15 per million is $1,785 a month spent restating a protocol that two of the three models did not need. A uniform interface proposed to save engineering time has a running cost, payable monthly, that nobody puts in the proposal.

The chain is two multiplications. Roughly 85 extra tokens on each of 1.4 million tool-using turns is about 119 million tokens, and 119 million is 119 lots of a million, so at $15 per million the bill is $1,785. Worth noticing which rate that is: these are output tokens, typically the dearer of the two a provider quotes, because the model has to write the protocol restatement out. The same overhead expressed once in the prompt would have been input tokens and would have cost a fraction of it.

Emulation in the other direction is worse. Wrapping the weak model's text protocol in a fake native-tools interface means the call site sees tool_calls on the response object and reasonably assumes the schema held, when in fact 2.3% of the time the adapter parsed something approximate out of prose and filled in the gaps. That converts a visible incompatibility into a silent data-quality problem, which is the general shape of what goes wrong when a capability is hidden.

The reframe worth carrying away is that the shape of a return value is a claim. A response object with a tool_calls field says: this came out of a mechanism that guarantees these are tool calls. Every caller reads it that way, because that is what the field is for and there is nothing else to read it as. A wrapper filling the same field from a regex over prose has not translated a capability. It has made the claim without the mechanism, and the callers go on believing the claim. That is why this arrangement is worse than having no tools on that model at all: the honest version fails where the tool is called, and this one succeeds with a value somebody guessed at.

Declare capabilities, require them, and fail at configuration time#

The workable interface keeps the shared core and adds capabilities as first-class, declared per model and checkable by the caller. A call site that needs a guarantee asks for it, and a model that cannot supply it fails when the configuration is loaded rather than degrading quietly at three in the morning.

class Capability(StrEnum):
    STRICT_SCHEMA   = "strict_schema"
    NATIVE_TOOLS    = "native_tools"
    PARALLEL_TOOLS  = "parallel_tools"
    ASSISTANT_PREFILL = "assistant_prefill"
    REASONING_EFFORT  = "reasoning_effort"
    PROMPT_CACHE      = "prompt_cache"

REGISTRY: dict[str, frozenset[Capability]] = {
    "family-a-v4": frozenset({STRICT_SCHEMA, NATIVE_TOOLS, PARALLEL_TOOLS, PROMPT_CACHE}),
    "family-b-v2": frozenset({NATIVE_TOOLS, ASSISTANT_PREFILL, PROMPT_CACHE}),
    "family-c-v2": frozenset({REASONING_EFFORT}),
}

@dataclass(frozen=True)
class UseCase:
    core: Core
    requires: frozenset[Capability]
    degradation: Literal["forbid", "documented"] = "forbid"

INVOICE_EXTRACTION = UseCase(core=EXTRACT_CORE,
                             requires=frozenset({Capability.STRICT_SCHEMA}))

def bind(use_case: UseCase, model: str) -> Adapter:
    missing = use_case.requires - REGISTRY[model]
    if missing and use_case.degradation == "forbid":
        raise IncompatibleModel(f"{model} lacks {sorted(missing)} for this use case")
    return ADAPTERS[model]

The value of requires is that it makes the assumption explicit at the point where it was made. An invoice extractor depends on strict schema decoding not as a preference but as the thing standing between a wrong number and the ledger, and writing that down means the person who later routes extraction to the cheap failover model gets an exception instead of a 0.75% silent-error rate.

Two separate things are being bought there. One is where the record sits: requires is written at the definition of the use case, which is where the person who knew about the assumption was standing when they made it. An assumption recorded anywhere else is an assumption that has to be found before it can be honoured. The other is when the failure happens. Binding runs as configuration loads, so an incompatible pairing fails on deploy, with somebody watching, a stack trace naming the missing capability, and nothing yet in production. The same mismatch caught at call time fails on live traffic at whatever hour the failover triggered. The silent version does not fail at all — it writes wrong numbers into a ledger and surfaces in a reconciliation months later, by which point the mismatch and the symptom have no visible connection.

degradation: "documented" exists for the cases where running without a capability is acceptable — a summarisation path that would like prompt caching and functions without it. Choosing that value is a decision someone makes in a code review, which is the difference between a considered trade and a default.

Measuring whether the boundary is in the right place#

The boundary is an engineering judgement and it is falsifiable, which means it comes with numbers rather than with opinions.

The first is the capability tax. For each model and each use case, score the abstraction's output against a hand-written direct call that uses everything the provider offers, on the same frozen set. A gap under about one point is an abstraction that costs nothing worth arguing about; the 5.7 points in the tool-calling table is a boundary drawn in the wrong place. Running this quarterly is enough, and it is the only measurement that answers the question directly.

The threshold there is neither arbitrary nor precise. A frozen-set comparison has a spread, and on sets of the size this topic has been working with, a gap of under a point sits inside that spread — so it is not a measurement of the abstraction at all, it is a measurement of the noise. Five and seven tenths of a point is outside any spread six hundred conversations produce, which is what makes it a finding rather than a reading. The rule underneath the number is to quote the gap alongside the spread established for that set, and to treat anything the spread contains as zero.

The second is adapter size and its distribution. The 25-line budget from the previous chapter doubles as a boundary alarm: an adapter that has grown past it is absorbing something structural, and the growth shows up before the quality does.

The third is the count of workaround sites — places in application code that special-case behaviour the interface was supposed to make uniform, however obliquely. A grep for retry logic that exists only because one model's output needs repairing tells you the abstraction is leaking even when nobody has filed a ticket about it.

The fourth is a probe suite that tests the registry against the API rather than against the documentation. For each declared capability there is one small request whose failure is unambiguous: a schema with an enum the model could not satisfy by luck, a tool definition with a required argument, a two-token prefill, a request setting a reasoning-effort value. Twelve models by six capabilities is seventy-two requests costing a few cents, run weekly in CI, and it fails the build when a model's behaviour and its registry entry disagree in either direction. The direction people forget is the pleasant one: a probe that starts passing on a model whose entry says the capability is absent means the provider shipped something, and the adapter carrying a workaround can lose it.

Seventy-two is the two counts multiplied, because every model has to be asked about every capability separately: a model's answer on one says nothing about its answer on another. Spending those cents is worth it because a registry entry and a documentation page are different kinds of claim. Documentation states intent. It is written before release, and it is not revised when a rollout is partial, when a feature is limited to some regions, or when a model version is retired behind an alias. A probe states behaviour, on the endpoint you actually call, today. Only one of the two can be wrong for months without anybody finding out.

The fifth is the metric the proposal is actually optimising, which is time to add or swap a model, and it should be measured on both sides of the change. If a boundary that keeps native tool calling exposed turns a three-week migration into a five-day one while a fully uniform interface would make it three days, the two extra days are the price of six points of tool accuracy, and that is a sentence a team can evaluate.

A registry goes stale, and model names still do not belong in services#

None of this is a defence of model-specific text scattered through services, which is the problem the RFC was written to solve and which it does solve. The disagreement is about placement rather than about whether the abstraction should exist: capability differences belong in one declared, reviewable place, and the failure mode of the strong version is that they end up in no place at all.

It is also worth being honest that the capability registry is a maintenance burden that will go stale, because providers ship features continuously and nobody's job is to notice. A registry claiming a model lacks strict schema support six months after the provider added it is a boundary drawn around a fact that is no longer true, and it costs you the guarantee you were trying to protect. The probe suite is the mitigation and it is a partial one: it tests the capabilities you already thought to declare, and a capability nobody has an enum member for is invisible to it.

The registry is also the wrong instrument for a difference that is neither wording nor capability but degree — a model that supports native tool calling and is simply worse at choosing among six tools. Nothing in REGISTRY expresses that, and nothing should; it belongs in the divergence register and in the per-model measurements, where a number can sit beside it. The boundary this chapter draws separates what an adapter may hide from what it may not, and it says nothing about quality, which remains something you measure per model per prompt.

That staleness is a mild version of a sharper problem, which is that everything in this topic so far has assumed the model behind an identifier stays the same model. It does not, and the day it changes underneath you, no code was deployed and no adapter was edited.