Settling a conformance dispute

Prevalence looks like evidence and is not. Two clients that agree about a protocol detail are frequently one implementation observed twice, because most MCP clients are built on a small number of shared SDKs — the official TypeScript and Python packages between them account for a large share of what you will ever be connected to. A vote among clients is a vote among SDKs, and there are far fewer of those than there are products.

That matters because the dispute this chapter is about has a predictable shape. A vendor's support engineer asserts conformance and headcount, you assert conformance and eleven months of working deployments, and neither claim is checkable. The way out is to stop trading assertions and produce artefacts that the other side can run without trusting you, which takes an afternoon and changes what kind of reply the thread gets.

Build the exchange the document requires, by hand#

Both parties in a dispute like this arrive holding logs, and it is worth working out why neither set helps before spending a day reading them. A log is written by the implementation whose behaviour is in question. What it records is what that implementation believes it did: the method it thinks it was asked for, the status it thinks it returned, the reason it thinks a request was bad. Every one of those entries has passed through the same reading of the specification that is under dispute. An implementation that has misread a clause does not log the misreading — it logs ordinary correct operation, because by its own lights that is what it is doing. A log from either side is therefore a faithful record of that side's beliefs and no evidence at all about the clause the two sides disagree over.

The assertion that usually travels with the logs has the same defect and is easier to miss, because it is phrased in the language of fact. "Our client is spec-compliant" is a claim about a party, made by that party, concerning precisely the property in dispute. It is not dishonest and it is not usable: the engineer who writes it believes it for the same reason their log says what it says. Two things follow. Piling up further logs and further assertions cannot break the tie however many arrive, since all of them are drawn from the same two readings. And evidence that can break it has to originate outside both readings — the bytes themselves, captured where nothing interprets them; the specification text, which belongs to neither party; or an implementation written by somebody with no stake in the answer. The artefacts below are an attempt at those three, and the attempt is only partly successful: one of them is written by you, carries your reading inside it, and is worth listing with that limitation stated rather than hidden.

The first artefact starts with performing the minimal conforming session yourself, with no client and no inspector, so that every byte is one you chose and can point at.

BASE=https://orders.internal.example.com/mcp

# 1. initialize — no session header yet, none is expected
curl -sD headers.txt -X POST "$BASE" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{
        "protocolVersion":"2025-06-18",
        "capabilities":{},
        "clientInfo":{"name":"by-hand","version":"0"}}}'

SESSION=$(grep -i '^mcp-session-id:' headers.txt | cut -d' ' -f2 | tr -d '\r')

# 2. initialized — a notification, so an empty 202 is the correct answer
curl -si -X POST "$BASE" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -H "Mcp-Session-Id: $SESSION" \
  -H 'MCP-Protocol-Version: 2025-06-18' \
  -d '{"jsonrpc":"2.0","method":"notifications/initialized"}'

# 3. the first real request
curl -s -X POST "$BASE" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -H "Mcp-Session-Id: $SESSION" \
  -H 'MCP-Protocol-Version: 2025-06-18' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'

When that passes, it is worth being clear about what "it passes" establishes, because it is narrower than it feels: the server answers correctly for one particular spelling of the session header, one ordering of the handshake, and one set of headers, all of which you wrote while holding the same assumptions that produced the server. It is a necessary artefact and it is not the diagnosis.

The general shape of that limitation is worth carrying past this chapter. A specification describes a space of conforming sessions, and the by-hand exchange is one point sampled from that space by the same person who wrote the code being sampled. The points somebody picks are the ones they thought of, and the ones they thought of are precisely the ones their implementation already handles, which is what it means for an assumption to be invisible. Passing your own exchange is therefore necessary, in the sense that failing it would have ended the dispute immediately and against you. It is nowhere near sufficient, because the failing client is by definition standing somewhere you did not sample.

The diagnosis comes from the second artefact, which is the failing client's frames, captured verbatim rather than described. Ask for a HAR file — the archive a browser or a proxy can export containing every HTTP request and response verbatim, headers included — or put a frame-logging proxy of the kind chapter 3 builds in front of the server, or read your own access log with header capture turned on. Here is a request that 404s, with the headers as they arrived:

POST /mcp HTTP/1.1
content-type: application/json
accept: application/json, text/event-stream
mcp-session-id: 019a4f2c-7b13-7c1e-9e51-6a2f0c8d4411
mcp-protocol-version: 2025-06-18

{"jsonrpc":"2.0","id":2,"method":"tools/list"}

Everything in that request is valid. The session id is the one the server issued. The only difference from the by-hand version is the casing of the header names, and that is a difference only because the session middleware reads headers["Mcp-Session-Id"] out of a plain object rather than through a case-insensitive lookup, gets undefined, and returns 404 for an unknown session. HTTP header field names are case-insensitive — that is RFC 9110, not an MCP question at all — and clients built on a shared SDK send them in the casing that SDK happens to use, which is the casing the lookup was written against.

initialize succeeds in that session because it is the one request that carries no session header, so the middleware never runs its lookup. That is why the failure has the shape it has, and the shape is a fingerprint: a handshake that works followed by everything failing points at per-request state — the session, an auth header, a protocol-version check — rather than at any of your tool code.

Deriving that fingerprint takes one observation about the protocol, and the derivation is what makes it usable on failures other than this one. initialize is the only request permitted to arrive without a session, since it is the request that creates one. Every other request carries per-request state the server has to look up before it can dispatch: the session id, an authorisation header, a protocol-version header. A defect in the looking-up is therefore invisible to initialize by construction and fatal to everything after it. Three conclusions drop out of that single property at once. The transport is exonerated, because frames are plainly crossing in both directions and being parsed. The tool handlers are exonerated, because nothing ever reaches them. And the search narrows to the code that runs between a request arriving and a handler being chosen, which in most servers is a short stretch of middleware you can read in one sitting.

Three artefacts, three different jobs#

The third row is where these threads usually go wrong. "The spec says clients should use the session id" is a sentence two people can both agree with and continue disagreeing about. A quoted line, with the revision date it came from — the MCP specification is versioned by date, and 2025-06-18 and 2025-03-26 differ in ways that matter, including the removal of JSON-RPC batching and the addition of the MCP-Protocol-Version header — is a thing that can be checked rather than argued with. When the relevant rule is not in the MCP specification at all, as with header casing, cite the document it is in.

Sending all three at once also changes the conversation's shape. The vendor's engineer is not being obstinate; they have a support queue and a claim with no evidence attached, and the default response to that is a restatement of their own position. A message containing a reproduction they can run, a capture of their own client's traffic and a quoted clause is a different kind of object, and it tends to get a different kind of reply.

It is worth being exact about what a settled dispute settles, because the relief of settling one invites a second conclusion that does not follow from the first. The three artefacts together establish which side's behaviour departs from the document. They do not establish who changes their code. Whose defect it is decides who owes the fix in principle; who ships one is decided by which side can deploy this week, whose users are blocked in the meantime, and what a workaround would cost the party that did not cause the problem. Those are separate questions with separate answers, and a server author who has proved the client wrong and is still down a week later has won the argument without winning anything else.

When the client does something the specification permits, you are wrong#

A specification defines a space of permitted behaviour, and everything in that space is something your server has to handle whether or not anyone has ever sent it. Lowercase header names are permitted. So are string request ids, since JSON-RPC 2.0 allows "id": "req-88" as readily as "id": 88 — a server that stores pending requests in a map keyed by integer will match neither, and will produce a session where responses are never correlated. So are unknown fields inside params, which a conforming implementation ignores rather than rejecting, so a strict schema validator on your inbound frames will reject a client that adds a _meta field the next revision introduced. So is a client that sends notifications/initialized and then waits several seconds before its first request, or one that opens the server-to-client stream before its first tool call rather than after.

The headcount argument fails for a reason worth spelling out, because "two of my three clients work" does feel like data. Conformance is a relation between one implementation and a document. Working is a relation between two implementations. A client that accepts more than the document requires — that tolerates a missing field, an unexpected casing, a stale protocol header — will report success against a server that honours the clause and against one that violates it, so its success discriminates between nothing. Counting a hundred such clients counts the same non-discrimination a hundred times over. What a majority of working clients establishes is that the majority is permissive, which is a fact about them rather than about you. It takes one client to establish that a server depends on something the document never promised, and that one is the client that broke.

"Nobody sends that" is a statement about the clients you have met. The useful reflex when a report like this lands is to ask which of your server's behaviours depend on an assumption the document does not license, and the honest answer is usually that you have never enumerated them. Reading the transport section of the specification with your session and framing code open beside it takes an hour and typically produces two or three items, most of which are one-line fixes.

The reverse case exists and is decided the same way. If a client omitted notifications/initialized entirely and went straight to tools/list, that would violate a requirement stated with a MUST, which is the word specifications reserve for behaviour an implementation is obliged to exhibit rather than merely allowed to, and the correct response is to quote it, decline the patch, and be prepared for the vendor to take a while. The asymmetry is not about who is bigger; it is that permitted behaviour obliges you and required behaviour obliges them.

That asymmetry is not an arbitrary convention either, and it comes out of what a specification is for. Its purpose is to let two implementations that have never met interoperate, which requires each one to be able to predict what the other will accept. A MUST is a promise every implementation makes to every other, so a client that breaks one has broken the prediction, and nothing you could have written would have anticipated it. A permission is the opposite kind of statement: it announces that a behaviour is available, which means any implementation may exercise it at any moment without telling anyone first. Building on the assumption that nobody will exercise a permission is building on a promise the document never made. The two halves are one rule seen from its two ends — you are obliged to handle everything the document allows to be sent to you, and entitled to assume only what the document requires to be sent to you.

A dispute is worth one test each time#

The by-hand exchange has a longer life than the argument that produced it. Once written, it is a fixture: a sequence of raw frames and headers, sent in order, with expected status codes and response shapes. Keeping it in the repository next to the inspector fixtures from chapter 2, on driving a server directly, costs nothing and means the next dispute starts from a suite rather than from a terminal.

The value comes from adding a case for every permitted-but-unexpected thing anyone has ever caught you on. The header-casing case goes in with lowercase names. A case with "id": "req-88" goes in beside the one with an integer id. A case with an extra _meta object inside params checks that you ignore unknown fields instead of rejecting the request. A case that waits three seconds between notifications/initialized and the first call checks nothing times out in between. Each is a few lines of the same shape, and each encodes one assumption someone in the industry has already violated.

for f in conformance/*.http; do
  ./replay.sh "$BASE" "$f" > "out/$(basename "$f").json"
  jq -e -f "conformance/$(basename "$f" .http).jq" "out/$(basename "$f").json" \
    || { echo "FAIL $f"; exit 1; }
done

What this suite does that unit tests do not is exercise the layers below your handlers — the header parsing, the session lookup, the framing, the status codes — which is exactly where every defect in this chapter lived. A handler test constructs a request object that your own code shaped, so it inherits your assumptions by construction; a frame replay sends bytes. It is also the artefact to attach when you are the one asserting conformance: "our server passes these fourteen raw exchanges, here they are, run them yourself" is a claim a vendor can verify in five minutes without trusting you.

The two-hour workaround needs a name on it#

Suppose the diagnosis had gone the other way and Client C really were violating a MUST. You can ship a two-hour workaround today that unblocks every affected user, or you can decline and wait on a vendor whose last three fixes took between six weeks and never. Both are defensible; the shallow version of this decision is the one where somebody ships the workaround as a temporary concession to someone else's bug and then nobody removes it.

What makes that outcome likely is that the workaround arrives with no owner and no exit. Two years on it is a branch in the session handler with a comment saying "Client C sends these lowercase" and no record of whether Client C still does, and every engineer who reads it treats it as load-bearing because they cannot prove otherwise. The server has accumulated a behaviour nobody can justify from the specification, which is the precise condition that makes the next conformance dispute unwinnable.

Shipping it defensibly costs about twenty extra minutes. Scope the workaround to the client that needs it, keyed off clientInfo.name from the handshake rather than applied globally, so its blast radius is one implementation. Put the specification clause in the comment, along with the vendor's ticket number and the version you expect the fix in. Add a test that asserts the workaround is still being triggered — one that fails when the condition stops occurring, which converts "is this still needed" from an archaeology problem into a red build. Give it a named owner and a review date in the same tracker everyone else's work lives in, and log a counter each time it fires so that "how many sessions still need this" is a number on a dashboard.

Declining is equally defensible when the workaround would compromise something. A concession that requires relaxing validation for everyone, or that makes two clients see different behaviour in a way your tools cannot distinguish, buys a week of goodwill at the cost of a permanent ambiguity in what your server does. The decision that is not defensible is the one made without stating which of these applies.

Prevalence is a claim about a population and conformance is a claim about a document, and only one of them can be checked. Three artefacts convert the argument into something checkable: an exchange anyone can run, a capture nobody can dispute, and a clause with a date on it. Producing them usually settles which side is wrong before the vendor replies, and often the answer is that your server depended on something the specification never promised.

That whole procedure, though, rests on the failure being reproducible. Every method in this topic so far has assumed you can make the thing happen — in the agent, in an inspector, in a frame log, or by hand with curl. The case that defeats all of them is a tool that fails roughly one run in twenty, always through the agent, never once when you call it yourself.