Per-call overhead of a protocol boundary

A protocol boundary costs something on every call that crosses it, and the number is usually argued about rather than measured. Measuring it requires excluding two things that would otherwise swamp the result. The model has to go, because a model call is between five hundred and five thousand times more expensive than either path and would hide the entire question. Session setup has to go too, since the process spawn and handshake are paid once per connection and charging them against a single call would answer a different question. What is left is the thing that repeats.

The measurement below uses one SQLite file, one query, and one laptop, comparing a Python function called in-process against the same function exposed by a stdio MCP server — one where the client launches the server as a child process and speaks newline-delimited JSON-RPC over its pipes. Nothing about the work changes between the two paths: the same LIKE, the same index, the same twenty rows.

Two numbers from the same laptop#

import statistics, time
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from server import search_tickets            # the same function the server wraps

def bench(fn, n=1000):
    samples = []
    for _ in range(n):
        t = time.perf_counter()
        fn()
        samples.append((time.perf_counter() - t) * 1000)
    samples.sort()
    return statistics.median(samples), samples[int(0.95 * n)]

async def main():
    direct = bench(lambda: search_tickets("refund", limit=10))
    async with stdio_client(StdioServerParameters(command="python", args=["server.py"])) as (r, w):
        async with ClientSession(r, w) as session:
            await session.initialize()
            await session.list_tools()                    # warm; excluded from timing
            await session.call_tool("search_tickets", {"query": "refund", "limit": 10})
            over_mcp = await abench(
                lambda: session.call_tool("search_tickets", {"query": "refund", "limit": 10})
            )
    print(f"direct   p50 {direct[0]:.1f} ms  p95 {direct[1]:.1f} ms")
    print(f"over mcp p50 {over_mcp[0]:.1f} ms  p95 {over_mcp[1]:.1f} ms")

direct   p50  4.1 ms  p95  5.2 ms
over mcp p50 31.4 ms  p95 47.9 ms

Three details of that harness are what make the comparison fair rather than rhetorical. The session is initialised and its tools listed before timing begins, so the 234 ms of setup is charged to neither number and the measurement is of steady state. One call is made and discarded before the loop, because the first invocation pays for an import the server deferred and a page cache that is still cold. And the statistic reported is a median over a thousand samples rather than a mean, since a single garbage collection pause in either process moves a mean by several milliseconds and tells you nothing about what a typical call costs. Running the whole thing again on a loaded machine shifts both numbers upward by three or four milliseconds and leaves the difference between them essentially where it was, which is the sign that the difference is structural rather than an artefact of the afternoon.

The tail is the second finding and gets less attention than it deserves. The two figures printed for each path are the p50, which is the median call, and the p95, which is the time 95% of calls come in under and therefore describes the slow ones a user notices. The direct call's p95 sits 1.1 ms above its median because the only variance is the database. The MCP path's p95 is 16.5 ms above its median, four times the entire cost of the query, because there are now two event loops, two garbage collectors and an operating system scheduler between the caller and the work. A slow call is no longer a slow query; it can be a slow anything.

Reading those two spreads as a subtraction rather than as two p95 figures is what makes the comparison mean something, and the quantity being compared is the variability the boundary adds rather than the time it adds. On the direct path the spread is what one process doing one thing looks like when the only thing that can vary is a database file that is usually warm. On the MCP path the spread is larger than the entire piece of work being done, which is the uncomfortable part. It follows from what was added rather than being a surprise: two runtimes may each pause for garbage collection, two processes must each be scheduled onto a core by an operating system with other commitments, and the call cannot complete until both have had their turn. A stall in either one delays the result, so the combined tail is worse than either tail on its own — which is a general property of putting two independent sources of delay in series, and the reason adding a hop degrades a p95 more than it degrades a median.

Where the twenty-seven milliseconds actually goes#

The intuition most people have is that the cost is the pipe, and the pipe is almost free. Instrumenting both processes gives a very different distribution:

Before reading any row of that table, it is worth checking the two things that make it trustworthy, because a breakdown that does not reconcile with the measurement it explains is a list of guesses. The first check is that the stages sum to the overhead total on the bottom row. Add the eight non-italic lines — serialise, build and validate, pipe, parse and validate, dispatch and coerce, build and serialise the result, parse the result, schedule — and they come to the stated total with nothing left over. There is no "other" line, which matters: an unexplained remainder would mean some cost had been located nowhere, and a remainder is exactly where a wrong conclusion hides.

The second check is the one that closes the argument, and it uses an instrument the table does not. The query line is set in italics because it is not overhead; it is the work, and it is the same work the direct path did. So the overhead total plus the query ought to reproduce the end-to-end median measured in the previous section by a stopwatch that knew nothing about stages. It does, to within a few tenths of a millisecond. Two independent routes to the same number — one an outside measurement of the whole call, the other an inside accounting of its parts — agreeing is what licenses every claim made below about where to optimise. Without that agreement the table would only be a hypothesis about where the time went.

Moving bytes between two processes on the same machine costs 0.2 ms. Everything else is the SDK doing what an SDK does: constructing typed objects, validating them against schemas, running them through an async runtime that yields control at each await. Roughly 13 ms of the 27 is model construction and validation performed twice on the way out and twice on the way back, and another 6.5 ms is the cost of being asynchronous at all.

Those two subtotals are additions over the same table and they are worth doing, because which rows go into which is the whole basis of the optimisation advice that follows. The validation subtotal is the client building and validating its request, the server parsing and validating it, and the client parsing and validating the result — three rows, and their sum is the figure quoted. The scheduling subtotal is one row on its own. Add those two and subtract from the overhead total, and what is left is the honest remainder: turning arguments into JSON, moving the bytes, coercing arguments into typed parameters, and serialising the result. That remainder is small enough that no amount of work on it changes the answer, which is why what follows is about validation and language rather than about buffers.

The same table read by process rather than by kind gives one more subtotal, and it is the one the optimisation advice turns on. Three rows happen inside the server — parsing and validating the request, coercing arguments, building and serialising the result — and their sum is the server-side share of the overhead. That is the share a compiled implementation attacks, because it is the share made of dynamic object construction, and it is why the language the server is written in matters more to this number than the transport does.

That breakdown changes which optimisations are worth trying. Switching transports does almost nothing, because the transport was 0.2 ms; a local HTTP transport measures 33 ms on the same harness, marginally worse than stdio rather than better. Payload size does matter, since the 5 ms of result serialisation is a function of the 6 KB going over, and a tool that returns 200 KB of rows will spend most of its budget there. Language matters more than anything else: a Go or Rust server, where deserialisation and validation are not building dynamic objects, brings the server-side half from about 11 ms down to around 1 ms, which is why the same measurement run against a Go implementation of the identical tool comes out near 16 ms rather than 31 ms. The 27 ms is not a property of MCP, it is a property of MCP as implemented in Python SDKs in 2025, and it is the number most teams will actually observe because most teams will use those SDKs.

The Go figure is an end-to-end measurement of a different implementation rather than the Python figure with the server-side share subtracted out, and it is worth resisting the temptation to reconcile the two by arithmetic. Changing the language changes rows the table attributes to the client as well — the client is still waiting on an event loop, but it is waiting on a peer that no longer yields at every await — and it changes how the two processes interleave, which is not a line item anywhere. It is a second reading taken under different conditions, not a derivation from the first, and reading it as a derivation invites the reasonable question of why the subtraction does not come out — a question the table cannot answer, because the table describes only one of the two implementations. What both readings do agree on is the claim being made: the server's implementation language moves this number substantially, and the transport does not move it at all.

Concurrency helps less than it looks, and the reason is in the same breakdown. A large share of the 27 ms is CPU spent serialising and validating in your own process, so firing forty calls concurrently over one session does not divide the cost by forty; it overlaps the 0.2 ms of pipe time and leaves the rest competing for the same interpreter. Measured on the same harness, forty concurrent calls come in around 780 ms against 1,256 ms sequential, an improvement worth having and nothing like the order of magnitude people expect.

The baseline that makes the number mean something#

Comparing 31 ms against 4 ms produces the sentence "MCP made my tool eight times slower", which is arithmetically true and operationally useless. A tool call does not happen on its own. It happens inside an agent turn, and the turn is dominated by a model call that costs between 1.5 and 4 seconds for a request carrying a few thousand tokens of tools and context, with time to first token — the wait between sending the request and the first word of the answer arriving — typically 400 to 900 ms on its own.

Every cell in that table is one of the two medians multiplied by a count, and doing the multiplications is what turns it from a summary into something you can extend to your own call pattern. The first row is one call, so it is the medians unchanged, expressed as a share of a three-second turn: four milliseconds against three thousand is about an eighth of a percent, and thirty-one against three thousand is about one percent. The second row is three calls across three turns, so both columns are their medians tripled and the denominator is three turns of three seconds. The third row is forty calls with no turn around them at all, so both columns are their medians times forty, and the denominator disappears because there is nothing left to hide inside.

That third row is where the two figures quoted in this chapter's opening come from, and they are the same subtraction and the same division done twice. Forty direct calls is the work; forty calls over the boundary is the work plus forty lots of overhead; the difference between the two columns is therefore forty times the per-call overhead, and dividing that difference by the larger column gives the share the boundary occupies. Notice that the ratio between the two columns in that row is identical to the ratio between the two medians in the first row, because multiplying both by forty cannot change a ratio. So the eight-fold regression named at the top of this section and the sevenfold one named below it are one number, a little under eight, rounded upward where it is doing the work of making the boundary look expensive and downward where it is describing an endpoint. When a ratio turns up twice with different roundings, confirming it is the same ratio takes two seconds and saves reading a second finding into it.

The first two rows are the case where the answer is that nobody will ever notice. Adding 27 ms to a turn that takes three seconds is a one percent regression that no user perceives and no dashboard flags, and trading it for the reuse property — one server, many clients, no per-client adapters — is straightforwardly a good deal.

The third row is a different system. When a tool is called in a loop by your own code rather than by the model — enriching forty search hits, checking forty ids against a registry, fetching forty documents to rank — there is no model call for the overhead to hide behind. The work is 164 ms and the protocol is 1,092 ms, so the boundary is now 87% of the elapsed time and the endpoint is seven times slower than it was. This is the case where the honest engineering answer is to not put that capability behind a protocol at all, or at minimum to expose a batch tool that takes forty ids in one call and turns forty round trips into one, which brings the 1,092 ms of overhead back to about 40 ms and makes the payload the only thing that grew.

Why batching works so completely here, when it is often only a modest win, is visible in the stage table and nowhere else. Almost every line in that table is charged once per message rather than once per item: the request object is built and validated once, the JSON is parsed once, the event loops are scheduled once. Collapsing forty messages into one therefore removes thirty-nine copies of all of those, and what remains is one copy plus the one line that genuinely does scale with the amount of data — serialising a result that is now forty times larger. The overhead stops being proportional to the number of ids and becomes proportional to the number of round trips, and the number of round trips is now one. That is also the test for whether batching will help any particular boundary: if the dominant cost is per message, it collapses, and if the dominant cost is per byte, it does not.

Latency is also only one of the three things the comparison should cover, and the other two move in the same direction. Setup time for the direct integration is zero, because there is nothing to set up beyond an import; for the stdio server it is the 234 ms per session, and for an HTTP deployment it is a connection, a TLS handshake and the same two round trips, plus whatever your infrastructure charges for a cold instance. Implementation surface is the one that shows up in a diff: the direct version was twelve lines of glue registering a function in a dispatch table, and the server version is a package with an entry point, a transport, a declared schema, a deployment manifest and a version number, none of which is difficult and all of which is now yours to keep alive. Measuring all three is the point of the exercise, since a team that only measures latency will conclude that 27 ms is nothing and miss that they have acquired a deployable.

Two implementations of one capability#

That fourth cost only appears later, and it is the one that bites. The function and the server can drift. Version skew between a client pinned to one protocol revision and a server that moved on is the visible form; the more damaging form is two copies of the contract-search logic, one in the function the original agent still calls and one in the server everybody else calls. From the outside it is one capability with one name, which is exactly why nobody thinks to ask whether the two paths are the same code when they start returning different result sets.

The number itself also has a scope worth stating, because it is easy to quote out of it. 27 ms is a Python-to-Python stdio measurement on one laptop with a 6 KB result. A Go server halves it, a 200 KB payload inflates it, a remote HTTP deployment adds a network round trip that dwarfs it, and a batch interface removes forty of them at once. What survives all those variations is the shape rather than the value: the boundary costs a fixed amount per call, dominated by serialisation and validation rather than by transport, and it is invisible next to a model call and decisive next to none.

Which is why the two comparisons in this chapter give opposite answers and both are correct. Against the 4 ms of work, the protocol is an eight-fold regression and looks indefensible. Against the three-second turn the call actually sits in, it is one percent and looks free. The number that decides is neither of those: it is how many times the tool is called between model calls, because that is what multiplies 27 ms into something a user can feel.