Dropping to raw HTTP

Provider capabilities arrive on the HTTP API before they arrive in the language clients, often by weeks and sometimes behind a beta header — an opt-in flag naming a capability the provider has released for testing but not yet made standard. Sooner or later you need one of those on a deadline, and the question becomes what to do about a client that has no parameter for it.

Three proposals show up every time: fork the client and add the parameter, patch the client at runtime so the right thing gets attached, or call that one endpoint over raw HTTP and leave everything else alone. Calling one endpoint over raw HTTP beats forking the whole client, and the rest of this chapter is the argument for why. This chapter weighs them against the same criteria, describes the supported extension points that make most of the argument unnecessary, and then works through what you take on when a raw call genuinely is the answer — which is precisely the content of the earlier chapters in this topic, and the reason they came first.

Weighing the three against the same criteria#

The forking proposal is the one that sounds most responsible, which is what makes it worth taking apart. It keeps a single code path, it produces a diff you can offer upstream, and it does not introduce a second way of talking to the provider. What it costs is ownership. A modern provider client is a few hundred files covering request construction, the retry and backoff policy, streaming decoders for every framing, pagination, typed models for every response shape, and per-endpoint method signatures — and once you fork it, all of that is yours. Every upstream release becomes a merge, and the merges are not small because these clients are largely generated, so a regeneration reflows files your patch touched.

That last point is the one people underestimate, and it behaves differently from ordinary merge pain. A hand-written library changes where somebody edited it, so a patch in a quiet corner stays quiet for years. A generated library is emitted whole from a machine-readable description of the API, so the tool that produces it rewrites its output every time anything in that description moves — including formatting, ordering and helper code that has nothing to do with the endpoint you touched. Your patch sits in a file that is regenerated, so it conflicts on schedule rather than on relevance. The size of the merge is a function of how often the vendor releases, not of how invasive your change was, which is why "it is only one header" is not the mitigation it sounds like.

You forked hundreds of files to add one header, and the empirical pattern is that the fork outlives the beta flag: the client ships the feature in six weeks, nobody notices because the fork works, and eighteen months later you are three minor versions behind on a client you cannot cleanly upgrade.

The runtime patch deserves its own note, because there is a legitimate version of it that is not a patch at all. Both clients let you set default headers at construction, and both accept per-call extra_headers, extra_body and extra_query arguments precisely so that features the typed signature does not know about can still be sent. That is a supported extension point, and if it covers your case it is better than everything else in the table, because there is no second code path at all.

Those arguments are not an accident of a permissive design, and understanding why they exist is what makes the ordering rule below obvious rather than arbitrary. The vendor knows the client lags the HTTP API, because the client is built after the endpoint and released on its own schedule. They also know what happens when callers have no sanctioned way to bridge the gap: people reach into private methods, and then a routine internal refactor breaks working production code and arrives as a support ticket. Providing an explicit hole in the typed surface is cheaper for the vendor than dealing with that, so extra_headers and extra_body come with the only thing that matters here — a promise that they will keep working.

That promise is the entire difference between the supported route and the monkeypatch, and it is worth stating in those terms because both look equally hacky in a diff. Both send a header the signature does not know about. One of them is a documented contract the vendor has committed to; the other depends on the current shape of a private method, which the vendor is free to change in a patch release and has no reason to warn you about. The code is similar. The guarantee is not, and the guarantee is what you are actually choosing between.

msg = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": prompt}],
    extra_headers={"anthropic-beta": "some-beta-flag-2025-08-07"},
    extra_body={"an_unshipped_parameter": {"enabled": True}},
)

Both clients also expose a generic escape hatch one level down — an untyped post that takes a path, a body and a target type — which reaches endpoints the client has no method for while still using its transport, its retries and its authentication. That is the smallest possible seam and it should be the first thing you try.

What people usually mean by monkeypatching is different: reaching into a private method such as the request builder, or mutating the underlying transport's header dictionary from module-level code at import time. Those work, and they work by depending on internals that carry no compatibility promise, so the next patch release can break them silently. The worse property is discoverability. The call site looks like an ordinary SDK call, so the next engineer who debugs a request has no reason to suspect that something in a different module altered it, and a capture of the outbound request becomes the only way to find out what is actually being sent.

The order in which to try things follows from that table, and it is short enough to run through in a meeting. Ask first whether the capability is a header or an extra body field, because if it is, extra_headers and extra_body cover it and there is no second path. Ask next whether it is an endpoint the client has no method for, because the untyped post covers that while keeping the transport. Only when the client would actively reject or mangle what you need — a different content type, a response shape its deserialiser cannot represent, a streaming framing it does not implement — does the raw call become necessary. Most requests that begin as "the SDK doesn't support it" turn out to stop at the first or second question, and the difference between stopping there and writing a raw path is a day of work and a permanent maintenance obligation.

The middle item on that list of blockers — a response shape the deserialiser cannot represent — is the one worth understanding properly, because it is the only one you cannot detect by reading your own code. The typed response models are generated from the same description the methods are, so a model lists the fields the description knew about and builds an object out of exactly those. Anything else in the JSON is stepped over during parsing, and nothing anywhere records that it was stepped over. So the object in your hands is not an incomplete copy of the response. It is a complete copy of the schema, which is a different thing, and no amount of inspecting it will reveal what the schema left out. This is why a new capability can be fully working on the wire and entirely absent from your program, with no error at any layer, and why the only test that settles it is one that reads the raw body.

What you take on when you leave the client#

Choosing the isolated raw call means writing a function that behaves, from the outside, exactly like the SDK method it replaces — same arguments, same return type, same exceptions — and reimplementing what the client was doing underneath.

import httpx, time, random

BASE = "https://api.anthropic.com/v1/messages"
HEADERS = {
    "x-api-key": os.environ["ANTHROPIC_API_KEY"],
    "anthropic-version": "2023-06-01",
    "anthropic-beta": "some-beta-flag-2025-08-07",
    "content-type": "application/json",
}

_pool = httpx.Client(timeout=httpx.Timeout(60.0, connect=5.0), http2=False)

def create_message(body: dict, max_attempts: int = 3) -> dict:
    for attempt in range(max_attempts):
        r = _pool.post(BASE, headers=HEADERS, json=body)
        log.info("provider_attempt", status=r.status_code,
                 request_id=r.headers.get("request-id"), attempt=attempt)
        bucket = classify(r.status_code, r.json())          # from the error chapter
        if bucket is Bucket.OK:
            return r.json()
        if bucket is not Bucket.RETRY or attempt == max_attempts - 1:
            raise ProviderError(bucket, r.status_code, r.json())
        delay = float(r.headers.get("retry-after", 0)) or min(8.0, 0.5 * 2 ** attempt)
        time.sleep(delay * (0.5 + random.random()))
    raise AssertionError("unreachable")

Five things in thirty lines, and each one is a thing the client was doing for you: the version and content-type headers that a naive replay omits, a connection pool with a connect timeout separate from the read timeout, per-attempt logging with the provider's request identifier, a retry policy that prefers the server's retry-after and jitters its own backoff, and a classification step that decides whether a second attempt has any chance.

Streaming adds a sixth, and it is where the work is concentrated, because a raw streaming call gives up the client's decoder entirely. What you get back is an SSE body: framed events on the wire, with nothing between them and your code. Everything the decoder used to do — reassembling text deltas, accumulating partial tool arguments until they parse, reading the stop reason and usage off the terminal event, and treating a stream that ended without that event as a failure — becomes explicit code. None of it is difficult once you have the framing in front of you, and all of it is easy to leave half-finished under deadline pressure, which is why a raw streaming path deserves a test that asserts on the terminal event rather than on the concatenated text.

You also give up the typed response models, so the function returns a dictionary where the SDK returned an object with attributes. That is worth fixing at the boundary rather than letting raw dictionaries spread: parse into whatever internal type your code already uses, so that callers cannot tell which path served them. The same applies to exceptions — raise your own error type carrying the bucket, the status and the body, and make the SDK path raise it too by translating the client's exceptions at the same boundary.

Two implementation details on this path cause more production trouble than the interesting parts. The client object must be created once and reused, because a new one per call means a new TLS handshake per call and, under load, a process that exhausts its file descriptors — a mistake that is invisible in a unit test and obvious in a load test.

The reason that mistake hides so well is worth two sentences, because it is the same reason it then arrives all at once. A client object owns a pool of open connections and hands them out to successive requests, so the handshake cost is paid once and amortised over everything after it. Construct a fresh client per call and every request starts with an empty pool, pays the handshake, and leaves behind a socket that is not immediately reusable by anything. At one request at a time — a unit test, a notebook, a staging environment with no traffic — the only visible effect is that each call is slightly slower, which nobody notices. At production concurrency the sockets accumulate faster than they are released, and the process stops being able to open any, which presents as a total failure rather than as a gradual slowdown. And the observability has to reach parity deliberately: whatever your SDK path emits for latency, attempt counts, request identifiers and token usage, the raw path must emit too, or the flag that switches between them also switches off half your dashboards and any comparison between the two becomes impossible at exactly the moment you need it.

Keeping the decision reversible#

The reason the isolated call wins is not that raw HTTP is pleasant. It is that the change is contained and therefore undoable, and containment is a property you have to design rather than one you get automatically.

Put the raw call in one module with the same function signature as the SDK path, and have both sit behind a selector — a configuration flag or an environment variable — so switching between them is a deploy and not a refactor. Write one contract test that runs the same request through both paths and asserts the same parsed result, skipping the capability-specific field; that test is what tells you the day the client catches up, because it will start passing with the flag flipped. Record the reason in the module docstring with the issue link and the client version you expect to supersede it, so that the next person reading it knows the exit condition rather than inferring that this is how the team likes to write code.

Two further cases justify raw HTTP, and both are temporary in the same way. The first is verifying during an incident what actually went over the wire, which is a debugging act rather than an architectural one. The second is a provider endpoint the client has no method for at all, where the untyped post usually suffices.

What does not justify it is a general preference. A colleague arguing that bypassing the client is always a code smell is wrong about the beta case and right about the ambient case: as a default style, hand-rolling every call means owning retries, backoff, streaming, timeouts, connection reuse and error classification for every endpoint you touch, and doing all of it worse than the vendor does, forever.

The rule that holds up is that a raw call is a scoped exception with a stated exit condition. Scoped means one endpoint, one module, one flag, behind the same signature as the path it replaces. Stated means written down — the issue, the version you expect to supersede it, and a test that will notice. If you cannot write down what makes it go away, you are not making an exception, you are choosing a policy, and the policy you are choosing is to maintain a worse copy of the vendor's client.

The seam in this chapter is drawn around one endpoint, which is what makes it cheap. Teams routinely reach for the same reasoning one level up and propose a seam around the providers themselves — one internal interface, several backends, chosen by configuration — on the theory that it prevents lock-in. That is a much larger claim, and it is the same claim a vendor makes when they advertise that their endpoint is compatible with somebody else's.