Choosing between SSE and WebSockets

Server-Sent Events, introduced in the first chapter of this topic, is a server-to-client sequence of small text events carried inside an ordinary HTTP response whose body is left open. A WebSocket is the other thing browsers can hold open: a connection that begins as an HTTP request, is upgraded, and afterwards is a symmetric channel of length-prefixed frames on which either side may send at any time. Both are available to any browser, both are one TCP connection, and the choice between them is made in nearly every team that writes a streaming ticket.

The instinct that drives the choice is usually that WebSockets are the fast transport, and they are not faster in any sense that applies here. The per-message overhead does differ — a WebSocket text frame adds 2 to 14 bytes of header, an SSE event adds data: and two newlines, so roughly 8 bytes plus whatever JSON envelope the provider uses — and at 60 events per second that is a few hundred bytes per second on a connection already carrying kilobytes. It does not appear in a latency measurement. The differences that do matter are structural, and this chapter is about weighing them for an endpoint whose job is to push tokens at a browser.

The reason the overhead does not appear in a latency measurement is worth following, because it is the step people skip when they treat "lower overhead" and "lower latency" as the same claim. The interval between a server writing a token and a browser being able to read it is set by the path: how far the signal has to travel, how long it waits in a queue at each hop, and whether anything has to be sent twice. Both protocols run over one TCP connection between the same two machines, so both inherit the same path, the same queues and the same congestion state — a connection that has backed off because the network is busy is equally backed off whichever framing is riding on it. Neither protocol changes any term in that sum. What each one adds is a small number of header bytes per message, and header bytes only turn into time when they push a message across a packet boundary or when the link is saturated. A token delta is far below the size at which either happens. The event carrying the word "Lisbon" therefore becomes readable at the same instant under both, and the nine milliseconds in the scenario later in this chapter are what that looks like when somebody measures it.

The reputation being traded on here was earned honestly, which is why the instinct is so durable. WebSockets really are the transport under the most latency-sensitive things on the web, and the teams that build those things are not confused. What those products need is a channel on which the client can send continuously while the server is sending — bidirectionality, and small binary frames to carry it — and a token stream uses neither. Buying a protocol for its reputation means buying the operational cost of a property you have no use for.

The difference that is real#

The protocols diverge at the handshake, and everything that matters follows from what the connection becomes afterwards. SSE never stops being an HTTP response: you make a request, the server replies 200 with content-type: text/event-stream, and the body is written incrementally until it ends. A WebSocket starts as an HTTP request and then leaves HTTP behind.

GET /chat HTTP/1.1
Host: api.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

After that 101 there are no further requests, responses, or status codes on the socket. What remains is a bidirectional pipe of frames, text or binary, that either side may send at any time. The concrete difference is direction and framing: SSE gives a server-to-client sequence of text events with HTTP semantics intact, and a WebSocket gives a symmetric channel with HTTP semantics discarded.

Losing HTTP semantics is the part that costs money. The Authorization header, cookies, CORS preflight, per-request access logs, request-scoped tracing, rate limiting keyed on a route, the rules in a web application firewall (WAF) that inspect traffic for attacks, and retry policy all operate on requests. A WebSocket has exactly one request — the handshake — and everything after it is invisible to that machinery. Authentication becomes a message you define and validate yourself, usually a first frame carrying a token, and now you own the window between connection and authentication. Rate limiting becomes a counter you maintain per connection. Your access log has one line per conversation instead of one per turn, which is a real loss the first time you try to answer "how many chat completions did tenant 44 run yesterday" from logs you already have.

Every item on that list fails for one reason, and seeing the reason once is worth more than memorising the list. Each of those mechanisms is a function of a request: a WAF rule runs when a request arrives and inspects its headers and body, a rate limiter increments a counter keyed on a route and a caller, an access log writes a line per request, a tracing library opens a span per request, an authenticator reads a header off a request. A WebSocket produces exactly one request in its lifetime, at the handshake, and everything that happens afterwards is frames on a socket that no longer has requests in it. So the machinery does not misbehave, and it does not need reconfiguring. It observes one event per conversation because one event per conversation is all there is to observe, and each of those functions has to be written again inside your application against a vocabulary you invent.

Weighing the two for a token-push endpoint#

Two of those rows deserve numbers. On HTTP/1.1 a browser allows six concurrent connections per origin, and a long-lived SSE response occupies one for its entire duration, so a user with four tabs open on a streaming answer has four of six slots gone and the fifth tab's image requests queue behind them. Over HTTP/2 the streams multiplex on one connection and the limit stops applying, which makes "is production HTTP/2 end to end" a question worth answering before you dismiss the issue. WebSockets are exempt from that budget in both cases, and browsers permit far more of them.

The limit bites for a structural reason rather than an arbitrary one, and it is the reason HTTP/2 dissolves it. On HTTP/1.1 a connection carries one request at a time: the response has to be read to its end before the connection is free for the next request, because there is nothing in the framing that would let two responses interleave and still be told apart. A streaming response has no end until the generation stops, so it holds the connection for the whole generation. HTTP/2 adds exactly the missing thing — every frame carries a stream identifier, so responses can interleave on one connection and be reassembled — which means an open stream no longer occupies a connection slot and the six stops applying. "Is production HTTP/2 end to end" is worth answering rather than assuming, because the answer is a property of the whole path and one hop that downgrades is enough to bring the limit back.

The second is idle behaviour. Managed load balancers idle out connections that have carried no bytes for a while — 60 seconds is the common default — and both protocols need something on the wire before that. SSE sends a comment line, the : ping from the first chapter of this topic, which the parser skips. A WebSocket sends a protocol-level ping frame, which the browser answers automatically. Neither is hard; both are things you will forget until a connection dies at exactly 60 seconds in staging.

Why a balancer reaps a healthy connection at all is worth one step, because understanding it tells you what the heartbeat is and is not evidence of. A connection costs the balancer a slot whether or not anything is moving on it, and from the balancer's position an idle connection to a live peer and an idle connection to a peer that vanished without a FIN look identical — no bytes have arrived in either case. Having no way to distinguish them, it distinguishes them by time. A byte on the wire resets the clock because a byte is the only evidence available. What that byte proves is that the path and the writer are alive; it proves nothing about whether anybody is reading, which is a distinction a later chapter in this topic turns on.

The row about load balancing hides the largest operational difference. An SSE request is a request: it lands on whichever node the balancer picks, that node calls the provider, and when the answer is done the node is free. A WebSocket binds a conversation to a process for its lifetime, so any message that needs to reach that user has to reach that node, which means a publish/subscribe layer — Redis, NATS, a message bus — over which any node can hand a message to whichever node is holding that connection, and a subscription per connection. That is a component you now operate, monitor, and pay for. Rolling deploys become interesting too: every connection on a node being replaced is severed, so a fleet of 10,000 connections cycling over a five-minute deploy produces 10,000 reconnections in bursts, and reconnect logic without jitter — a small random delay added to each client's retry so that they do not all fire at the same instant — turns each burst into a synchronised thundering herd against the nodes that are still up. None of this is unsolvable, since chat products have run on WebSockets for a decade, but it is the actual bill for the transport, and it is not paid in latency.

It is worth seeing that none of that bill was a design choice anybody made. A socket is held by one process on one machine, and a socket cannot be handed to a different process. A WebSocket keeps its socket for the life of the conversation, so the conversation is bound to that process for the same period. Anything with something to say to that user therefore has to reach that process specifically, and a system that must address a process rather than a service needs a message bus and a subscription per connection. The connection manager, the sticky routing and the reconnection policy follow from the same root: a long-held socket is a piece of durable state living somewhere that was built to be stateless. SSE avoids all of it by being over when the answer is over, leaving each node free to forget the conversation entirely.

There is a third option that rarely gets named in these arguments, which is to stream newline-delimited JSON over an ordinary chunked HTTP response. No data: prefixes, no blank-line framing, just one JSON object per line, read with the same fetch and ReadableStream machinery and split on \n. It is marginally simpler than SSE, it has no EventSource baggage, and it works fine. What SSE has over it is that every provider already speaks it, so a relay that consumes SSE and emits SSE has one framing to reason about instead of two; that intermediaries recognise text/event-stream and treat it specially, which is worth having for reasons a later chapter on buffering proxies makes concrete; and that the comment-line heartbeat is part of the format rather than something you invent. NDJSON is a reasonable choice for an internal service-to-service stream and a slightly worse one for a browser endpoint.

For a chat endpoint that pushes token deltas, SSE is the right pick, and the argument is not aesthetic. The interaction is one request in, one long answer out, and that is precisely the shape SSE encodes; using a symmetric channel for an asymmetric problem means writing a message protocol, an auth handshake, and a reconnection policy in order to express something an HTTP request already expresses. The existing route keeps its middleware, its logs, and its auth. The client change is contained: fetch returns a body you read incrementally rather than a JSON promise.

What would change the answer#

The requirement that flips this is client-to-server traffic during generation, which is not the same as client-to-server traffic at all. Cancelling a generation does not need a WebSocket: closing the HTTP response cancels it, which is the subject of a later chapter in this topic. What needs one is input that arrives while the model is producing output and must reach the same in-flight session.

Voice is the clearest case. A speech-to-speech assistant streams microphone audio up as 20 ms binary frames while streaming synthesised audio down, and the user interrupting mid-sentence — barge-in — has to be detected on the upstream audio and applied to the running generation. That is two continuous flows in opposite directions with binary payloads and hard latency budgets, and it is why the realtime APIs from major providers are WebSocket or WebRTC — the browser's peer-to-peer transport for live audio and video — rather than SSE. A collaborative document where several people's edits must land in one shared model session is the same shape. So is a long-running agent that pushes progress to a client which has no request open, since SSE requires a client-initiated request to push anything at all.

That third case is worth separating from the other two, because teams building agents run into it and misdiagnose it as a chat problem. An agent that runs for four minutes across a dozen tool calls does need to push progress to a client, and if the client has no request open then SSE has nothing to push over. The usual resolution is not a WebSocket but a second SSE endpoint: the client opens a long-lived GET /runs/{id}/events subscription that is independent of whatever request kicked the run off, and the server writes to it as the run progresses. The connection is still client-initiated and still one-directional, which is all SSE requires, and the run's identity is in the URL rather than in a connection you must keep alive. That pattern also survives a page reload, which a WebSocket-bound session does not without work.

The move that makes that work is worth naming, because it looks like a trick and is not one. SSE's constraint is that the server can only write into a request the client opened; it is not that the server can only write into the request that started the work. Separating those two lifts the whole apparent limitation: the run is identified by an id that lives in a URL, and the subscription is a fresh request the client may open, close and reopen as often as it likes, including after a reload that destroyed everything else about the page. What a WebSocket offers instead is a channel the server can write into unprompted, and the price of that channel is that its identity is the connection — so the moment the connection goes, the identity goes with it and there is nothing to come back to.

There is one honest advantage on the WebSocket side. Because it stops being HTTP after the upgrade, intermediaries that buffer response bodies cannot buffer it, so the failure mode where a proxy collects the whole answer and releases it at the end does not arise. That is a genuine property, though it argues for configuring your proxy rather than for changing protocol, and it comes attached to the requirement that every hop supports upgrades — which some enterprise proxies and some serverless platforms still do not.

The property is real for a specific reason, and the reason also bounds it. An intermediary buffers by holding a response body, and a response body is a thing that exists because a response exists. After the 101 there is no response and no body — only frames on a socket the intermediary agreed to stop interpreting — so there is nothing of the right shape for it to hold. That is why the guarantee is stronger than a configuration setting: it does not depend on any hop being told to behave. It also explains the attached requirement, since a hop that does not understand upgrades will not agree to stop interpreting, and the handshake fails outright rather than degrading into something buffered.

Worth separating from all of this: neither protocol makes a stream durable. Both die with the process. If a deploy restarts the node holding an in-flight generation, the browser loses the answer either way, and the WebSocket's automatic reconnect will reconnect to a node that has never heard of that conversation. Resumable generation is a server-side problem — persist the partial text, key it by a generation id, let the client ask to resume — and no transport choice supplies it.

So the decision rule is short enough to apply in a design review. If a requirements document says the user can steer the answer while it is being written, or that the client sends audio or video continuously, or that the server notifies idle clients of background job progress, the extra machinery is buying something and a WebSocket is the honest answer. Absent one of those, the WebSocket is a connection manager, a pub/sub layer and a reconnection policy maintained for no delivered benefit, and the same nine milliseconds.

Whichever you pick, the shape of the system is now fixed: the provider streams to your API, and your API streams to the browser. That middle hop looks like piping bytes from one socket to another, and the next chapter is about why it cannot be.