Server-Sent Events vs WebSockets for Agent Streaming

Choosing between Server-Sent Events and WebSockets for agent streaming isn't a generic real-time-systems question. It comes down to how AI agents actually talk: mostly the server pushing tokens out, occasionally the client needing to jump in. Once you look at it through that lens, the "which protocol is faster" debate mostly evaporates, and a different, more useful question takes its place.

The connection-limit problem that held SSE back, and what fixed it

For years, SSE had a real skeleton in its closet. Under HTTP/1.1, browsers cap you at six concurrent connections per origin. An SSE stream never closes, so each one just sits there, permanently occupying a slot like a guest who won't leave the party. Open six SSE tabs and every other request to that domain queues up behind them. If you're the kind of person who never closes anything and you've got eight tabs open, tabs seven and eight just stall out, waiting their turn.

HTTP/2 addressed this well. Multiplexing lets all your SSE streams share a single TCP connection as separate logical streams, with room for roughly 100 concurrent streams by default. HTTP/3, riding on QUIC, pushes this further still, adding resilience on flaky networks where a dropped packet used to mean a dropped connection.

So the six-connection objection you might've read in a five-year-old blog post is a legacy concern now. It only bites teams still serving HTTP/1.1 clients, which, by 2024, is a shrinking club. This fix landed at almost the exact moment the industry needed it most: right as LLM adoption exploded, creating huge demand for exactly the pattern SSE was designed for, one request in, a long stream of tokens out.

Why every major LLM provider converged on SSE for their streaming APIs

Pick any major LLM provider, OpenAI, Anthropic, Google Gemini, and you'll find SSE running the show under the hood. Anthropic's Claude API is a clean example: set "stream": true when you create a Message, and SSE delivery kicks in. Their Python and TypeScript SDKs quietly handle the parsing so you never have to think about it.

Claude's streaming responses break into nine distinct event types, and tool-use blocks arrive as partial JSON strings tucked inside delta.partial_json. You concatenate those fragments before parsing them into real JSON. It's a small detail, but it shows how naturally SSE's event model maps onto what an agent is actually doing: producing structured output piece by piece, not all at once.

Frameworks followed the money. tRPC v11 added SSE subscriptions and recommends them by default, and GraphQL Yoga uses SSE as its default subscription transport. The Vercel AI SDK, OpenAI's SDK, Anthropic's SDK, all of them consume SSE somewhere below the surface. LLM inference is a server-driven stream responding to one client prompt, and SSE's one-way shape fits that like a glove. Most teams building on these APIs never actually "choose" SSE. It's already there, waiting, before they've made a single architectural decision.

What the performance numbers actually show — and what they don't settle

Here's the benchmark headline, so we can stop pretending speed decides this: SSE and WebSocket throughput are nearly identical in production. WebSocket does hold a latency edge of around 3 milliseconds, but that gets swallowed whole by normal network round-trip time, a rounding error dressed up as an argument.

Timeplus ran benchmarks in 2024 that put SSE at a max of 2.7 million events per second client-side under high concurrency, versus 2.4 million for WebSocket. But WebSocket can squeeze more out of the CPU in certain configurations, so nobody walks away from these numbers with a clean win. For AI streaming specifically, the number that actually matters isn't throughput at all, it's time to first token: how fast the user sees anything on screen. Streaming takes that perceived wait from something in the 5 to 15 second range down to under 500 milliseconds, and that's the number users feel in their bones.

There's also a wire-overhead comparison worth knowing: delivering 1,000 events costs 119 KB over WebSocket versus 885 KB over long polling, with SSE landing somewhere in between. Useful context if you're picking between approaches, though hardly a decisive factor on its own.

Put together, these benchmarks don't crown a winner. What they do is remove performance from the table entirely, which forces the real decision back onto communication model and infrastructure fit, where it belonged the whole time.

How SSE's HTTP-native design simplifies infrastructure at scale

SSE connections are just long-lived HTTP requests, and that fact alone does a lot of quiet, unglamorous work. Load balancers, TLS termination, CDN caching, request monitoring: all of it treats an SSE stream like any other HTTP traffic, with no special config and no separate playbook for your ops team to memorize at 2am.

SSE servers are stateless by default, so you don't need sticky sessions, and horizontal scaling doesn't demand a coordination layer sitting in the middle. Edge deployment falls out of this almost for free: Cloudflare Workers or AWS Lambda@Edge can fan a single origin stream out to a huge number of clients without you redesigning anything.

The spec even bakes in reconnection. If a client drops, it resumes from the last event ID automatically, on a retry interval the server controls, no application code required. And because SSE never leaves HTTP, it slides through corporate firewalls and packet inspection layers that would choke on a raw WebSocket handshake. The one wrinkle worth testing for yourself: some proxies silently buffer the stream unless you set headers like X-Accel-Buffering: no on Nginx, and you won't always control every intermediary between you and your user.

Compare that to WebSocket's scaling story. Persistent, stateful connections mean sticky sessions or specialized load-balancer rules. Scale horizontally and you need Redis pub/sub or a dedicated WebSocket gateway to keep everyone talking to the same server. AWS's Application Load Balancer closes idle WebSocket connections after 60 seconds by default; you can push that up to thousands of seconds, but someone on your team has to remember to configure it, and someone always forgets.

Where SSE's one-way model breaks down for agentic workflows

SSE is fine, more than fine, for a single prompt going in and a long answer streaming out. The whole model works exactly as long as the client has nothing urgent to say while the response is running. That assumption holds for a chatbot, but it falls apart the moment you're building an agent.

Agent workflows need the client to interrupt. Sometimes that's cancellation, stopping generation before it finishes something you no longer want. Sometimes it's tool-call approval, a human checking a box before the agent is allowed to actually do the thing it's proposing. Sometimes it's steering, nudging the agent's direction mid-task instead of waiting for it to finish going the wrong way.

None of those signals can travel over an active SSE connection. Each one needs its own separate HTTP request, which means a new round-trip and a real chance of a race condition against the event stream that's still running. Think about a coding assistant like Open Hands, which runs agents continuously in your infrastructure: keystrokes, cursor moves, cancel signals, all of it needs to reach the server while tokens are still streaming in. SSE forces that traffic onto a second connection, whether you like the extra plumbing or not.

The failure mode here is about correctness more than speed. A cancellation request that shows up over a separate HTTP call after the agent has already taken the action you were trying to stop is the wrong outcome, delivered late. That's the moment the directionality gap stops being a minor inconvenience and becomes an architecture problem.

OpenAI's WebSocket mode for the Responses API as a case study in agentic latency

OpenAI ran into this in a very concrete way. As inference speed climbed from 65 tokens per second to dramatically higher speeds, the bottleneck stopped being the model and started being the network. Every round-trip between agent steps added latency that had nothing to do with how fast the model could think.

Their fix was a persistent WebSocket connection to the Responses API, keeping the sampling loop alive across tool calls instead of tearing down and rebuilding an HTTP connection at every step. The mechanics are almost elegant: after the model samples a tool call, the server sends a response.done event and blocks. The client goes off and runs the tool, then sends a response.append event with the result back over that same open connection, and the model picks up right where it left off.

They looked at gRPC bidirectional streaming too, and chose WebSockets anyway, mainly because it let existing developers keep their current Responses API input and output shapes untouched. Ergonomics won over what might've been the more "elegant" engineering choice, which, honestly, is usually how these decisions go in the real world.

The results back it up: OpenAI reported up to a 40% cut in end-to-end latency. Vercel's AI SDK integration saw similar gains, also around 40%. One coding-agent team reported a 39% improvement on multi-file workflows. This case study draws a pretty clean line: when an agent runs many sequential steps and each one used to cost a full HTTP round-trip, a persistent bidirectional connection earns back its added complexity fast.

MCP's transport layer as a live example of the tension between the two protocols

Model Context Protocol defines how agents talk to tools and data sources, and its transport choice ripples directly into how builders integrate with it. The original spec (version 2025-03-26) went with a hybrid: server-to-client messages ride on SSE, client-to-server messages go out over separate HTTP POST requests.

That hybrid is basically an admission. It bakes the SSE limitation right into the protocol, acknowledging that client input can't share the same channel as the event stream and routing it around the side instead. It's a workaround with a spec number attached to it.

There's an active GitHub discussion (issue #1288) about whether MCP should move toward WebSockets or a Streamable HTTP transport for future versions. The debate circles the exact same tradeoffs this whole piece has been circling: bidirectionality versus simplicity, persistent connections versus stateless scaling. Nothing here is settled, even at the standards level, and teams adopting MCP today are making a transport bet the spec authors are still arguing about themselves.

How to match protocol to agent communication pattern in practice

Throughput and latency aren't the deciding factors here, not really. What matters is which direction your agent needs to talk during a session, and how often.

SSE is the right default when the interaction is basically request-response with a long streamed answer, tokens, status updates, progress events. It fits when client input is rare and can survive a separate HTTP round-trip, like a new prompt or a config change between sessions. It fits when you want infrastructure that behaves itself: normal load balancers, CDN compatibility, no sticky sessions, no coordination layer to babysit. And if you're building on top of OpenAI, Anthropic, or Gemini, SSE is already the transport underneath you, so fighting it would be strange.

WebSockets earn their complexity when the agent runs multi-step workflows where each step feeds the next without a fresh HTTP handshake in between. They matter when cancellation, approval, or steering signals need to land mid-stream with minimal lag, and when the agent's holding onto persistent state (tool context, memory, partial results) that makes reconnecting from scratch every step wasteful. They matter most once inference gets fast enough that the network, not the model, becomes the thing you're waiting on.

The hybrid pattern, SSE out and HTTP POST in, is a legitimate middle ground; MCP's original transport proves it works. Just know it opens up its own race-condition surface, and someone on your team needs to reason through that explicitly rather than discover it in production. For agent platforms running continuously, handling tool-call approval, and needing visibility across every step, an open-source project like OpenHands is a good example, the bidirectional demands push toward WebSockets or some other persistent transport at the agent-loop level. Even then, the individual LLM calls feeding that loop are probably still speaking SSE underneath, quietly doing the job it was built for.