What is the difference between WebSockets and SSE for streaming LLM responses?hard

Type
conceptual
Topic
websockets
Frequency
common
Tags
FastAPI, WebSockets, SSE, streaming, LLM
Answer

WebSockets provide a persistent bidirectional connection over a single TCP connection — either side can send messages at any time. SSE (Server-Sent Events) is server-to-client only, HTTP-based, simpler to proxy, and increasingly preferred for LLM token streaming.

Explanation

HTTP is stateless and unidirectional: client requests, server responds, connection closes. WebSockets upgrade an HTTP connection to a persistent socket via a protocol handshake. FastAPI's @app.websocket decorator handles the connection lifecycle — always handle WebSocketDisconnect since clients drop unexpectedly. For simple LLM token streaming, StreamingResponse with an async generator is often cleaner than WebSockets: it works over standard HTTP/2 and is easier to proxy and cache. Use WebSockets when the client needs to send messages mid-stream, such as in interactive multi-turn agent interfaces.

Follow-upHow would you stream LLM tokens back to a browser in real time?
Follow-upWhen would you choose WebSockets over SSE?
Follow-upHow does StreamingResponse work in FastAPI?