What is the difference between WebSockets and SSE for streaming LLM responses?hard
WebSockets provide a persistent bidirectional connection over a single TCP connection — either side can send messages at any time. SSE (Server-Sent Events) is server-to-client only, HTTP-based, simpler to proxy, and increasingly preferred for LLM token streaming.
HTTP is stateless and unidirectional: client requests, server responds, connection closes. WebSockets upgrade an HTTP connection to a persistent socket via a protocol handshake. FastAPI's @app.websocket decorator handles the connection lifecycle — always handle WebSocketDisconnect since clients drop unexpectedly. For simple LLM token streaming, StreamingResponse with an async generator is often cleaner than WebSockets: it works over standard HTTP/2 and is easier to proxy and cache. Use WebSockets when the client needs to send messages mid-stream, such as in interactive multi-turn agent interfaces.