What Broke When We Hit 100k WebSocket Connections (And How Realtime Orchestration Saved Us)
A system streaming AI model outputs in realtime experienced severe latency spikes, message loss, and connection storms after scaling to 100,000 WebSocket connections, revealing the inadequacy of a sim…