My SaaS has been using WebSockets for the last 9 years. I plan to stop using them and move to very simple HTTP-based polling. I found that scalability isn't a problem (it rarely is these days). The real problem is crappy network equipment all over the world that will sometimes break websockets in strange and mysterious ways. I guess not all network equipment vendors test with long-lived HTTP websocket connections wit…
The hidden complexity of scaling WebSockets
71–72 of 72 posts
Re: The hidden complexity of scaling WebSockets
#72Earlier quoted context omitted.
I think it's more that WebSockets are held open for a long time, so if you're not careful, you can get "hot" backends with a lot of connections that you can't shift to a different instance. It can also be harder to rotate backends since you know you are disrupting a large number of active clients.
Assuming you control the client code, you can periodically disconnect and reconnect. This could also simplify deployment.
Of course you do want to make sure the client has exponential backoff and jitter when reconnecting, as to avoid thundering herd problems.
The relevant state will need to be available to all servers as well. Anything that's only known to the original server will be lost as it drops connections. On a modern deployment with a database and probably redis available this isn't too big of an ask.