The hidden complexity of scaling WebSockets • Hacker News Board

Re: The hidden complexity of scaling WebSockets

#11

Post by 10000truths » Fri, Jan 24, 2025, 10:53 PM UTC

The key to managing this complexity is to avoid mixing transport-level state with application-level state. The same approach for scaling HTTP requests also works for scaling WebSocket connections:

* Read, write and track all application-level state in a persistent data store.

* Identify sessions with a session token so that application-level sessions can span multiple WebSocket connections.

It's a lot easier to do this if your application-level protocol consists of a single discrete request and response (a la RPC). But you can also handle unidirectional/bidirectional streaming, as long as the stream states are tracked in your data store and on the client side.

Re: The hidden complexity of scaling WebSockets

#12

Post by mirekrusin » Fri, Jan 24, 2025, 10:58 PM UTC

hombre_fatal wrote: post #7

atul-jalan wrote: post #5

Earlier quoted context omitted.

Node has similar libraries like Socket.IO too, but it over-abstracts it a bit in my opinion.

I've done my share of building websocket servers from scratch, but when you don't use libraries like ActiveCable or socket.io, you have to build your own MessageID reconciliation so that you can have request/response cycles. Which is generally what you want (or eventually want) in a websocket-heavy application. send(payload).then(reply => ...)

Or just use jsonrpc.

Re: The hidden complexity of scaling WebSockets

#13

Post by dboreham » Fri, Jan 24, 2025, 11:37 PM UTC

arccy wrote: post #10

exabrial wrote: post #3

I recall another complication with websockets: IIRC it's with proxy load balancers, like binding a connection to a single connection server, even if the backend connection is using HTTP/2. I probably have the details wrong. I'm sure someone will correct my statement.

I think it's more that WebSockets are held open for a long time, so if you're not careful, you can get "hot" backends with a lot of connections that you can't shift to a different instance. It can also be harder to rotate backends since you know you are disrupting a large number of active clients.

The trick to doing this efficiently is to arrange for the live session state to be available (through replication or some data bus) at the alternative back end before cut over.

Re: The hidden complexity of scaling WebSockets

#14

Post by magicalhippo » Fri, Jan 24, 2025, 11:47 PM UTC

10000truths wrote: post #11

The key to managing this complexity is to avoid mixing transport-level state with application-level state. The same approach for scaling HTTP requests also works for scaling WebSocket connections: * Read, write and track all application-level state in a persistent data store. * Identify sessions with a session token so that application-level sessions can span multiple WebSocket connections. It's a lot easier to do th…

Currently another thread is going[1] which advocates very similar things, in order to reduce complexity when dealing with distributed systems.

Then again, the frontend and backend are a distributed system, so not that weird one comes to similar conclusions.

[1]: https://news.ycombinator.com/item?id=42813049 Every System is a Log: Avoiding coordination in distributed applications

Re: The hidden complexity of scaling WebSockets

#15

Post by hinkley » Fri, Jan 24, 2025, 11:57 PM UTC

peteforde wrote: post #2

This is all true, but it also serves to remind us that Rails gives developers so much out of the box, even if you're not aware of it. ActionCable is Rails' WebSockets wrapper library, and it addresses basically every pain point in the post. However, it does so in a way that all Rails developers are using the same battle-tested solution. There's no need for every project to hack together its own proprietary approach.…

Elixir’s lightweight processes are also a good fit. Though I’ve seen some benchmarks that claim that goroutines can hit even lower overhead per connection.

Re: The hidden complexity of scaling WebSockets

#16

Post by hinkley » Sat, Jan 25, 2025, 12:01 AM UTC

10000truths wrote: post #11

The key to managing this complexity is to avoid mixing transport-level state with application-level state. The same approach for scaling HTTP requests also works for scaling WebSocket connections: * Read, write and track all application-level state in a persistent data store. * Identify sessions with a session token so that application-level sessions can span multiple WebSocket connections. It's a lot easier to do th…

Functional core, imperative shell makes testing and this fast iteration a lot easier. It’s best if your business logic knows very little about transport mechanisms.

I think part of the problem is that early systems wanted to eagerly process requests while they are still coming in. But in a system getting 100s of requests per second you get better concurrency if you wait for entire payloads before you waste cache lines on attempting to make forward progress on incomplete data. Which means you can divorce the concept of a payload entirely from how you acquired it.

Re: The hidden complexity of scaling WebSockets

#17

Post by SerCe » Sat, Jan 25, 2025, 12:24 AM UTC

I wrote about the way we handle WebSocket connections at Canva a while ago [1]. Even though some small things have changed here and there since the post was published, the overall approach has held up pretty well handling many millions of concurrent connections.

That said, even with great framework-level support, it's much, much harder to build a streaming functionality compared to plain request/response if you've got some notion of a "session".

[1]: https://www.canva.dev/blog/engineering/enabling-real-time-co...

Re: The hidden complexity of scaling WebSockets

#18

Post by moralestapia » Sat, Jan 25, 2025, 1:03 AM UTC

mirekrusin wrote: post #12

hombre_fatal wrote: post #7

Earlier quoted context omitted.

I've done my share of building websocket servers from scratch, but when you don't use libraries like ActiveCable or socket.io, you have to build your own MessageID reconciliation so that you can have request/response cycles. Which is generally what you want (or eventually want) in a websocket-heavy application. send(payload).then(reply => ...)

Or just use jsonrpc.

???

That solves none of the issues outlined in the post or the comments.

Re: The hidden complexity of scaling WebSockets

#19

Post by XorNot » Sat, Jan 25, 2025, 1:19 AM UTC

atul-jalan wrote: post #8

hombre_fatal wrote: post #7

Earlier quoted context omitted.

I've done my share of building websocket servers from scratch, but when you don't use libraries like ActiveCable or socket.io, you have to build your own MessageID reconciliation so that you can have request/response cycles. Which is generally what you want (or eventually want) in a websocket-heavy application. send(payload).then(reply => ...)

Yep, for our application, we have an `executionId` that is sent in essentially every single WebSocket message. But client and server use it to maintain a record of events.

Isn't this JSON-RPC's approach?

Re: The hidden complexity of scaling WebSockets

#20

Post by p_l » Sat, Jan 25, 2025, 2:27 AM UTC

hombre_fatal wrote: post #7

atul-jalan wrote: post #5

Earlier quoted context omitted.

Node has similar libraries like Socket.IO too, but it over-abstracts it a bit in my opinion.

I've done my share of building websocket servers from scratch, but when you don't use libraries like ActiveCable or socket.io, you have to build your own MessageID reconciliation so that you can have request/response cycles. Which is generally what you want (or eventually want) in a websocket-heavy application. send(payload).then(reply => ...)

If you add Content-Negotiation it will have ALL the OSI layers! /s

Honestly, I'm a little surprised and more than a bit depressed how we effectively reinvent the OSI stack so often...