Live data from Hacker News

The hidden complexity of scaling WebSockets

composehq.com

21–30 of 72 posts

Re: The hidden complexity of scaling WebSockets

#21
post #15

This is all true, but it also serves to remind us that Rails gives developers so much out of the box, even if you're not aware of it. ActionCable is Rails' WebSockets wrapper library, and it addresses basically every pain point in the post. However, it does so in a way that all Rails developers are using the same battle-tested solution. There's no need for every project to hack together its own proprietary approach.…

Elixir’s lightweight processes are also a good fit. Though I’ve seen some benchmarks that claim that goroutines can hit even lower overhead per connection.

That makes sense, Erlang/Elixir processes are a much higher-level construct than goroutines, and they trade off performance for fault tolerance and observability.

As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service. In Elixir a websocket handler can crash anywhere without impacting the application. This comes at a cost, because to make this safe Elixir has to isolate the processes so they don't share memory, so each process has its own individual heap, and data gets copied around more often than in Go.

Re: The hidden complexity of scaling WebSockets

#22

Earlier quoted context omitted.

Or just use jsonrpc.

??? That solves none of the issues outlined in the post or the comments.

It solves the very limited problem of bike-shedding envelope shapes for request/reply protocols, which I think was all they meant to say.

At its core, JSON-RPC boils down to "use `id` and `method` and work the rest out", which is acceptably minimal but does leave you with a lot of other issues to deal with.

Re: The hidden complexity of scaling WebSockets

#23
post #21
post #15

Earlier quoted context omitted.

Elixir’s lightweight processes are also a good fit. Though I’ve seen some benchmarks that claim that goroutines can hit even lower overhead per connection.

That makes sense, Erlang/Elixir processes are a much higher-level construct than goroutines, and they trade off performance for fault tolerance and observability. As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service. In Elixir a websocket handler can crash anywhere without impacting the application. This comes at a cost, because to make this sa…

> As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service.

Unless you're the default `net/http` library and simply recover from the panic: https://github.com/golang/go/blob/master/src/net/http/server...

Re: The hidden complexity of scaling WebSockets

#24
> WebSocket connections can be unexpectedly blocked, especially on restrictive public networks.

What? How would public network even know you’re running a websocket if you’re using TLS? I dont think it’s really possible in general case

> Since SSE is HTTP-based, it's much less likely to be blocked, providing a reliable alternative in restricted environments.

And websockets are not http-based?

What article describes as challenges seems like very pedestrian things that any rpc-based backend needs to solve.

The real reason websockets are hard to scale is because they pin state to a particular backend replica so if the whole bunch of them disconnect at scale the system might run out of resources trying to re-load all that state

Re: The hidden complexity of scaling WebSockets

#25

> WebSocket connections can be unexpectedly blocked, especially on restrictive public networks. What? How would public network even know you’re running a websocket if you’re using TLS? I dont think it’s really possible in general case > Since SSE is HTTP-based, it's much less likely to be blocked, providing a reliable alternative in restricted environments. And websockets are not http-based? What article describes as…

The initial handshake will usually include an `Upgrade: websocket` header, which can be inspected by networks.

Re: The hidden complexity of scaling WebSockets

#26

Earlier quoted context omitted.

Node has similar libraries like Socket.IO too, but it over-abstracts it a bit in my opinion.

I've done my share of building websocket servers from scratch, but when you don't use libraries like ActiveCable or socket.io, you have to build your own MessageID reconciliation so that you can have request/response cycles. Which is generally what you want (or eventually want) in a websocket-heavy application. send(payload).then(reply => ...)

At this point why even use a websocket vs a normal request/reply technology like grpc or json-rpc?

Re: The hidden complexity of scaling WebSockets

#27

> WebSocket connections can be unexpectedly blocked, especially on restrictive public networks. What? How would public network even know you’re running a websocket if you’re using TLS? I dont think it’s really possible in general case > Since SSE is HTTP-based, it's much less likely to be blocked, providing a reliable alternative in restricted environments. And websockets are not http-based? What article describes as…

The initial handshake will usually include an `Upgrade: websocket` header, which can be inspected by networks.

No, it literally can not be because by the time Upgrade header appears the connection is already encrypted.

Re: The hidden complexity of scaling WebSockets

#28
for me, the most important lesson i've learned when using websockets is to not use them whenever possible.

i don't hate them, they're great for what they are, but they're for realtime push of small messages only. trying to use them for the rest of your API as well just throws out all the great things about http - like caching and load balancing, and just normal request/response architecture. while you can use websockets for that it's only going to cause you headaches that are already solved by simply using a normal http api for the vast majority of your api.

Re: The hidden complexity of scaling WebSockets

#29
post #17

I wrote about the way we handle WebSocket connections at Canva a while ago [1]. Even though some small things have changed here and there since the post was published, the overall approach has held up pretty well handling many millions of concurrent connections. That said, even with great framework-level support, it's much, much harder to build a streaming functionality compared to plain request/response if you've go…

> it's much, much harder to build a streaming functionality compared to plain request/response if you've got some notion of a "session"

This touches something that I think is starting to become understood- the concept of a "session backend" to address this kind of use case.

See the complexity of disaggregation a live session backend on AWS versus CloudFlare: https://digest.browsertech.com/archive/browsertech-digest-cl...

I wrote about session backends as distinct from durable execution: https://crabmusket.net/2024/durable-execution-versus-session...

Re: The hidden complexity of scaling WebSockets

#30
The only complexity I have found with regards to scaling WebSockets is knowing the minimum delay between flush event completion and actual message completion to destination. It takes longer to process a message, even on IPC routing, than it does to kill a socket. That has upstream consequences with consideration of redirection and message pipes between multiple sockets. If you kill a socket too early after a message is flushed from the socket there is a good chance the destination sees the socket collapse before it has processed the final message off the socket and that processing delay is not something a remote location is easily aware of.

I have found for safety you need to allow an arbitrary delay of 100ms before killing sockets to ensure message completion which is likely why the protocol imposes a round trip of control frame opcode 8 before closing the connection the right way.

Post reply on HN