Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The hidden complexity of scaling WebSockets

composehq.com

Post reply on HN
Print view View original

31–40 of 72 posts

Re: The hidden complexity of scaling WebSockets

#31

Post by mirekrusin »

crabmusket wrote: post #22
moralestapia wrote: post #18

Earlier quoted context omitted.

??? That solves none of the issues outlined in the post or the comments.

It solves the very limited problem of bike-shedding envelope shapes for request/reply protocols, which I think was all they meant to say. At its core, JSON-RPC boils down to "use `id` and `method` and work the rest out", which is acceptably minimal but does leave you with a lot of other issues to deal with.

It's a bit misnomer because it defines rpcs _and_ notifications.

What people seem to be often missing for some reason is that those two map naturally to existing semantics of the programming language they're already using.

What it means in practice is that you are exposing and consuming functions (ie. on classes) – just like you do in ordinary libraries.

In js/ts context it usually means async functions on classes annotated with decorators (to register method as rpc and perform runtime assertions) that are event emitters – all concepts already familiar to developers.

To summarize you have system that is easy to inspect, reason about and easy to use - almost like any other package in your dependency.

Introducing backward compatible/incompatible changes also becomes straight forward for everybody ie. following semver on api surface just like in any ordinary package you depend on.

Those straight forward facts are often missed and largely underappriciated.

ps. in our systems we're introducing two deviations – error code can also be strings, not just numbers (trivial); and we support async generators (emitting individual objects for array results) – which helps with head of line blocking issues for large resultsets (still compatible with jsonrpc at protocol level, although it would be nice if they supported it upstream as dedicated semantic in jsonrpc 2.1 or something). They could also specify registering and unregistering notification listeners at the spec level so everybody is using the same scheme.

Re: The hidden complexity of scaling WebSockets

#32

Post by jauco »

dilyevsky wrote: post #27
atul-jalan wrote: post #25

Earlier quoted context omitted.

The initial handshake will usually include an `Upgrade: websocket` header, which can be inspected by networks.

No, it literally can not be because by the time Upgrade header appears the connection is already encrypted.

Restricted environments in larger corporations can do a full mitm proxy

Re: The hidden complexity of scaling WebSockets

#33

Post by neillyons »

flakes wrote: post #23
ramchip wrote: post #21

Earlier quoted context omitted.

That makes sense, Erlang/Elixir processes are a much higher-level construct than goroutines, and they trade off performance for fault tolerance and observability. As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service. In Elixir a websocket handler can crash anywhere without impacting the application. This comes at a cost, because to make this sa…

> As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service. Unless you're the default `net/http` library and simply recover from the panic: https://github.com/golang/go/blob/master/src/net/http/server...

You still need to be careful, as this won't catch panics from go routines launched from your http handler.

Re: The hidden complexity of scaling WebSockets

#34

Post by pk-protect-ai »

dilyevsky wrote: post #24

> WebSocket connections can be unexpectedly blocked, especially on restrictive public networks. What? How would public network even know you’re running a websocket if you’re using TLS? I dont think it’s really possible in general case > Since SSE is HTTP-based, it's much less likely to be blocked, providing a reliable alternative in restricted environments. And websockets are not http-based? What article describes as…

I agree here. I have had an experience of scaling WebSockets server to 20M connections on a single server (with this one https://github.com/ITpC/LAppS.git). However there are several issues with scaling WebSockets, on the backends as well: mutex locking, non-parallel XOR of input stream, utf8 validation. I do not know the state of the above repository code, it seems that it was never updated for at least 5 years. There were bugs in HTTP parsing in the client part for some cases. Though vertical scalability was excellent. Sad this thing never reached production state.

Re: The hidden complexity of scaling WebSockets

#35

Post by ramchip »

flakes wrote: post #23
ramchip wrote: post #21

Earlier quoted context omitted.

That makes sense, Erlang/Elixir processes are a much higher-level construct than goroutines, and they trade off performance for fault tolerance and observability. As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service. In Elixir a websocket handler can crash anywhere without impacting the application. This comes at a cost, because to make this sa…

> As an example, with a goroutine you have to be careful to handle all errors, because a panic would take down the whole service. Unless you're the default `net/http` library and simply recover from the panic: https://github.com/golang/go/blob/master/src/net/http/server...

Yeah, I'm simplifying a bit. It may not cause an immediate exit, but it can leave the service broken in unpredictable ways. See this discussion for instance: https://iximiuz.com/en/posts/go-http-handlers-panic-and-dead...

Re: The hidden complexity of scaling WebSockets

#36

Post by peteforde »

dilyevsky wrote: post #26
hombre_fatal wrote: post #7

Earlier quoted context omitted.

I've done my share of building websocket servers from scratch, but when you don't use libraries like ActiveCable or socket.io, you have to build your own MessageID reconciliation so that you can have request/response cycles. Which is generally what you want (or eventually want) in a websocket-heavy application. send(payload).then(reply => ...)

At this point why even use a websocket vs a normal request/reply technology like grpc or json-rpc?

For scenarios requiring a constant exchange of information, such as streaming data or real-time updates. After the initial handshake, data is exchanged directly over the connection with minimal overhead. Lower latency is especially beneficial for high-frequency message exchanges. Gaming, live auctions, or real-time dashboards are well suited. I also think that real time collaboration is under-explored.

JSON-RPC is request-response only; the server cannot send unsolicited messages. gRPC supports bidirectional streaming, but I understand that setting it up is more complex than WebSockets.

I will concede that horizontal scaling of RPC is easier because there's no connection overhead.

Ultimately, it really depends on what you're trying to build. I also don't underestimate the cultural aspect; fair or not, JSON-RPC feels very "enterprise microservices" to me. If you think in schemas, RPC might be a good fit.

Re: The hidden complexity of scaling WebSockets

#37

Post by superjan »

arccy wrote: post #10
exabrial wrote: post #3

I recall another complication with websockets: IIRC it's with proxy load balancers, like binding a connection to a single connection server, even if the backend connection is using HTTP/2. I probably have the details wrong. I'm sure someone will correct my statement.

I think it's more that WebSockets are held open for a long time, so if you're not careful, you can get "hot" backends with a lot of connections that you can't shift to a different instance. It can also be harder to rotate backends since you know you are disrupting a large number of active clients.

Assuming you control the client code, you can periodically disconnect and reconnect. This could also simplify deployment.

Re: The hidden complexity of scaling WebSockets

#40

Post by ignoramous »

hinkley wrote: post #16
10000truths wrote: post #11

The key to managing this complexity is to avoid mixing transport-level state with application-level state. The same approach for scaling HTTP requests also works for scaling WebSocket connections: * Read, write and track all application-level state in a persistent data store. * Identify sessions with a session token so that application-level sessions can span multiple WebSocket connections. It's a lot easier to do th…

Functional core, imperative shell makes testing and this fast iteration a lot easier. It’s best if your business logic knows very little about transport mechanisms. I think part of the problem is that early systems wanted to eagerly process requests while they are still coming in. But in a system getting 100s of requests per second you get better concurrency if you wait for entire payloads before you waste cache line…

> system getting 100s of requests per second you get better concurrency if you wait for entire payloads before you waste cache lines

At what point should one scale up & switch to chips with embedded DRAMs ("L4 cache")?

Post reply on HN