The hidden complexity of scaling WebSockets

Why can't the server send unsolicited messages in JSON-RPC? I've implemented bidirectional JSON-RPC in multiple projects and things work just fine, even going as far as sharing most of the code.

Many years ago, we used to start a streaming session with an http request, then upgrading to websockets after obtaining a response (this was our original "StreamSense" mechanism). In recent years, we changed StreamSense to go websocket first and fallback to http streaming or http long polling in case of issues. At Lightstreamer, we started streaming data 25 years ago over http, then moving to websockets. We've seen so many different behaviors in the wild internet and got some much feedback from the fieldsl in these decades that we believe our current version of Lightstreamer includes heuristics and mechanisms for virtually every possible aspect of websockets that could go wrong. From massive disconnections and reconnections, to enterprise proxies with deep inspections, to mobile users continuously switching networks. I recall when a big customer required us to support one million live websocket connections for each server (mid-sized) keeping low latency. It was challenging but forced us to come up with a brand new internal architecture. So many stories to tell covering 25 years of evolution...

Eh, if you're dealing with corporate network proxies all bets are already off. They keep blocking connections for the most random reasons until everyone is opening ssh tunnels just to get work done. Websockets are a standard feature of the web, if you cut off your ears don't complain about loss of hearing. Unless, you're explicitly targeting such corporations as clients, in which case - my condolences.

> non-parallel XOR of input stream

I remember this one in particular making me upset, simply because of another extra buffer pass for security reasons that I believe are only to prevent proxies doing shit they never should have done in the first place?

I have been working on an idea/Node.js library called vramework.dev recently, and a big part of it focuses on addressing the main complexities mentioned below.

For a bit of background, in order to tackle scalability, the initial approach was to explore serverless architecture. While there are both advantages and disadvantages to serverless, a notable issue with WebSockets on AWS* is that every time a message is received, it invokes a function. Similarly, sending a message to a WebSocket requires invoking an HTTP call to their gateway with the websocket / channel id.

The upside of this approach is that you get out-of-the-box scalability by dividing your code into functions and building things in a distributed fashion. The downside is latency, due to all the extra network hops.

This is where vramework comes in. It allows you to define a few functions (e.g., onConnect, onDisconnect, onCertainMessage) and provides the flexibility to run them locally using libraries like uws, ws, or socket.io, or deploy them in the cloud via AWS or Cloudflare (currently supported).

When running locally, the event bus operates locally as well, eliminating latency issues. If you apply the same framework to serverless, latency increases, but you gain scalability for free.

Additionally, vramework provides the following features:

- Standard Tooling

Each message is validated against its typescript signature at runtime. Any errors are caught and sent to the client. (Note: The error-handling mechanism has not yet been given much thought into as an API). Rate limiting is also incorporated as part of the permissioning system (each message can have permissions checked, one of them could rate limiting)

- Per-Message Authentication

It guards against abuse by ensuring that each message is valid for the user before processing it. For example, you can configure the framework to allow unauthenticated messages for certain actions like authentication or ping/pong, while requiring authentication for others.

- User Sessions

Another key feature is the ability to associate each message with a user session. This is essential not only for authentication but also for the actual functionality of the application. This is done by doing a call to a cache (optionally) which returns the user session associated with the websocket. This session can be updated during the websocket lifetime if needed (if your protocol deals with auth as part of it's messages and not on connection)

Some doc links:

https://vramework.dev/docs/channels/channel-intro

A post that explains vramework.dev a bit more in depth (linked directly to a code example for websockets):

https://presentation.vramework.dev/#/33/0/5

And one last thing, it also produces a fully typed websocket client, so if using routes (where a property in your message indicates which function to use, the approach AWS uses serverless).

Would love to get thoughts and feedback on this!

edit: *and potentially Cloudflare, though I’m not entirely sure of its internal workings, just the Hibernation server and optimising for cost saving