Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you scale WebSocket?

news.ycombinator.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Ask HN: How do you scale WebSocket?

#11

Post by ravirajx7 »

monroewalker wrote: post #8

I could be wrong, but I wouldn't think the autoscaling or load balancing would affect the websocket connection. There may be another aspect of the infrastructure that's preventing the connection though. Can you share more about the setup? Strange that the connection would succeed sometimes but not others.. This could be related to the configuration of some intermediate network layer. Eg. if Nginx is used, you may hav…

Hi, We're not using Nginx. We are using Docker image & everything is configured using cloudflare (whose details are something which I am not aware of properly).

Though, here the problem with websocket is that they are stateful and whenever a connection is established it is directly getting established with one of the instance from the list of several instances due to loadbalancer. Now, whenever a new Webhook response comes as it's a normal post request and it doesn't have information regarding which instance was used earlier for making the websocket connection, it may send request to one of the instance where the connection was not established and thus our backend is not able to process this request from this particular instance.

Re: Ask HN: How do you scale WebSocket?

#12

Post by ravirajx7 »

banashark wrote: post #5
ravirajx7 wrote: post #3

Earlier quoted context omitted.

Hi, Thank you for responding. Actually we have one external API which provides this detail as soon as the payment is done. But we wanted something wherein once the payment is done we can directly send back the details directly to the connected client using that incoming webhook response instead of making the client(browser) to call one another API/fetch state present in DB periodically.

Without knowing more than the details you've just provided, I would suggest giving the various options a second look and weighing the pros/cons. Websockets to me don't seem like the ideal approach here, since the communication is just from the server to the client (an update of a data payload once an event occurs in the backend system). Websockets have quite a bit of technical complexity requiring significant archite…

Hi, I agree with each of your points you mentioned. Still it's kind of curiosity within me which is making me think to achieve this goal as we have ample time and this indeed is a business need.

It would be very nice of you if you can share few other solution other than using Ably's API :(

Re: Ask HN: How do you scale WebSocket?

#13

Post by blablablub »

ravirajx7 wrote: post #10
blablablub wrote: post #6

" the connection is not getting established all the time" Which connection does not get established? The webhook connection to your backend or the websocket connection to your backend? Or do you get a webhook response, but failing to send a response via websocket?

It's the websocket connection which is not getting established. Webhook response is a simple Post request and it's coming fine. The problem here is sometimes this webhook response comes to that particular instance where the websocket connection was not established with the client.Ideally I would be happy if this response comes to both of the instance through some configuration on loadbalancer.

Easiest way is to check on the client side if the websocket connection is established, if not, just try to reconnect after a couple of seconds. Maybe there is a problem on the backend that dns is faster than establishing a new instance, so retrying is your best option.

Re: Ask HN: How do you scale WebSocket?

#14

Post by blablablub »

ravirajx7 wrote: post #10
blablablub wrote: post #6

" the connection is not getting established all the time" Which connection does not get established? The webhook connection to your backend or the websocket connection to your backend? Or do you get a webhook response, but failing to send a response via websocket?

It's the websocket connection which is not getting established. Webhook response is a simple Post request and it's coming fine. The problem here is sometimes this webhook response comes to that particular instance where the websocket connection was not established with the client.Ideally I would be happy if this response comes to both of the instance through some configuration on loadbalancer.

As per your other answer, websocket connections are established just fine

"Now, whenever a new Webhook response comes ... it doesn't have information regarding which instance was used earlier for making the websocket connection, it may send request to one of the instance where the connection was not established and thus our backend is not able to process this request from this particular instance."

just include the internal ip of the websocket connection to the data you send to your billing operator and then forward the post request appropiately.

Re: Ask HN: How do you scale WebSocket?

#15

Post by toast0 »

From what you've commented elsewhere, it sounds like the immediate question is how do you route an external event (webhook) to the right websocket.

You need to include information that comes back in the event that identifies the websocket, either directly (server id + an id the server would understand) or indirectly (something that you can look up in a table/hash to get direct information).

More likely than not, you'll actually want to do the indirect option with something that identifies the browser (session id?) in case the browser reconnects to the websocket before the event comes in. Connectivity is fragile, and clients roam across wifi access points or between wifi and lte, or sometimes between lte boundaries that necessitate different IPs, or sometimes their modem is reset while they wait and they get a new IP, etc. Or something in their network path hates the world and closes idle connections on a very aggressive timeout, or closes connections after a short timeout regardless of idle. Lots of scenarios where a reconnection is likely.

Finally, since you asked about horizontal scaling, my best advice is to scale vertically first. It's usually simpler to manage one server than can do 1M connections than 1k servers that can each do 1k connections. Depending on details, less servers, but larger can be less expensive than many smaller servers; although that changes when your large servers start getting exotic, more than two cpu sockets is a big inflection point, you most likely want to scale horizontally rather than get a quad socket monster (but they exist, and so do eight sockets)

Re: Ask HN: How do you scale WebSocket?

#16

Post by monroewalker »

ravirajx7 wrote: post #11
monroewalker wrote: post #8

I could be wrong, but I wouldn't think the autoscaling or load balancing would affect the websocket connection. There may be another aspect of the infrastructure that's preventing the connection though. Can you share more about the setup? Strange that the connection would succeed sometimes but not others.. This could be related to the configuration of some intermediate network layer. Eg. if Nginx is used, you may hav…

Hi, We're not using Nginx. We are using Docker image & everything is configured using cloudflare (whose details are something which I am not aware of properly). Though, here the problem with websocket is that they are stateful and whenever a connection is established it is directly getting established with one of the instance from the list of several instances due to loadbalancer. Now, whenever a new Webhook response…

One thing to consider would be Kafka. When a webhook comes in, you publish a message to your topic that includes some reference to the user such as the payment id or user id. Each server consumes from the topic and if it finds a message for which it has an active websocket (or long polling) connection, then it pushes that message back to the user. This page shows how to have all consumers consume all messages: https://stackoverflow.com/questions/23136500/how-kafka-broad.... Spring Boot has really convenient integration with Kafka which makes setup pretty straightforward. The AWS Kafka service is also quite easy to setup. Having all servers consume doesn't scale ideally since more payments and webooks will result in more kafka messages sent to each server, but partitioning would probably be too tricky with a variable number of servers due to autoscaling.

Another approach could be to save the association between the server and the session in a database. When a webhook comes in, if the current server doesn't have the target session, lookup which server does and make a request to an internal endpoint on that server to send the message over.

You could also look into Redis for this. Have the server which is handling the websocket subscribe to key changes for a key associated with the user's payment. When a webhook is received, just update that key in Redis

Re: Ask HN: How do you scale WebSocket?

#17

Post by unraveller »

https://medium.com/@14domino/using-nats-to-build-a-very-func...

no database, just have the user's websocket reach a simple websocket server which always sends on requests to a fuller API server which can speak back to the NATS server who triggers a push to the user since the websocket server is coupled to NATS. This gives horizontal scale to API servers (if they ignore users/work not for them) and websocket servers.

Post reply on HN