It's not difficult to see that such amount of connections is only useful if your server performs a trivial task. Otherwise, the bottleneck will be your CPU(s).
600k concurrent websocket connections on AWS using Node.js (2015)
111–120 of 141 posts
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#112Does anyone know what latency overhead switching to ECS/EKS would add?
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#113Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#114A better solution would be to use nchan+nginx and then your Node API is just a simple stateless REST service. Will scale better and be easier to maintain.
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#115Earlier quoted context omitted.
> 1) Consider not running the largest instance you need to handle your workload, but instead distributing it across smaller instances. This allows for progressive rollout to test new versions, reduces the thundering herd when you restart or replace an instance, etc. I came here to say this. Horizontal compute is a miracle.
How meaningful is the benchmark for handling idle connections? Handling more is better, of course, but if the server melts down when 0.1% of them have any activity, maybe maximizing idle connections isn't the right place to expend optimization effort?
For us, the experiment (https://blog.mozilla.org/services/2018/10/03/upcoming-push-s...) with mostly idle clients was very helpful, since it flushed out problems with out load balancing layer and in the end gave us confidence that it, our app, and our persistence layer could safely handle many more connections.
If we had begun having problems once we started sending more push messages, we would have simply stopped using the new service (https://github.com/mozilla-services/megaphone) responsible for that until we worked through them.
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#116Earlier quoted context omitted.
If you have a lot of trafic you shoudn't use an ELB in the first place, should be using an NLB by default it comes with a 40Gb pipe and multi millions connections. https://docs.aws.amazon.com/elasticloadbalancing/latest/netw...
We did not use an NLB because it is expensive in its pricing model for TLS connections (and we prefer to let AWS terminate TLS). For 20 million connections, an NLB would cost $28,800/month. It's drastically cheaper to use an ELB, provided you can talk to AWS to have them scale it for you.
Is it drastically cheaper to use an NLB without letting AWS terminate TLS? I thought AWS was expensive for TLS termination, generally, unless you need the client locality for lower latency.
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#117Earlier quoted context omitted.
Ok, here's a better comparison: "Websocket Shootout: Clojure, C++, Elixir, Go, NodeJS, and Ruby" https://hashrocket.com/blog/posts/websocket-shootout
That benchmark is from 2016, but the use of `golang.org/x/net/websocket` package for Go should be replaced with `github.com/gorilla/websocket` and re-run, if it hasn't been already. Based on their git repo, gorilla/websocket has existed in some form since at least 2013 though not sure how production-ready it would've been in 2016.
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#118Earlier quoted context omitted.
If you have a lot of trafic you shoudn't use an ELB in the first place, should be using an NLB by default it comes with a 40Gb pipe and multi millions connections. https://docs.aws.amazon.com/elasticloadbalancing/latest/netw...
We did not use an NLB because it is expensive in its pricing model for TLS connections (and we prefer to let AWS terminate TLS). For 20 million connections, an NLB would cost $28,800/month. It's drastically cheaper to use an ELB, provided you can talk to AWS to have them scale it for you.
When I last discussed this with AWS, having them scale it for you meant that you actually had to contact them to pre-warm the ELB whenever you expected more traffic. It didn't work for our use cases where we didn't know when to expect more traffic.
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#119A few pieces of advice based on running https://github.com/mozilla-services/autopush-rs , which handles tens of millions of concurrent connections across a fleet of small EC2 instances. 1) Consider not running the largest instance you need to handle your workload, but instead distributing it across smaller instances. This allows for progressive rollout to test new versions, reduces the thundering herd when you restar…
Could you explain or post a link to something about #2? I’ve never heard of that before!
I can corroborate this, at Verizon we had problems scaling out ELBs for a very similar use case.
Re: 600k concurrent websocket connections on AWS using Node.js (2015)
#120I have written and deployed message queues in Node.js that take data from Redis and push it out on websockets. It is a pain to deal with the GC in weird cases. This was about 5 years ago, so the details might not accurate.
Things worked fine until some day some customer started sending multi-megabyte strings over the system. It is difficult to actually track down that it is GC that is halting the system and then figuring out ways to fix the issue. We ended up not using JavaScript strings and instead using Node.js buffers to do the transport - I don't recall the Node.js library for Redis supporting that out of the box.