Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

600k concurrent websocket connections on AWS using Node.js (2015)

blog.jayway.com

Post reply on HN
Print view View original

81–90 of 141 posts

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#81

Post by verttii »

gravypod wrote: post #56
sciurus wrote: post #15

A few pieces of advice based on running https://github.com/mozilla-services/autopush-rs , which handles tens of millions of concurrent connections across a fleet of small EC2 instances. 1) Consider not running the largest instance you need to handle your workload, but instead distributing it across smaller instances. This allows for progressive rollout to test new versions, reduces the thundering herd when you restar…

> 1) Consider not running the largest instance you need to handle your workload, but instead distributing it across smaller instances. This allows for progressive rollout to test new versions, reduces the thundering herd when you restart or replace an instance, etc. I came here to say this. Horizontal compute is a miracle.

Distribution itself might be a world of pain depending on the nature of your application. For example stateful apps where low latency is expected.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#82

Post by PopeDotNinja »

Thaxll wrote: post #31
anildigital wrote: post #19

I don't get point of using Node.js when compared to something like Elixir. Elixir's Phoenix can handle more numbers of concurrent connections as well as provide reliability with better programming abstractions, distribution, pretty good language.

This is actually not true, if you take a really optimzied C/C++ library for WS Nodejs will crush Elixir by a large margin. Elixir / Erlang is slow and consume a lot of memories compared to more native languages or C libraries. Ex: https://github.com/uNetworking/uWebSockets

Isn't this like saying one can bolt a rocket onto a Ford to make it much faster than a Toyota? One can incorporate C/C++ into Elixir, too.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#83

Post by jsd1982 »

cpursley wrote: post #52
truth_seeker wrote: post #49

Earlier quoted context omitted.

Not fair ! You are comparing "4 CPUs and 15GB of memory" for NodeJS with "40 CPUs and 128 GB of memory" for Elixir/Phoenix

Ok, here's a better comparison: "Websocket Shootout: Clojure, C++, Elixir, Go, NodeJS, and Ruby" https://hashrocket.com/blog/posts/websocket-shootout

That benchmark is from 2016, but the use of `golang.org/x/net/websocket` package for Go should be replaced with `github.com/gorilla/websocket` and re-run, if it hasn't been already. Based on their git repo, gorilla/websocket has existed in some form since at least 2013 though not sure how production-ready it would've been in 2016.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#84

Post by codesushi42 »

Thaxll wrote: post #31
anildigital wrote: post #19

I don't get point of using Node.js when compared to something like Elixir. Elixir's Phoenix can handle more numbers of concurrent connections as well as provide reliability with better programming abstractions, distribution, pretty good language.

This is actually not true, if you take a really optimzied C/C++ library for WS Nodejs will crush Elixir by a large margin. Elixir / Erlang is slow and consume a lot of memories compared to more native languages or C libraries. Ex: https://github.com/uNetworking/uWebSockets

At that point, you may as well implement your own server in C++ using epoll.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#85

Post by truth_seeker »

deorder wrote: post #72
truth_seeker wrote: post #54

Earlier quoted context omitted.

> NodeJS performance is hampered by its single-threaded architecture, but given that fact it performs very well. The NodeJS server was the smallest overall in lines of code and extremely quick and easy to write. Javascript is the lingua franca for web development, so NodeJS is probably the easiest platform for which to hire or train developers. Do you actually read the blog post before making the claim ? The author i…

It seems that the author actually used the "cluster" module, but just does not mention it in the article: https://github.com/hashrocket/websocket-shootout/blob/master... I expected all benchmarks that use the (linux) kernel for I/O events would perform similarly. The code is so small that the interpreter overhead should not degrade the performance too much. May be the JSON parsing / serialization?

So you think that "sticky-session" is not important ? Can you tell why ?

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#86

Post by toast0 »

deanward81 wrote: post #69
zaphirplane wrote: post #13

Earlier quoted context omitted.

Surely not on one endpoint

We’re spread over 9 servers but only due to ephemeral port and handle exhaustion issues. Each server is fronted by 4 HAProxy frontends that each handle ~18k connections. Since Nick’s post we’ve moved from StackExchange.NetGain to the managed websocket implementation in .NET Core 3 using Kestrel and libuv. That sits at around 2.3GB RAM and 0.4% CPU. Memory could be better (it used to be We could run on far fewer machi…

Ephemeral port exhaustion is easy to handle if you control HAProxy and the origins.

You'll need the source [1] option on your server lines, and you also need to adjust to allow more connections, one of these will do: have the origin server listen on more ports, add more ips to the origin server and listen to those too, add more ips to the proxy and use those to connect as well.

I'm not sure about handle exhaustion? I've run into file descriptor limits, those are usually simple to set (until you run into a code enforced limit)

[1] https://cbonte.github.io/haproxy-dconv/1.7/configuration.htm...

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#87

Post by dgelks »

axismundi wrote: post #55

Does anyone have a more recent experience? I currently use socket.io 2.2 with node v10.16, no v8 tweaks in a docker container. At ~1000 sockets, sometimes the server receives spikes of 8000 HTTP reqs/sec, which it has to distribute to the websockets, up to 100 msgs/sec, ~1kb/msg to each socket. These spikes are making the server unstable, socket.io switches most of the clients from websockets to xhr polling.

I found this article very useful to resolve connections moving to xhr polling on our Node.js 10.16 server https://medium.com/@k1d_bl4ck/a-quick-story-about-node-js-so...

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#88

Post by tschellenbach »

anildigital wrote: post #19

I don't get point of using Node.js when compared to something like Elixir. Elixir's Phoenix can handle more numbers of concurrent connections as well as provide reliability with better programming abstractions, distribution, pretty good language.

You could make the same argument comparing Elixir and Go, or Go and C++

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#89

Post by mtberatwork »

monus21 wrote: post #30
anildigital wrote: post #19

I don't get point of using Node.js when compared to something like Elixir. Elixir's Phoenix can handle more numbers of concurrent connections as well as provide reliability with better programming abstractions, distribution, pretty good language.

It's not about which is the best technology. The barrier to entry for Node.js is next to nothing and the size of the ecosystem is incomparable to Elixir. Which means tons of companies will go with Node and feed back to the ecosystem and so on...

If we are going on size of ecosystem/what companies are invested in/etc, then you may as well stick with Java and the JVM with something like Vert.x. No need for Node.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#90

Post by song »

arrty88 wrote: post #42
Thaxll wrote: post #36

Earlier quoted context omitted.

If you have a lot of trafic you shoudn't use an ELB in the first place, should be using an NLB by default it comes with a 40Gb pipe and multi millions connections. https://docs.aws.amazon.com/elasticloadbalancing/latest/netw...

You can also employ DNS round robin with health checks in Route53

DNS round robin has the disadvantage of not guaranteeing that the traffic will be equally distributed. This can be a problem if there's a lot of automated traffic.

But, with that said, in term of price, it's unbeatable.

Post reply on HN