Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

600k concurrent websocket connections on AWS using Node.js (2015)

blog.jayway.com

Post reply on HN
Print view View original

131–140 of 141 posts

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#131

Post by windlep »

xyzzy_plugh wrote: post #116
windlep wrote: post #73

Earlier quoted context omitted.

We did not use an NLB because it is expensive in its pricing model for TLS connections (and we prefer to let AWS terminate TLS). For 20 million connections, an NLB would cost $28,800/month. It's drastically cheaper to use an ELB, provided you can talk to AWS to have them scale it for you.

Is it drastically cheaper to use an NLB without letting AWS terminate TLS? I thought AWS was expensive for TLS termination, generally, unless you need the client locality for lower latency.

Yes, if you terminate TLS its drastically cheaper. Without TLS, each NLB unit lets you have 100,000 concurrent connections. With TLS, a single unit is 3,000 connections. Similar pricing occurs with ALB's.

Terminating TLS yourself incurs some CPU cost and a bit more memory cost. How much CPU/memory is eaten depends on the efficiency of your code. Our Rust implementation roughly matches C code efficiency, so we could handle terminating TLS ourselves if ELB stops being feasible at some point.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#132

Post by windlep »

mnutt wrote: post #118
windlep wrote: post #73

Earlier quoted context omitted.

We did not use an NLB because it is expensive in its pricing model for TLS connections (and we prefer to let AWS terminate TLS). For 20 million connections, an NLB would cost $28,800/month. It's drastically cheaper to use an ELB, provided you can talk to AWS to have them scale it for you.

When I last discussed this with AWS, having them scale it for you meant that you actually had to contact them to pre-warm the ELB whenever you expected more traffic. It didn't work for our use cases where we didn't know when to expect more traffic.

Yup, that hasn't changed in our experience. If you're looking at bursty traffic beyond what AWS will pre-warm for you, then this likely won't work. We have fairly consistent loads throughout the day.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#133

Post by truth_seeker »

axismundi wrote: post #55

Does anyone have a more recent experience? I currently use socket.io 2.2 with node v10.16, no v8 tweaks in a docker container. At ~1000 sockets, sometimes the server receives spikes of 8000 HTTP reqs/sec, which it has to distribute to the websockets, up to 100 msgs/sec, ~1kb/msg to each socket. These spikes are making the server unstable, socket.io switches most of the clients from websockets to xhr polling.

Have you tried "sticky-cluster" ?

https://github.com/uqee/sticky-cluster

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#134

Post by deanward81 »

toast0 wrote: post #86
deanward81 wrote: post #69

Earlier quoted context omitted.

We’re spread over 9 servers but only due to ephemeral port and handle exhaustion issues. Each server is fronted by 4 HAProxy frontends that each handle ~18k connections. Since Nick’s post we’ve moved from StackExchange.NetGain to the managed websocket implementation in .NET Core 3 using Kestrel and libuv. That sits at around 2.3GB RAM and 0.4% CPU. Memory could be better (it used to be We could run on far fewer machi…

Ephemeral port exhaustion is easy to handle if you control HAProxy and the origins. You'll need the source [1] option on your server lines, and you also need to adjust to allow more connections, one of these will do: have the origin server listen on more ports, add more ips to the origin server and listen to those too, add more ips to the proxy and use those to connect as well. I'm not sure about handle exhaustion? I…

I might be missing some details there to be honest; I'm more familiar with the application side of things than the infrastructure :)

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#135

Post by toredash »

dylanz wrote: post #17
sciurus wrote: post #15

A few pieces of advice based on running https://github.com/mozilla-services/autopush-rs , which handles tens of millions of concurrent connections across a fleet of small EC2 instances. 1) Consider not running the largest instance you need to handle your workload, but instead distributing it across smaller instances. This allows for progressive rollout to test new versions, reduces the thundering herd when you restar…

Could you explain or post a link to something about #2? I’ve never heard of that before!

This is what you need to know: "Not all flows of traffic are tracked. If a security group rule permits TCP or UDP flows for all traffic (0.0.0.0/0) and there is a corresponding rule in the other direction that permits all response traffic (0.0.0.0/0) for all ports (0-65535), then that flow of traffic is not tracked. The response traffic is therefore allowed to flow based on the inbound or outbound rule that permits the response traffic, and not on tracking information."

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#136

Post by chatmasta »

toredash wrote: post #135
dylanz wrote: post #17

Earlier quoted context omitted.

Could you explain or post a link to something about #2? I’ve never heard of that before!

This is what you need to know: "Not all flows of traffic are tracked. If a security group rule permits TCP or UDP flows for all traffic (0.0.0.0/0) and there is a corresponding rule in the other direction that permits all response traffic (0.0.0.0/0) for all ports (0-65535), then that flow of traffic is not tracked. The response traffic is therefore allowed to flow based on the inbound or outbound rule that permits t…

Very interesting. It makes a lot of sense from a Linux perspective. The conntrack table requires memory, so there is some physical limit on its size. That also explains why it scales per instance type.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#137

Post by arrty88 »

song wrote: post #90
arrty88 wrote: post #42

Earlier quoted context omitted.

You can also employ DNS round robin with health checks in Route53

DNS round robin has the disadvantage of not guaranteeing that the traffic will be equally distributed. This can be a problem if there's a lot of automated traffic. But, with that said, in term of price, it's unbeatable.

I have had success DNS round robin-ing a cluster of HAproxy LBs which handle SSL and then proxy to my backend websocket gateways which keeps the real load evenly distributed where it matters and also run health checks on the nodes.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#138

Post by toredash »

chatmasta wrote: post #136
toredash wrote: post #135

Earlier quoted context omitted.

This is what you need to know: "Not all flows of traffic are tracked. If a security group rule permits TCP or UDP flows for all traffic (0.0.0.0/0) and there is a corresponding rule in the other direction that permits all response traffic (0.0.0.0/0) for all ports (0-65535), then that flow of traffic is not tracked. The response traffic is therefore allowed to flow based on the inbound or outbound rule that permits t…

Very interesting. It makes a lot of sense from a Linux perspective. The conntrack table requires memory, so there is some physical limit on its size. That also explains why it scales per instance type.

This is also why "endless scaling" is a fud. At some point or some level, there is either a physical, soft, standard or some kind of limit

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#139

Post by winrid »

codesushi42 wrote: post #84
Thaxll wrote: post #31

Earlier quoted context omitted.

This is actually not true, if you take a really optimzied C/C++ library for WS Nodejs will crush Elixir by a large margin. Elixir / Erlang is slow and consume a lot of memories compared to more native languages or C libraries. Ex: https://github.com/uNetworking/uWebSockets

At that point, you may as well implement your own server in C++ using epoll.

Nah. Devs don't touch C++ code in this case.

Re: 600k concurrent websocket connections on AWS using Node.js (2015)

#140

Post by winrid »

PunchTornado wrote: post #122
mtberatwork wrote: post #89

Earlier quoted context omitted.

If we are going on size of ecosystem/what companies are invested in/etc, then you may as well stick with Java and the JVM with something like Vert.x. No need for Node.

nodejs ecosystem is much larger than the java one. also the amount of money companies invested in node is at least on par with java. think only about google chrome.

There's no advantage to using Node in a lot of cases though. Maybe if you're creating a microservice to generate email html or something. Data/state management, stream/job processing? Java or Kotlin all the way.

I have written a lot of Node services. Spent six years doing it. Gimme Java plz.

Post reply on HN