Achieving 100k connections per second with Elixir • Hacker News Board

Re: Achieving 100k connections per second with Elixir

#71

Post by Thaxll » Tue, Mar 05, 2019, 10:07 PM UTC

StreamBright wrote: post #64

alexgaribay wrote: post #60

Earlier quoted context omitted.

In terms of raw performance, Go will be faster. However, the differentiator here is that the BEAM gives you the guarantees and tools to write highly concurrent applications with a sane mental model, fault-tolerance, and isolated processes. As a sibling said, it's fairly easy cluster applications. Additionally, if something truly needs to be ran in another language for performance, you can write a NIF in Rust or somet…

Highly depends on which libraries are you talking about. Essentially you cannot claim that Go will be faster. You can say that fasthttp is faster than Cowboy for a hello world application. Every real world application is much more complex and the stack's performance will be decided by its slowest element.

Go is faster than Erlang and not by a small margin, Erlang is a dynamic and immutable language, it hurts performance compare to languages like Go.

Some benchmark about pure CPU computation:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Erlang is really slower than Java

Go and Java now:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

You see the big difference, Go and Java are on part but Go usually takes way less memory than Java.

You can't have everything, message passing / immutability with no performance hit.

Re: Achieving 100k connections per second with Elixir

#72

Post by keithnoizu » Tue, Mar 05, 2019, 10:25 PM UTC

dzik wrote: post #5

This article is quite good, especially part about bottleneck caused by single supervisor in ranch. However I have to say that title is a bit misleading because all of this has nothing to do with Elixir, it's all about Linux kernel and Erlang, cowboy and ranch are written in Erlang. Having said that, I will add that I think it is good to have Elixir.

And here i'm happy with my 8k requests per second Elixir IOT solution.

I use long lived processes and had to come up with some magic to work around the supervisor behavior with high child counts, etc.

Roughly I randomly assign worked to a node in the cluster if they have not yet been assigned. (there is some logic tracking total nodes on cluster and max / target that influences the decision). I verify if the remote (or local) worker is alive or migrating by checking a fragmented process Registry and unique identifier via :rpc (because I do some recovery logic if it's offline and let the caller specify if messages should force a spawn or can be ignored if the process is offline) and then pad the call with some meta information for rerouting so the receiver can confirm it is the same process the message was initially sent to (since processes cycle so frequently that the initial process may have died and a new process may have spawned in the time it took to forward the message).

If the process has changed mid transit or the worker has been flagged for migration the message gets rerouted to the new destination+process. If a process does not yet exist a hash based on the worker type + id and available worker pools is used to select which of the auto generated but named and registered (WorkType.PoolSupervisor_123) worker supervisors spawns the child node.

It's a trip, and needs to be heavily documented. Starting from scratch i'd probably change some things and it probably needs some refinement later this year before the next batch of 250-500k devices get added to the network, but the costs per reporting device are fantastic with plenty of low hanging fruit for improving the cogs further so I'm happy.

https://github.com/noizu/SimplePool/

Re: Achieving 100k connections per second with Elixir

#73

Post by alexgaribay » Tue, Mar 05, 2019, 10:26 PM UTC

StreamBright wrote: post #64

alexgaribay wrote: post #60

Earlier quoted context omitted.

In terms of raw performance, Go will be faster. However, the differentiator here is that the BEAM gives you the guarantees and tools to write highly concurrent applications with a sane mental model, fault-tolerance, and isolated processes. As a sibling said, it's fairly easy cluster applications. Additionally, if something truly needs to be ran in another language for performance, you can write a NIF in Rust or somet…

Highly depends on which libraries are you talking about. Essentially you cannot claim that Go will be faster. You can say that fasthttp is faster than Cowboy for a hello world application. Every real world application is much more complex and the stack's performance will be decided by its slowest element.

True. It's highly dependent on what is compared. I should have made my point more clear, however. I care more about what the BEAM provides over performance I can get in other language.

Re: Achieving 100k connections per second with Elixir

#74

Post by vasilia » Tue, Mar 05, 2019, 10:27 PM UTC

jamra wrote: post #69

vasilia wrote: post #68

I can handle 120k connections per second with my custom made, highly optimized multiprocess C++ server. But the main problem is business logic. Just make 2 SQL queries to MySQL on each HTTP request and look at how it will degrade.

That seems like a job for sharded databases and caches.

Of course, we have both, but 100k is nothing if it's not a CDN server which stores static file in-memory. Moreover, the main metric is latency, not a number of connections. You can scale a number of connections with an L3/L4 load balancer, but not latency.

Re: Achieving 100k connections per second with Elixir

#75

Post by shaklee3 » Tue, Mar 05, 2019, 10:43 PM UTC

Thaxll wrote: post #71

StreamBright wrote: post #64

Earlier quoted context omitted.

Highly depends on which libraries are you talking about. Essentially you cannot claim that Go will be faster. You can say that fasthttp is faster than Cowboy for a hello world application. Every real world application is much more complex and the stack's performance will be decided by its slowest element.

Go is faster than Erlang and not by a small margin, Erlang is a dynamic and immutable language, it hurts performance compare to languages like Go. Some benchmark about pure CPU computation: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Erlang is really slower than Java Go and Java now: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... You see the big difference, Go and Java are on part…

Instead of downvoting this comment, can someone write why this is incorrect? I realize it might not be a popular writing style, but references were provided.

Re: Achieving 100k connections per second with Elixir

#76

Post by ramchip » Tue, Mar 05, 2019, 10:54 PM UTC

lstodd wrote: post #44

dzik wrote: post #40

Earlier quoted context omitted.

That's the thing. You can always hack something in C to prove there is a better way for a specific task. In the past I did things like that just for fun. But in the real world it does not work like that. You buy into things as a whole, accepting their pros and cons as a whole. If you need to hack - change your tools.

Please do not beat the strawman. And don't set him on fire. He's innocent. I offered to beat whatever you've done by tweaking the Py3 stdlib. Not by writing a plain C implementation. If you for some reason doubt that this old python thing is of the real world - let me disappoint you. It was done because nothing else could do those 100K rps back then. And it did the thing for five years, until the whole stack was ditc…

I think you’re misinterpreting the point of the article. It’s not gloating about how much they scale, or saying their particular tech beats other techs. It’s just explaining how to solve a specific scaling issue on a specific platform.

As an Elixir user who had to deal with high connections/s in the past, I found it interesting and useful. I use Elixir for reasons that have nothing to do with performance so a language comparison isn’t particularly interesting.

Re: Achieving 100k connections per second with Elixir

#77

Post by RobertKerans » Tue, Mar 05, 2019, 11:04 PM UTC

shaklee3 wrote: post #75

Thaxll wrote: post #71

Earlier quoted context omitted.

Go is faster than Erlang and not by a small margin, Erlang is a dynamic and immutable language, it hurts performance compare to languages like Go. Some benchmark about pure CPU computation: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Erlang is really slower than Java Go and Java now: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... You see the big difference, Go and Java are on part…

Instead of downvoting this comment, can someone write why this is incorrect? I realize it might not be a popular writing style, but references were provided.

Because it's not hugely relevant. Yes, Erlang is not a "fast" language by many metrics, but that [very often] is not the reason a team would choose to use it. The sibling and parent comments make up a fairly considered discussion regarding this. A comment in the middle of this conversation saying Go etc win in some benchmark game is a non sequitur

Re: Achieving 100k connections per second with Elixir

#78

Post by rargulati » Tue, Mar 05, 2019, 11:15 PM UTC

I'd love to see data on the average on-call incidents for an application written in language X (say Go) vs those written in Elixir.

Concretely, its it the case, for an application where Elixir/Erlang/Beam are a great choice, but also, another language would be fine, that the equivalent Elixir application results in less downtime/pages than the alternative. Anything from the perfect app to something with a ton of races/leaks.

Is this a fair question (maybe I'm presuming too much of BEAM/supervisor pattern, I zero experience with it)?

Re: Achieving 100k connections per second with Elixir

#79

Post by adontz » Tue, Mar 05, 2019, 11:19 PM UTC

ioquatix wrote: post #48

adontz wrote: post #46

Earlier quoted context omitted.

With Python/uvloop I can easily get 10K-12K connections per second per core, so 36 cores will be fine with Python too.

Can you show me your example code? Also, assuming it scales up linearly is a bit risky, although I agree with that kind of conn/s I am sure it will be sufficient.

Nothing special, just usual asyncio Protocol with uvloop policy.

Re: Achieving 100k connections per second with Elixir

#80

Post by keithnoizu » Tue, Mar 05, 2019, 11:20 PM UTC

Thaxll wrote: post #71

StreamBright wrote: post #64

Earlier quoted context omitted.

Highly depends on which libraries are you talking about. Essentially you cannot claim that Go will be faster. You can say that fasthttp is faster than Cowboy for a hello world application. Every real world application is much more complex and the stack's performance will be decided by its slowest element.

Go is faster than Erlang and not by a small margin, Erlang is a dynamic and immutable language, it hurts performance compare to languages like Go. Some benchmark about pure CPU computation: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Erlang is really slower than Java Go and Java now: https://benchmarksgame-team.pages.debian.net/benchmarksgame/... You see the big difference, Go and Java are on part…

Elixir/Erlang is slower than java for computationally expensive non tail optimized large struct operations maybe. For simple requests with ginormous thread counts the point of failure is going to get hit way sooner with Java than Erlang. If connecting directly to ETS/Mnesia/Dets instead of converting back and forth between an external persistence layer Erlang/Elixir can be amazingly fast. Doing Map Reduce type operations or performing massive data processing tasks in parallel Erlang/Elixirs is likely going to outperform Java. Anywhere where thread overhead is more critical than raw per thread performance Erlang/Elixir can perform better than alternative solutions.