Earlier quoted context omitted.
Or `Cowboy` with `alias :cowboy, as: Cowboy` ;)
Stick with the :module notation. This makes it clear that you need to be in "erlang mode"...indexes starting at 1, [probably] charlists instead of binaries, and possibly weird (from an elixir programmer's point of view) argument order.
Achieving 100k connections per second with Elixir
101–110 of 117 posts
Re: Achieving 100k connections per second with Elixir
#102Earlier quoted context omitted.
Not na expert in any of the languages by any means, but Go and Erlanger/Elixir focus on different things: - Go wants to be performant at high concurrency scale - Erlang/Elixir wants to keep running at high concurrency scales, whatever the issues are in your application code. Performance comes second. There's no clear cut answer to your question; I guess if you trust yourself to write servers that will hold a large nu…
> Performance comes second. I want to state that performance is too generalize here. BEAM VM also have a goal of low latency which can be consider as performance. I'm not entirely sure if GO is aiming for that or not. I would never do any numerical stuff on BEAM though, it's very slow. This article is a bit dated but is interesting between Go and Erlang: https://www.theerlangelist.com/article/reducing_maximum_late...
Re: Achieving 100k connections per second with Elixir
#103I can handle 120k connections per second with my custom made, highly optimized multiprocess C++ server. But the main problem is business logic. Just make 2 SQL queries to MySQL on each HTTP request and look at how it will degrade.
There are simple tricks to make those queries not kill performance. Here is a dumb proof-of-concept I made a few months ago: https://github.com/MatthewSteel/carpool The general idea is combining queries from different HTTP requests into a single database query/transaction, amortising the (significant) per-query cost over those requests. For simple use-cases it doesn't add a whole lot of complexity, can reduce both lo…
E.g. please give me guidance on how to better structure my database model so that it doesn't effectively end up as a huge spaghetti heap of global variables. My personal horror: updating a single database field spurs 20 additional SQL queries creating several new rows in seemingly unrelated tables. Digging in I find this was due to an after_save hook in the database model which created an avalanche of other after_save/after_validation hooks to fire. The worst of it: Asking for how this has come to be I find out that each step of the way was an elegant solution to some code duplication in the controller, some forgotten edge case in the UI, some bug in the business logic. Basically ending up with extremely complex control flows is the default.
So of course, if your code has next to no isolation, batching up queries produces incalculable risks.
/rant, sorry.
Re: Achieving 100k connections per second with Elixir
#104I'd love to see data on the average on-call incidents for an application written in language X (say Go) vs those written in Elixir. Concretely, its it the case, for an application where Elixir/Erlang/Beam are a great choice, but also, another language would be fine, that the equivalent Elixir application results in less downtime/pages than the alternative. Anything from the perfect app to something with a ton of race…
> I'd love to see data on the average on-call incidents Don't have any hard data to compare but having been involved in debugging running Erlang systems. It's very nice having the ability to restart separate supervisors while the rest of the processes handle requests. Being able to do hot code loading to say fix bugs or add extra logging. And my all time favorite -- live tracing after connecting to a VM's remote shel…
Re: Achieving 100k connections per second with Elixir
#105Is Elixir/Erlang considered superior to Go for writing high concurrency web servers?
What does high concurrency mean? Both of them give you less flexibility than is necessary to achieve highly efficient use of all threads on a multiprocessor system. For that, you'll need something like a pool of event loops using async/await. This is the system most common in high performance networking in C++, C, and Rust. Erlang and Go both sacrifice efficiency to improve maintainability and safety by offering a mo…
The question is if you cannor want to write better concurrent code by rolling it yourself.
Re: Achieving 100k connections per second with Elixir
#106Earlier quoted context omitted.
There are simple tricks to make those queries not kill performance. Here is a dumb proof-of-concept I made a few months ago: https://github.com/MatthewSteel/carpool The general idea is combining queries from different HTTP requests into a single database query/transaction, amortising the (significant) per-query cost over those requests. For simple use-cases it doesn't add a whole lot of complexity, can reduce both lo…
Looks interesting! You mentioned in the docs that it would be simpler once abstractions develop and that made me realize it's similar to facebook/dataloader, just used across requests instead of batching up all of the queries per request. It's also of course a generalized form of it that represents batching a parametrized method more so than just batching retrievals by some kind of unique key. It may be able to serve…
Re: Achieving 100k connections per second with Elixir
#107Earlier quoted context omitted.
There are simple tricks to make those queries not kill performance. Here is a dumb proof-of-concept I made a few months ago: https://github.com/MatthewSteel/carpool The general idea is combining queries from different HTTP requests into a single database query/transaction, amortising the (significant) per-query cost over those requests. For simple use-cases it doesn't add a whole lot of complexity, can reduce both lo…
Since looking into modern concurrency concepts I've always thought such (in my opinion obvious) batching should be part of sophisticated ORM frameworks such as Rails' Active Records. Alas, their design decisions always seem to cater for making the dumb usages more performant (sometimes automagically, sometimes adding huge layers of cruft) than rewarding programmers who are willing to learn a few concepts by creating…
One mitigating factor, this sort of optimisation should be applied to frequent queries more than expensive queries. In some use-cases the former kind may be simple ("Is this user logged in?") even if the latter is not.
And on keeping that complexity down: the traditional story has been "normalise until you only need to update data in one place," but often requirements don't line up well to foreign-key constraints etc. The newer story can work, though: "Denormalise until you only have to update in one place, shunt the complexity to user code, and serialise writes." It's anathema to many, but it is becoming more common (usually in places that don't use RBDMSs though.)
Re: Achieving 100k connections per second with Elixir
#108im a simple man. i see Elixir, i upvote. that being said, this article was pretty informative. The bit about the proposed SO_REUSEPORT socket option was really interesting. Really fun to read about performance bottleneck detection and improvement. edit: wow, downvoting for making a simple joke about liking elixir. Cool.
Re: Achieving 100k connections per second with Elixir
#109im a simple man. i see Elixir, i upvote. that being said, this article was pretty informative. The bit about the proposed SO_REUSEPORT socket option was really interesting. Really fun to read about performance bottleneck detection and improvement. edit: wow, downvoting for making a simple joke about liking elixir. Cool.
I've found that humor in comments on HN is usually not well received. Not sure why, just an observation.
HUMOR? HUMOR HAS NO PLACE HERE! THIS IS A FORUM OF INTELLECTUAL DISCUSSION!
Re: Achieving 100k connections per second with Elixir
#110im a simple man. i see Elixir, i upvote. that being said, this article was pretty informative. The bit about the proposed SO_REUSEPORT socket option was really interesting. Really fun to read about performance bottleneck detection and improvement. edit: wow, downvoting for making a simple joke about liking elixir. Cool.
I've found that humor in comments on HN is usually not well received. Not sure why, just an observation.