You could have used minimal apis in C# and gotten a better throughput
Programming language comparison by reimplementing the same transit data app
21–30 of 65 posts
Re: Programming language comparison by reimplementing the same transit data app
#22Oh, hey, didn't expect to see this here. Thanks for submitting! I tried a few days ago but didn't really get any traction.[0] Since I submitted it, though, I posted it to /r/rust and got a lot of feedback. At the time, rust and dotnet were comparable and at the top of the list, with ~10k req/sec. Now rust is far and away the most performant at ~20k req/sec! I also was able to improve Go's performance 30% or so. Still…
The elixir implementation seems super slow. The first thing that jumps out to me is that we should be using streams instead of just reading the file. The second would be the `schedule_for_route` which should just be a GenServer lookup instead of reading from ets.
Yeah, I should benchmark streams. In my experience streams are rarely faster, though, and are instead something you should reach for if 1) you can't fit the file in memory or 2) you might stop the enumeration early.
Regarding GenServer, I'm not so sure about that. I suppose I should benchmark it, but intuitively I expect ETS to be better here. There's more overhead in getting the data, but it allows you to concurrently read it from different processes. A GenServer, meanwhile, could respond to a given message faster, but now you're serializing the (e.g.) 50 concurrent virtual users through a single bottleneck. I could have multiple copies of the data in several GenServers, I suppose, at the expense of much more memory use.
Re: Programming language comparison by reimplementing the same transit data app
#23Earlier quoted context omitted.
> I'm finding I can almost double my requests per second from simply switching the JSON encoder from Jason to Jiffy Personal plug, but have you tried Jsonrs[0]? I typically get much better performance out of it (especially in lean mode, which seems to be the mode you'd want to use for this benchmark) than Jiffy for large JSON encoding workloads. [0] https://hexdocs.pm/jsonrs/readme.html
This is great! Just pushed up a commit that uses it and updated the benchmarks[0]. I'm seeing a 1.6X - 2X improvement in overall performance. Not bad for a drop-in replacement. And since it's based on serde, I trust it, and I feel like trying out a different JSON library is within scope for me of not just "gaming the benchmarks", as this is actually something I'd now consider using at work. It's not quite as high as…
One major difference you'll run into here is support for encoding protocols. Jason, Poison, and Jsonrs (by default) will have protocol impls for Elixir structs that define special logic for how to serialize those structs into JSON strings. ie, you probably want a DateTime struct to serialize as a date string rather than a map containing the year, month, etc. Jiffy doesn't support any of that, so you end up with differing behavior between it and other libraries as soon as you start encoding structs that aren't meant to serialize as maps.
Jsonrs supports encoding protocols by default, but lets you turn it off for a speed boost if you're okay with more Jiffy-like behavior. Plugging it in as Phoenix's JSON library won't turn off that protocol handling, so you're getting the slower operational mode of Jsonrs (which is fine and probably what you want in most cases for correctness, but will definitely eat a few ops/sec in a benchmark)
Re: Programming language comparison by reimplementing the same transit data app
#24Earlier quoted context omitted.
The elixir implementation seems super slow. The first thing that jumps out to me is that we should be using streams instead of just reading the file. The second would be the `schedule_for_route` which should just be a GenServer lookup instead of reading from ets.
Yeah, I should benchmark streams. In my experience streams are rarely faster, though, and are instead something you should reach for if 1) you can't fit the file in memory or 2) you might stop the enumeration early. Regarding GenServer, I'm not so sure about that. I suppose I should benchmark it, but intuitively I expect ETS to be better here. There's more overhead in getting the data, but it allows you to concurrent…
I would guess that there's too much work being done at runtime. I would expect all of the data to be cached in such a way that it is a read operation without any transformation. It's doing multiple lookups and unnecessary maps when the data should just be formatted a single time and cached. You could even skip the JSON transformation by just doing it once if you want to get super fast.
Re: Programming language comparison by reimplementing the same transit data app
#25For a similar project, someone put together a spec for a basic clone of Medium split into a frontend design and backend API, and now has over 100 different components where you can pair any frontend with any backend (plus a few fullstack implementations).
https://github.com/gothinkster/realworld
Not as performance-focused with benchmarks, but a good point of comparison for various languages and frameworks implementing common behavior.
Re: Programming language comparison by reimplementing the same transit data app
#26Interesting results. Do you think it would be interesting /possible to try sqlite as a pure in memory db using :memory db file?
Re: Programming language comparison by reimplementing the same transit data app
#27The experience report on Scala I find pretty cathartic. It really is absolutely ridiculous the fetishization of extremely complex FP and type-level hacking that goes on in the ecosystem, to the point where, in the case of the author's hello world web server snippet, it's just so complex and laden with concepts you need to know that are unrelated to the problem you're trying to solve it could be mistaken for a parody…
The Scala 3 requirement really does cripple the choices available. The Scala 3 transition does feel a bit like the Python 3 transition where major libraries and frameworks are really dragging their feet on transitioning because of the high level of effort for very little obvious improvement. Scala 2 is still getting regular updates so there's no rush to use 3 just yet, I'd recommend picking the framework first and us…
In the long term it will be an improvement. The one thing dragging adoption down is the removal of the old macro system, which depended completely on the implementation details of the old compiler. That said, I agree that Scala 2 is still very much supported, so there's nothing wrong with picking it.
Re: Programming language comparison by reimplementing the same transit data app
#28The experience report on Scala I find pretty cathartic. It really is absolutely ridiculous the fetishization of extremely complex FP and type-level hacking that goes on in the ecosystem, to the point where, in the case of the author's hello world web server snippet, it's just so complex and laden with concepts you need to know that are unrelated to the problem you're trying to solve it could be mistaken for a parody…
I agree with the sentiment that the more FP-heavy parts of the Scala ecosystem are not suited to hello-world code _at all_. In this particular case, http4s is at its core a low-level library. The author was right to go for Play first.
But it turns out there's an alrernative. Tapir is emerging as that "pure FP, but approachable" HTTP framework. It allows developers to work at a higher level, turning the HTTP backend into an implementation detail. Simply define the application using Tapir, then choose the better backend for the use case.
Re: Programming language comparison by reimplementing the same transit data app
#29How do you think Node.js would fare here, similar to Deno?
Re: Programming language comparison by reimplementing the same transit data app
#30You mention Deno maybe not handling multiple cores “well”.
I’m 99.9% sure Deno doesn’t handle multiple cores at all. Everything in the node ecosystem is one big asynchronous single-threaded event loop.