https://eli.thegreenplace.net/2018/measuring-context-switchi...
it's not a good enough example of a "task" to reflect real world usage.
How much memory do you need in 2024 to run 1M concurrent tasks?
81–90 of 205 posts
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#82> high number of concurrent tasks can consume a significant amount of memory note absolute numbers here: in the worst case, 1M tasks consumed 2.7 GB of RAM, with ~2700 bytes overhead per task. That'd still fit in the cheapest server with room to spare. My conclusion would be opposite: as long as per-task data is more than a few KB, the memory overhead of task scheduler is negligible.
Except it’s more than that. Go and Java maintain a stack for every virtual thread. They are clever about it, but it’s very possible that doing anything more than a sleep would have blown up memory on those two systems.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#83Earlier quoted context omitted.
Is that the ideomatic way to do it, or the best way you can imagine?
> Is that the ideomatic way to do it Well... I'm actually not sure what ideomatic means (English isn't my first language), but it's the standard way of doing it. You'll even find it as step 2 and 3 here: https://go.dev/tour/concurrency/1 > or the best way you can imagine I would do a lot much more to tune it if you were in a position where you'd know it would run that many "tasks". I think what many non-Go programmer…
The post was edited, previously it just said roughly this part: "step 2 and 3 here: https://go.dev/tour/concurrency/1". Which - as far as I can tell - does not mention worker pools...
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#84Earlier quoted context omitted.
The accounting here is weird though; Go isn’t using that RAM, it’s expecting the application to. The reason that doesn’t happen is because it’s a micro benchmark that produces no useful work.. The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high. But they are not the same; that memory is in anti…
Isn’t that kind of dumb when none of the other languages do this? Apparently allocating memory is really fast? Maybe we should change the test to load 1MB of data in every task?
Most of those languages (excepting Java virtual threads) uses stackless coroutines. Go uses stackful coroutines which allocates some memory upfront for a goroutine to use
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#85Earlier quoted context omitted.
Is that the ideomatic way to do it, or the best way you can imagine?
> Is that the ideomatic way to do it Well... I'm actually not sure what ideomatic means (English isn't my first language), but it's the standard way of doing it. You'll even find it as step 2 and 3 here: https://go.dev/tour/concurrency/1 > or the best way you can imagine I would do a lot much more to tune it if you were in a position where you'd know it would run that many "tasks". I think what many non-Go programmer…
I'm torn.
As far as practicality goes I actually agree with you: if I knew I were trying to do something to the order of 1,000,000 tasks in Go I would probably use a worker pool for this exact reason. I have done this pattern in Go. It is certainly not unidiomatic.
However, it also isn't the obvious way to do 1,000,000 things concurrently in Go. The obvious way to do 1,000,000 things concurrently in Go is to do a for loop and launch a Goroutine for each thing. It is the native unit of task. It is very tightly tied to how I/O works in Go.
If you are trying to do something like a web server, then the calculus changes a lot. In Go, due to the way I/O works, you really can't do much but have a goroutine or two per connection. However, on the other hand, the overhead that goroutines imply starts to look a lot smaller once you put real workloads on each of the millions of tasks.
This benchmark really does tell you something about the performance and overhead of the Go programming language, but it won't necessarily translate to production workloads the way that it seems like it will. In real workloads where the tasks themselves are usually a lot heavier than the constant cost per task, I actually suspect other issues with Go are likely to crop up first (especially in performance critical contexts, latency.) So realistically, it would probably be a bad idea to extrapolate from a benchmark this synthetic to try to determine anything about real world workloads.
Ultimately though, for whatever purpose a synthetic benchmark like this does serve, I think they did the correct thing. I guess I just wonder exactly what the point of it is. Like, the optimized Rust example uses around 0.12 KiB per task. That's extremely cool, but where in the real world are you going to find tasks where the actual state doesn't completely eclipse that metric? Meanwhile, Go is using around 2.64 KiB per task. 22x larger than Rust as it may be, it's still not very much. I think for most real world cases, you would struggle to find too many tasks where the working set per task is actually that small. Of course, if you do, then I'd reckon optimized async Rust will be a true barn-burner at the task, and a lot of those cases where every byte and millisecond counts, Go does often lose. There are many examples.[1]
In many cases Go is far from optimal: Channels, goroutines, the regex engine, various codec implementations in the standard library, etc. are all far from the most optimal implementation you could imagine. However, I feel like they usually do a good job making the performance very sufficient for a wide range of real world tasks. They have made some tradeoffs that a lot of us find very practical and sensible and it makes Go feel like a language you can usually depend on. I think this is especially true in a world where it was already fine when you can run huge websites on Python + Django and other stacks that are relatively much less efficient in memory and CPU usage than Go.
I'll tell you what this benchmark tells me really though: C# is seriously impressive.
[1]: https://discord.com/blog/why-discord-is-switching-from-go-to...
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#86Reminder that Rust does not automatically schedule anything. Unless you _explicitly_ call `tokio::spawn` or `async_std::spawn` you are still living entirely in state-machine land. Rust's `join_all` uses `FuturesUnordered` behind the scenes, which is pretty intelligent in terms of keeping track of which tasks are ready to make progress, but it does not use tokio/async_std for scheduling. AFAICT the only thing being me…
I did this measurement, and using time -v, the maximum resident size in KB comes out to 440,424 kb for 1m tasks, 46,820 kb for 100k, and 7,156 kb for 10k.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#87Earlier quoted context omitted.
The accounting here is weird though; Go isn’t using that RAM, it’s expecting the application to. The reason that doesn’t happen is because it’s a micro benchmark that produces no useful work.. The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high. But they are not the same; that memory is in anti…
Isn’t that kind of dumb when none of the other languages do this? Apparently allocating memory is really fast? Maybe we should change the test to load 1MB of data in every task?
> Apparently allocating memory is really fast?
Apart from i/o, allocating memory is usually the slowest thing you can do on a computer, in my experience.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#88Earlier quoted context omitted.
Go is absolutely using that ram, it’s not available to other services on the system.
You know what I mean. If this was a real world program where those million tasks actually performed work, then this stack space is available for the application to do that work. It’s not memory that’s consumed by the runtime, it’s memory the runtime expects the program to use - it’s just that this program does no useful work.
I am not u/masklinn - but I don't know what you mean. Doesn't the runtime consume memory by setting it aside for future use? Like what else does "using" ram mean other than claiming it for a time?
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#89Regarding Java I'm pretty sure that benchmark is broken at least a little bit and testing something else as not specifying initial size for ArrayList means list of size 10 which gets resized all the time when `add()` is called, leading to big amount of unused objects needing garbage collection.
Yeah that is a junior mistake... They should've pre-sized the ArrayList, or better, used an array because that's more memory efficient (and I would say would be what any decent dev would do when the size of tasks is known beforehand).
> Some folks pointed out that in Rust (tokio) it can use a loop iterating over the Vec instead of join_all to avoid the resize to the list
Right, but some folks also pointed out you should've used an array in Java in the previous blog post, 2 years ago, and you didn't do that.
And folks also pointed out Elixir shouldn't have used Task in the previous benchmark (folk here being the creator of Elixir himself): https://github.com/pkolaczk/async-runtimes-benchmarks/pull/7
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#90I feel this benchmark compares apples to oranges in some cases. For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all. This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call. While this might be the idiomatic way of concurrency in the respective languages, it do…
Actually, I think this benchmark did the right thing, that I wish more benchmarks would do. I'm much less interested in what the differences between compilers are than in what the actual output will be if I ask a professional Go or Node.js dev to solve the same task. (TBF, it would've been better if the task benchmarked was something useful, eg. handling an HTTP request.) Go heavily encourages a certain kind of progr…
But you wouldn't call a million tasks with `Promise.all` in Node, right? That's just not a thing that one does.
Instead, there's usually going to be some queue outside the VM that will leave you with _some_ sort of chunking and otherwise working in smaller, more manageable bits (that might, incidentally, be shaped in ways that the VM can handle in interesting ways).
It's definitely true to say that the "idioamatic" way of handling things is worth going into, but if part of your synthetic benchmark involves doing something quite out of the ordinary, it feels suspicious.
I generally agree that a "real" benchmark here would be nice. It would be interesting if someone could come up with the "minimum viable non-trivial business logic" that people could use for these benchmarks (perhaps coupled with automation tooling to run the benchmarks)