No Erlang/Elixir?? The author would be surprised.
How much memory do you need in 2024 to run 1M concurrent tasks?
141–150 of 205 posts
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#142I’m still baffled that some people are bold enough to voluntarily posts those kind of most-of-the-time useless “benchmark” that will inevitably be riddled with errors.
I don’t know what pushes them. In the end you look like a clown more often than not.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#143Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#144Earlier quoted context omitted.
Most of those languages (excepting Java virtual threads) uses stackless coroutines. Go uses stackful coroutines which allocates some memory upfront for a goroutine to use
Then it is fair to compare the memory usage of a stackful coroutine to a stack less one as they are the idiomatic way to perform async task on each language.
I mean this is subjective, but as long as it’s clear that one number is “this is the memory the runtime itself consumes to solve this problem” and the other number is “this is the runtime memory use and it includes pre-allocated stack space that a real application would then use”, sure
Point being: Someone reading this to choose which runtime will fit their use case needs to be carefully to not assume the numbers measure the same thing. For some real world use cases the pre allocated stack will perform better than the runtimes that instead will do heap allocations.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#145Earlier quoted context omitted.
You know what I mean. If this was a real world program where those million tasks actually performed work, then this stack space is available for the application to do that work. It’s not memory that’s consumed by the runtime, it’s memory the runtime expects the program to use - it’s just that this program does no useful work.
I am not u/masklinn - but I don't know what you mean. Doesn't the runtime consume memory by setting it aside for future use? Like what else does "using" ram mean other than claiming it for a time?
I think he means that if the Go code had done something more useful, it would use about the same amount of memory. Compare that to another implementation, which might allocate nearly no memory when the tasks don't do anything significant but would quickly catch up to Go if they did.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#146Reminds me of the famous
SCO: "thread creation is about a thousand times faster than on native Linux"
Talk is cheap. Show me the code.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#147I came here to rage (im just honest) because the go code example is bad and absolute not representative. Im coding go code for multiple years, especially alot of multithreading, and what is presented there as result is just wrong. Apart from no necessaty to use a waitgroup for threading, as many others here already have stated even with waitgroup you cacn reduce the memory significantly down to like 130mb for 1mio threads.
Also some other languages seem to be missrepresented.
Seems like someone had good intentions but no idea about the languages he tried to compare and the result is this article.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#148Earlier quoted context omitted.
You know what I mean. If this was a real world program where those million tasks actually performed work, then this stack space is available for the application to do that work. It’s not memory that’s consumed by the runtime, it’s memory the runtime expects the program to use - it’s just that this program does no useful work.
I am not u/masklinn - but I don't know what you mean. Doesn't the runtime consume memory by setting it aside for future use? Like what else does "using" ram mean other than claiming it for a time?
If the example was extended to, say, once the sleep is completed then parse and process some JSON data (simulating the sleep being a wait on some remote service), then how would memory use be affected?
In the Go number reported, the majority of the memory is the stack Go allocated for the application code anticipating processing to happen. In the Node example, the processing instead will need heap allocation.
Point being that the two numbers are different - one measures just the overhead of the runtime, the other adds the memory reserved for the app to do work.
The result then looks wasteful for Go because the benchmark.. doesn’t do anything. In a real app though, preallocating stack can often be faster than doing just-in-time heap allocation.
Not always of course! Just noting that the numbers are different things; one is runtime cost, one is runtime cost plus an optimization that assumes memory will be needed for processing after the sleep.
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#149Go's performance has been neglected in past 10y. I even posted about it
Re: How much memory do you need in 2024 to run 1M concurrent tasks?
#150It seems there are just two clubs: you go with bare metal (Rust, C# native AOT) or you use some higher level abstraction (virtual machine, garbage collector) and then there is no significant difference between Java, Node, Go or Python.
For me Python worked surprisingly well, while Go was surprisingly high on memory consumption.