Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much memory do you need in 2024 to run 1M concurrent tasks?

hez2010.github.io

Post reply on HN
Print view View original

71–80 of 205 posts

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#71

Post by masklinn »

cperciva wrote: post #57
masklinn wrote: post #38

Earlier quoted context omitted.

> I have no idea what Go is doing to get 2500 bytes per task. TFA creates a goroutine (green thread) for each task (using a waitgroup to synchronise them). IIRC goroutines default to 2k stacks, so that’s about right. One could argue it’s not fair and it should be timers which would be much lighter. There’s no “efficient wait” for them but that’s essentially the same as the appendix rust program.

Aha, 2k stacks. I figured that stacks would be page size (or more) so 2500 seemed both too small for the thread to have a stack and too large for it to not have a stack. 2k stacks are an interesting design choice though... presumably they're packed, in which case stack overflow is a serious concern. Most threading systems will do something like allocating a single page for the stack but reserving 31 guard pages in ca…

Goroutines being go structures, the runtime can cooperate with itself so it doesn't need to do any sort of probing: function prologues can check if there's enough stack space for its frame, and grow the stack if not.

In reality it does use a guard area (technically I think it's more of a redzone? It doesn't cause access errors and functions with known small static frames can use it without checking).

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#72

Post by Aeolun »

jakewins wrote: post #46
threeseed wrote: post #33

Earlier quoted context omitted.

The requirement is to run 1 million concurrent tasks. Of course each language will have a different way of achieving this task each of which will have their unique pros/cons. That's why we have these different languages to begin with.

The accounting here is weird though; Go isn’t using that RAM, it’s expecting the application to. The reason that doesn’t happen is because it’s a micro benchmark that produces no useful work.. The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high. But they are not the same; that memory is in anti…

Isn’t that kind of dumb when none of the other languages do this? Apparently allocating memory is really fast? Maybe we should change the test to load 1MB of data in every task?

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#73

Post by Quothling »

Aeolun wrote: post #69
Quothling wrote: post #64

Earlier quoted context omitted.

The article is telling us that you can run really inefficient code. Goroutines should be run with worker pools and a buffered channel and it's silly to not do that and then compare it to things like an optimized Rust crate like Tokio.

Is that the ideomatic way to do it, or the best way you can imagine?

> Is that the ideomatic way to do it

Well... I'm actually not sure what ideomatic means (English isn't my first language), but it's the standard way of doing it. You'll even find it as step 2 and 3 here: https://go.dev/tour/concurrency/1

> or the best way you can imagine

I would do a lot much more to tune it if you were in a position where you'd know it would run that many "tasks". I think what many non-Go programmers might run into here is that Go doesn't come with any sort of "magic". Instead it comes with a highly opinionated way of doing things. Compare that to C# which comes with a highly optimized CLR and a bunch really excellent libraries which are continuously optimized by Microsoft and you're going to end up with an article like this. The async libraries are maintaining which tasks are running (though Promise.All is obviously also binding a huge amount of memory you don't have to), while the Go example is running 1 million at once.

You'll also notice that there is no benchmark for execution time. With Go you might actually want to pay with memory, though I'd argue that you'd almost never want to run 1 million Goroutines at once.

Though to be fair to this specific author, it looks like they copied the previous benchmarks and then ran it as-is.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#74

Post by uluyol »

cperciva wrote: post #57
masklinn wrote: post #38

Earlier quoted context omitted.

> I have no idea what Go is doing to get 2500 bytes per task. TFA creates a goroutine (green thread) for each task (using a waitgroup to synchronise them). IIRC goroutines default to 2k stacks, so that’s about right. One could argue it’s not fair and it should be timers which would be much lighter. There’s no “efficient wait” for them but that’s essentially the same as the appendix rust program.

Aha, 2k stacks. I figured that stacks would be page size (or more) so 2500 seemed both too small for the thread to have a stack and too large for it to not have a stack. 2k stacks are an interesting design choice though... presumably they're packed, in which case stack overflow is a serious concern. Most threading systems will do something like allocating a single page for the stack but reserving 31 guard pages in ca…

Go stacks are dynamically copied and resized. Stack overflow is not a concern.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#75

Post by abdellah123 »

Can we do something more real world at least? what's the cost (hetzner monthly) of maintaining 1M concurrent websocket connection where each make a query to a postgres db randomly every 1-4 seconds.

The cost wouldn't be just Memory because the network card and CPU also enter the game.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#76

Post by samsartor »

Reminder that Rust does not automatically schedule anything. Unless you _explicitly_ call `tokio::spawn` or `async_std::spawn` you are still living entirely in state-machine land.

Rust's `join_all` uses `FuturesUnordered` behind the scenes, which is pretty intelligent in terms of keeping track of which tasks are ready to make progress, but it does not use tokio/async_std for scheduling. AFAICT the only thing being measured about tokio/async_std is the heap size of their `sleep` implementations.

I'd be very interesting in seeing how Tokio's actual scheduler performs. The two ways to do that are:

- using https://docs.rs/tokio/latest/tokio/task/join_set/struct.Join... to spawn all the futures and then await them

- spawn each future in the global scheduler, and then await the JoinHandles using the for loop from the appendix

As other commenters have noted, calling `sleep` only constructs a state machine. So the Appendix isn't actually concurrent. Again, you need to either put those state machines into the tokio/async_std schedulers with `spawn`, or combine the state machines with `FuturesUnordered`.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#77

Post by cperciva »

uluyol wrote: post #74
cperciva wrote: post #57

Earlier quoted context omitted.

Aha, 2k stacks. I figured that stacks would be page size (or more) so 2500 seemed both too small for the thread to have a stack and too large for it to not have a stack. 2k stacks are an interesting design choice though... presumably they're packed, in which case stack overflow is a serious concern. Most threading systems will do something like allocating a single page for the stack but reserving 31 guard pages in ca…

Go stacks are dynamically copied and resized. Stack overflow is not a concern.

Oh yuck. Invalidating all the pointers to the stack? That's got to be expensive.

I guess if you're already doing garbage collection moving the stack doesn't make things all that much worse though... still, yuck.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#78

Post by vbsd »

davidatbu wrote: post #4

I write (async) Rust regularly, and I don't understand how the version in the appendix doesn't take 10x1,000,000 seconds to complete. In other words, I'd have expected no concurrency to take place. Am I wrong? UPDATE: From the replies below, it looks like I was right about "no concurrency takes place", but I was wrong about how long it takes, because `tokio::time::sleep()` keeps track of when the future was created,…

> because `tokio::time::sleep()` keeps track of when the future was created, (ie when `sleep()` was called) instead of when the future is first `.await`ed

I’m not a Rust programmer but I strongly suspect this updated explanation is erroneous. It’s probably more like this: start time is recorded when the task execution is started. However, the task immediately yields control back to the async loop. Then the async loop starts another task, and so on. It’s just that the async loop only returns the control to sleeping task no earlier than the moment 1s passes after the task execution was initialy started. I’d be surprised if it had anything to do with when sleep() was called.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#79

Post by codetiger »

gleenn wrote: post #68
AkshitGarg wrote: post #9

I feel this benchmark compares apples to oranges in some cases. For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all. This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call. While this might be the idiomatic way of concurrency in the respective languages, it do…

Also, for Java, Virtual Threads are a very new feature (Java 21 IIRC or somewhere around there). OS threads have been around for decades. As a heavy JVM user it would have been nice to actually see those both broken out to compare as well!

The original benchmark had the comparison between Java thread and Java Virtual thread. https://pkolaczk.github.io/memory-consumption-of-async/

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#80

Post by YetAnotherNick »

n2d4 wrote: post #41
AkshitGarg wrote: post #9

I feel this benchmark compares apples to oranges in some cases. For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all. This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call. While this might be the idiomatic way of concurrency in the respective languages, it do…

Actually, I think this benchmark did the right thing, that I wish more benchmarks would do. I'm much less interested in what the differences between compilers are than in what the actual output will be if I ask a professional Go or Node.js dev to solve the same task. (TBF, it would've been better if the task benchmarked was something useful, eg. handling an HTTP request.) Go heavily encourages a certain kind of progr…

The fundamental problem is there are two kind of sleep function. One that actually sleeps and other that is a actually a timer that just calls certain callback after a desired interval. Promise is just a syntactic sugar on top of second type. Go certainly could call another function after desired interval using `Timer`.

I think better comparison would be wasting CPU for 10 seconds instead of sleep.

Post reply on HN