Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much memory do you need in 2024 to run 1M concurrent tasks?

hez2010.github.io

Post reply on HN
Print view View original

191–200 of 205 posts

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#191

Post by liontwist »

masklinn wrote: post #143
blitzar wrote: post #123

Earlier quoted context omitted.

I am left wondering what happens at 4194305

You can fork bomb your system and observe.

The fork syscall will fail at the hard limit.

What’s missing here is that all these async/await, lightweight threads, etc features exist because presumably because OS processes and threads consume too many resources to feasibly serve a similar role.

However nobody seems to have any hard numbers about the topic. How bad is a Linux context switch? How much memory does a process use? Suddenly everyone is a micro optimizer seeking to gain a small constant multiple.

Since this is not ready at hand I suspect the rewards are much less clear. It’s more likely that the languages benefit from greater cross platform control and want to avoid cross platform inconsistencies.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#192

Post by fulafel »

aba_cz wrote: post #70

Regarding Java I'm pretty sure that benchmark is broken at least a little bit and testing something else as not specifying initial size for ArrayList means list of size 10 which gets resized all the time when `add()` is called, leading to big amount of unused objects needing garbage collection.

From the docs: "The add operation runs in amortized constant time, that is, adding n elements requires O(n) time".

Given the linear time time complexity, it seems obvious that adding a thread pointer to the list won't contribute substantially to the thread creation time.

The way these kinds of operations can be implemented for amortized linear time (also in Python, C realloc, etc) is explained in https://en.wikipedia.org/wiki/Dynamic_array#Geometric_expans...

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#193

Post by fulafel »

Mawr wrote: post #56

> Now Go loses by over 13 times to the winner. It also loses by over 2 times to Java, which contradicts the general perception of the JVM being a memory hog and Go being lightweight. Well, if it isn't the classic unwavering confidence that an artificial "hello world"-like benchmark is in any way representative of real world programs.

Also of course Java and C# overwhelmingly use threads and not async for this kind of thing.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#194

Post by davidatbu »

vbsd wrote: post #178
davidatbu wrote: post #134

Earlier quoted context omitted.

Someone linked the code in another comment, and the start time is most definitely recorded when the future is created: https://docs.rs/tokio/1.41.1/src/tokio/time/sleep.rs.html#12...

Huh, you're right about this, thanks. On the other hand, I maintain that this is an incidental rather than essential reason for the program finishing quickly. In that benchmark code, we can replace "sleep" with our custom sleep function which does not record start time before execution: async fn wrapped_sleep(d: Duration) { sleep(d).await } The following program will still finish in ~10 seconds. #[tokio::main] async…

In `wrapped_sleep` function body, where does the `sleep()` come from? It's still tokio::time::sleep, right? If so, the start time is recorded before the first `.await`.

Regardless, the program you provided _does_ actually run the futures concurrently, because of the `join_all()`. My point above was that in the original blog post, the appendix has a version without `join_all()`, which has no concurrency.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#195

Post by [deleted] »

masklinn wrote: post #151
xarope wrote: post #138

Earlier quoted context omitted.

The argument then, is what if we DO load 2K worth [0] of randomized data into each of those 1m goroutines (and equivalents in the other languages), and do some actual processing. Would we still see the equivalent 10x (whatever math works it out to be) memory "bloat"? And what about performance? We, as devs, have "4" such resources available to us, memory, network, I/O and compute. And it behooves us to not prematurel…

So the argument is “if you measure something completely different from and unrelated to the article you do not get the same result”? I guess that’s true. And to be clear, I do agree with the top comment (which seems to be by you), TFA uses timers in the other runtimes and go does have timers so using goroutines is unwarranted and unfair. And I said as much a few comment up (although I’d forgotten about AfterFunc so I…

[deleted]

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#196

Post by neonsunset »

fulafel wrote: post #193
Mawr wrote: post #56

> Now Go loses by over 13 times to the winner. It also loses by over 2 times to Java, which contradicts the general perception of the JVM being a memory hog and Go being lightweight. Well, if it isn't the classic unwavering confidence that an artificial "hello world"-like benchmark is in any way representative of real world programs.

Also of course Java and C# overwhelmingly use threads and not async for this kind of thing.

Very few .NET projects rely on explicit threading. There is almost never a reason to do so when tasks can be used instead, like in this benchmark.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#197

Post by Izkata »

liontwist wrote: post #191
masklinn wrote: post #143

Earlier quoted context omitted.

You can fork bomb your system and observe.

The fork syscall will fail at the hard limit. What’s missing here is that all these async/await, lightweight threads, etc features exist because presumably because OS processes and threads consume too many resources to feasibly serve a similar role. However nobody seems to have any hard numbers about the topic. How bad is a Linux context switch? How much memory does a process use? Suddenly everyone is a micro optimiz…

There's two main reasons:

* gunicorn doesn't have worker scaling, they're all always running, so async workers were to not waste those resources idling while still allowing lots of simultaneous clients

* The limit was 32768 not that long ago, which was at least possible to hit with a ton of clients

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#198

Post by vbsd »

davidatbu wrote: post #194
vbsd wrote: post #178

Earlier quoted context omitted.

Huh, you're right about this, thanks. On the other hand, I maintain that this is an incidental rather than essential reason for the program finishing quickly. In that benchmark code, we can replace "sleep" with our custom sleep function which does not record start time before execution: async fn wrapped_sleep(d: Duration) { sleep(d).await } The following program will still finish in ~10 seconds. #[tokio::main] async…

In `wrapped_sleep` function body, where does the `sleep()` come from? It's still tokio::time::sleep, right? If so, the start time is recorded before the first `.await`. Regardless, the program you provided _does_ actually run the futures concurrently, because of the `join_all()`. My point above was that in the original blog post, the appendix has a version without `join_all()`, which has no concurrency.

Ah, I missed that you were talking about the code in the appendix, not the top examples. Yeah, you’re right on every count then, aplologies.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#199

Post by OutOfHere »

lossolo wrote: post #188
OutOfHere wrote: post #187

Earlier quoted context omitted.

You obviously missed the chart. The memory usage difference isn't small; it's astronomical as far as scaling goes.

In what world is the cost of 2.5 GB of RAM per 1 million connections an issue? Are you telling me there's a service in this world handling, for example, 100 million active connections, and they can't afford 250 GB of RAM? It's not the 90's anymore. And we're talking about a naive microbenchmark. If you were actually building a service like that in Go (millions of active connections) and you were very concerned about…

Go is a friendly language and it is well liked, but it is inappropriate when the absolute best performance saves more money than the additional developer salaries needed to write it in a higher performance language. An example is for deep learning or other big numerical work, where you'd be wasting expensive hardware resources if using Go. Perhaps the one million concurrent users will however be fine with Go.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#200

Post by liontwist »

Izkata wrote: post #197
liontwist wrote: post #191

Earlier quoted context omitted.

The fork syscall will fail at the hard limit. What’s missing here is that all these async/await, lightweight threads, etc features exist because presumably because OS processes and threads consume too many resources to feasibly serve a similar role. However nobody seems to have any hard numbers about the topic. How bad is a Linux context switch? How much memory does a process use? Suddenly everyone is a micro optimiz…

There's two main reasons: * gunicorn doesn't have worker scaling, they're all always running, so async workers were to not waste those resources idling while still allowing lots of simultaneous clients * The limit was 32768 not that long ago, which was at least possible to hit with a ton of clients

Why would I be worried about processes using resources while blocked (not using CPU)? Virtual memory is a thing. And if it’s a web server that’s just not being used.
Post reply on HN