Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much memory do you need in 2024 to run 1M concurrent tasks?

hez2010.github.io

Post reply on HN
Print view View original

181–190 of 205 posts

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#181

Post by phillipcarter »

Mawr wrote: post #56

> Now Go loses by over 13 times to the winner. It also loses by over 2 times to Java, which contradicts the general perception of the JVM being a memory hog and Go being lightweight. Well, if it isn't the classic unwavering confidence that an artificial "hello world"-like benchmark is in any way representative of real world programs.

Yes, but also, languages like Java and C# have caught up a great deal over the past 10 years and run incredibly smoothly. Most peoples' perception of them being slow is really just from legacy tech that they encountered a long time ago, or (oof) being exposed to some terrible piece of .NET Framework code that's still running on an underprovisioned IIS server.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#183

Post by throwitaway1123 »

iforgotpassword wrote: post #100

Can someone explain the node version to me? My js knowledge is from a decade ago. AFAIK, setTimeout creates a timer and returns a handle to it. What does promisify do? I'd assume it's a general wrapper that takes a function that returns X and wraps it so that it returns Promise . So that code actually runs 10k tasks that each create a timer with a timeout of 10 seconds and return immediately.

Promisify converts a callback based function into a promise returning function [1]. If the function has a `promisify.custom` method, `promisify` will simply return the `promisify.custom` method instead of wrapping the original function. Calling `promisify` on `setTimeout` in Node is redundant because Node already ships a built in promisified version of `setTimeout`. So the following is true:

  setTimeout[promisify.custom] === require('node:timers/promises').setTimeout
You could of course manually wrap `setTimeout` yourself as well:

  const sleep = n => new Promise(resolve => setTimeout(resolve, n))
[1] https://nodejs.org/docs/latest-v22.x/api/util.html#utilpromi...

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#184

Post by theamk »

vlovich123 wrote: post #180
bilbo0s wrote: post #92

Earlier quoted context omitted.

I have a sneaky suspicion if you do anything other than the sleep during these 1 million tasks, you'll blow up memory on all of these systems. That's kind of the Achille's Heel of the benchmark. Any business needing to spawn 1 million tasks, certainly wants to do something on them. It's the "do something on them" part that usually leads to difficulties for these things. Not really the "spawn a million tasks" part.

The “do something” OP is referring to is simple things like a deeply nested set of function calls and on stack data structures allocated and freed before you sleep. This increases the size of the stack that Go needs to save. By comparison stackless coroutines only save enough information for the continuation, no more no less. That’s going to be strictly smaller than saving the entire stack. The argument you seem to b…

I have no doubt it's going to be strictly smaller, it's just the difference could be too small to care.

Like say you are making a server, and each client has 16KB of state. Then memory usage would be 17KB in Node vs 19 KB in Go. Smaller? Yes. Smaller enough that you want to rewrite the whole app? Probably not.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#185

Post by theamk »

OutOfHere wrote: post #169
jonathanstrange wrote: post #155

Earlier quoted context omitted.

No professional Go programmer would spawn 1M goroutines unless they're sure they have the memory for it (and even then, only if benchmarks indicate it, which is unlikely). Goroutines have a static stack overhead between 2KiB to 8KiB depending on the platform. You'd use a work stealing approach with a reasonable number of goroutines instead. How many are reasonable needs to be tested because it depends on how long eac…

The basis for running 1 million concurrent tasks is to support 1 million active concurrent user connections. They don't need to run in parallel if async is used. As shown, Rust and C# do well. How would you support it in Go?

Pay extra $1/month for bigger server? The numbers are really small here.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#186

Post by vlovich123 »

theamk wrote: post #184
vlovich123 wrote: post #180

Earlier quoted context omitted.

The “do something” OP is referring to is simple things like a deeply nested set of function calls and on stack data structures allocated and freed before you sleep. This increases the size of the stack that Go needs to save. By comparison stackless coroutines only save enough information for the continuation, no more no less. That’s going to be strictly smaller than saving the entire stack. The argument you seem to b…

I have no doubt it's going to be strictly smaller, it's just the difference could be too small to care. Like say you are making a server, and each client has 16KB of state. Then memory usage would be 17KB in Node vs 19 KB in Go. Smaller? Yes. Smaller enough that you want to rewrite the whole app? Probably not.

But it wouldn’t necessarily be 17kib vs 19kib then. It would be reasonable to assume that larger context would be paired with more stack usage so it’s 17kib vs 21 or 25kib and that’s 20-40% more memory being required for Go. That can be quite substantial as 1M concurrent clients represents a case where that memory starts becoming quite a premium.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#187

Post by OutOfHere »

theamk wrote: post #185
OutOfHere wrote: post #169

Earlier quoted context omitted.

The basis for running 1 million concurrent tasks is to support 1 million active concurrent user connections. They don't need to run in parallel if async is used. As shown, Rust and C# do well. How would you support it in Go?

Pay extra $1/month for bigger server? The numbers are really small here.

You obviously missed the chart. The memory usage difference isn't small; it's astronomical as far as scaling goes.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#188

Post by lossolo »

OutOfHere wrote: post #187
theamk wrote: post #185

Earlier quoted context omitted.

Pay extra $1/month for bigger server? The numbers are really small here.

You obviously missed the chart. The memory usage difference isn't small; it's astronomical as far as scaling goes.

In what world is the cost of 2.5 GB of RAM per 1 million connections an issue? Are you telling me there's a service in this world handling, for example, 100 million active connections, and they can't afford 250 GB of RAM? It's not the 90's anymore.

And we're talking about a naive microbenchmark. If you were actually building a service like that in Go (millions of active connections) and you were very concerned about memory usage, you wouldn't be naive enough to use a goroutine for every connection. Instead, you would use something like gnet or another solution based directly on epoll events, combined with a worker pool.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#189

Post by therealdrag0 »

Quothling wrote: post #113
pbhjpbhj wrote: post #102

Earlier quoted context omitted.

Idiomatic is the word the parent was looking for. The base word is idiom. It was probably the intent of the parent to mean 'making use of the particular features of the language that are not necessarily common to other languages'. I'm not a programmer, but you appear to give good examples. I hope I'm not teaching you to suck eggs... {That's an idiom, meaning teaching someone something they're already expert in. Like…

I actually did find "idiomatic" when I looked it up, but I honestly still didn't quite grasp it from the cambridge dictionary. Thanks for explaining it in a way I understand.

In programming Idiomatic is used to reference a programming language’s “best practices” and “style guide”. Obviously programming languages can solve problems in many different ways, but they often develop a “correct way” that matches their design or the personality of their influential community members. Following this advice is Ideomatic. Next time your coworker has their style wrong you can say “I don’t think this is Idiomatic” :D

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#190

Post by aeturnum »

jakewins wrote: post #148
aeturnum wrote: post #88

Earlier quoted context omitted.

I am not u/masklinn - but I don't know what you mean. Doesn't the runtime consume memory by setting it aside for future use? Like what else does "using" ram mean other than claiming it for a time?

If the example was extended to, say, once the sleep is completed then parse and process some JSON data (simulating the sleep being a wait on some remote service), then how would memory use be affected? In the Go number reported, the majority of the memory is the stack Go allocated for the application code anticipating processing to happen. In the Node example, the processing instead will need heap allocation. Point b…

I mean, so I guess you are saying that other languages are better at estimating the memory usage than Go - as go will never need this memory it has allocated? Like Go knows everything that will happen in that goroutine, it should be able to right-size it. I don't think it "looks" wasteful for Go to allocate memory it should know it doesn't need at compile time - I think it is wasteful. Though it's probably not a meaningful waste most of the time.

I agree that it would also be interesting to benchmark some actual stack or heap usage and how the runtimes handle that, but if you are running a massively parallel app you do sometimes end up scheduling jobs to sleep (or perhaps, more likely, to prepare to act but they never do and get cancelled). So I think this is a valid concern, even though it's not the only thing that amtters.

Post reply on HN