Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How much memory do you need in 2024 to run 1M concurrent tasks?

hez2010.github.io

Post reply on HN
Print view View original

171–180 of 205 posts

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#171

Post by neonsunset »

mrighele wrote: post #162
xargon7 wrote: post #133

There's a difference between "running a task that waits for 10 seconds" and "scheduling a wakeup in 10 seconds". The code for several of the languages that are low-memory usage that do the second while the high memory usage results do the first. For example, on my machine the article's go code uses 2.5GB of memory but the following code uses only 124MB. That difference is in-line with the rust results. package main i…

I agree with you. Even something as simple as a loop like (pseudocode) for (n=0;n Changes the results quite a bit: for some reasons java use a _lot_ more memory and takes longer (~20 seconds), C# uses more that 1GB of memory, while python struggles with just scheduling all those tasks and takes more than one minute (beside taking more memory). node.js seems unfazed by this change. I think this would be a more reasona…

Indeed, looping over a Task.Delay likely causes a lot of churn in timer queues - that's 10M timers allocated and scheduled! If it is replaced with 'PeriodicTimer', the end result becomes more reasonable.

This (AOT-compiled) F# implementation peaks at 566 MB with WKS GC and 509 MB with SRV GC:

    open System
    open System.Threading
    open System.Threading.Tasks

    let argv = Environment.GetCommandLineArgs()

    [1..int argv[1]]
    |> Seq.map (fun _ ->
        task {
            let timer = PeriodicTimer(TimeSpan.FromSeconds 1.0)
            let mutable count = 10
            while! timer.WaitForNextTickAsync() do
                count  Task)
    |> Task.WaitAll
To Go's credit, it remains at consistent 2.53 GB and consumes quite a bit less CPU.

We're really spoiled with choice these days in compiled languages. It takes 1M coroutines to push the runtime and even at 100k the impact is easily tolerable, which is far more than regular applications would see. At 100K .NET consumes ~57 MB and Go consumes ~264 MB (and wins at CPU by up to 2x).

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#173

Post by brabel »

sfn42 wrote: post #109
brabel wrote: post #89

Earlier quoted context omitted.

Yeah that is a junior mistake... They should've pre-sized the ArrayList, or better, used an array because that's more memory efficient (and I would say would be what any decent dev would do when the size of tasks is known beforehand). > Some folks pointed out that in Rust (tokio) it can use a loop iterating over the Vec instead of join_all to avoid the resize to the list Right, but some folks also pointed out you sho…

The difference between an arraylist with correct initial size and an array is almost nothing. Arraylist itself is just a wrapper around an array.

It can be a big difference if boxing is involved. Or if the list is very big, because all access to items in the list require casting at the bytecode level (due to type erasure).

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#175

Post by jonathanstrange »

OutOfHere wrote: post #169
jonathanstrange wrote: post #155

Earlier quoted context omitted.

No professional Go programmer would spawn 1M goroutines unless they're sure they have the memory for it (and even then, only if benchmarks indicate it, which is unlikely). Goroutines have a static stack overhead between 2KiB to 8KiB depending on the platform. You'd use a work stealing approach with a reasonable number of goroutines instead. How many are reasonable needs to be tested because it depends on how long eac…

The basis for running 1 million concurrent tasks is to support 1 million active concurrent user connections. They don't need to run in parallel if async is used. As shown, Rust and C# do well. How would you support it in Go?

The servers I use have limits far below 1M active connections, realistically speaking about 60k simultaneously active connections. So I can't really answer that question. However, it's easy to find answers to that question online [1]. Go is not forcing you to spawn Goroutines when you don't really need them. As I said, the correct way in Go is to use worker pools, the size of which depends on measurable performance because it is connected to how much i/o each Goroutine performs and how long it waits on average.

[1] https://www.freecodecamp.org/news/million-websockets-and-go-...

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#176

Post by polyrand »

Out of curiosity, I checked if using uvloop[0] in Python changed the numbers.

This is the code:

  # /// script
  # requires-python = ">=3.12"
  # dependencies = ["uvloop"]
  # ///
  
  import asyncio
  import sys
  
  import uvloop
  
  
  async def main(num_tasks):
      tasks = []
  
      for task_id in range(num_tasks):
          tasks.append(asyncio.sleep(10))
  
      await asyncio.gather(*tasks)
  
  
  if __name__ == "__main__":
      num_tasks = int(sys.argv[1])
      # uvloop.run(main(num_tasks))
      asyncio.run(main(num_tasks))
I ran it with 100k tasks:

  /usr/bin/time -l -p -h uv run async-memory.py 100000
On my M1 MacBook Pro, using asyncio reports (~170MB):

  170835968  maximum resident set size
Using uvloop (~204MB):

  204259328  maximum resident set size
I kept the `import uvloop` statement when just using asyncio so that both cases start in the same conditions.

[0]: https://github.com/MagicStack/uvloop/

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#177

Post by lmm »

OutOfHere wrote: post #170
lmm wrote: post #53

Earlier quoted context omitted.

> The requirement is to run 1 million concurrent tasks. That's not a real requirement though. No business actually needs to run 1 million concurrent tasks with no concern for what's in them.

If you want to support 1 million concurrent active users, you can need it.

Maybe. But in that case you will need to do something for each of those users, and which languages are good at that might look quite different from this benchmark.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#178

Post by vbsd »

davidatbu wrote: post #134
vbsd wrote: post #78

Earlier quoted context omitted.

> because `tokio::time::sleep()` keeps track of when the future was created, (ie when `sleep()` was called) instead of when the future is first `.await`ed I’m not a Rust programmer but I strongly suspect this updated explanation is erroneous. It’s probably more like this: start time is recorded when the task execution is started. However, the task immediately yields control back to the async loop. Then the async loop…

Someone linked the code in another comment, and the start time is most definitely recorded when the future is created: https://docs.rs/tokio/1.41.1/src/tokio/time/sleep.rs.html#12...

Huh, you're right about this, thanks.

On the other hand, I maintain that this is an incidental rather than essential reason for the program finishing quickly. In that benchmark code, we can replace "sleep" with our custom sleep function which does not record start time before execution:

  async fn wrapped_sleep(d: Duration) {
      sleep(d).await
  }
The following program will still finish in ~10 seconds.

  #[tokio::main]
  async fn main() {
      let num_tasks = 100;
      let mut tasks = Vec::new();
      for _ in 0..num_tasks {
          tasks.push(wrapped_sleep(Duration::from_secs(10)));
      }
      futures::future::join_all(tasks).await;
  }

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#180

Post by vlovich123 »

bilbo0s wrote: post #92
pkulak wrote: post #82

Earlier quoted context omitted.

Except it’s more than that. Go and Java maintain a stack for every virtual thread. They are clever about it, but it’s very possible that doing anything more than a sleep would have blown up memory on those two systems.

I have a sneaky suspicion if you do anything other than the sleep during these 1 million tasks, you'll blow up memory on all of these systems. That's kind of the Achille's Heel of the benchmark. Any business needing to spawn 1 million tasks, certainly wants to do something on them. It's the "do something on them" part that usually leads to difficulties for these things. Not really the "spawn a million tasks" part.

The “do something” OP is referring to is simple things like a deeply nested set of function calls and on stack data structures allocated and freed before you sleep. This increases the size of the stack that Go needs to save. By comparison stackless coroutines only save enough information for the continuation, no more no less. That’s going to be strictly smaller than saving the entire stack. The argument you seem to be making is that that could be the same size as the stack (eg heap allocations) but I think that’s being unreasonably optimistic. It should always end up being strictly smaller.
Post reply on HN