Live data from Hacker News

How much memory do you need in 2024 to run 1M concurrent tasks?

hez2010.github.io

201–205 of 205 posts

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#201
post #197

Earlier quoted context omitted.

There's two main reasons: * gunicorn doesn't have worker scaling, they're all always running, so async workers were to not waste those resources idling while still allowing lots of simultaneous clients * The limit was 32768 not that long ago, which was at least possible to hit with a ton of clients

Why would I be worried about processes using resources while blocked (not using CPU)? Virtual memory is a thing. And if it’s a web server that’s just not being used.

RAM, not CPU, which goes right back to this post's benchmark. Python code takes up memory too, not just data. The workers consume this memory on startup, not just while handling a request.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#202

Earlier quoted context omitted.

I agree with everything you said and I think you contributed a lot to what I said making things much more clear. > I'll tell you what this benchmark tells me really though: C# is seriously impressive. The C# team has done some really great work in recent years. I personally hate working with it and it's "magic", but it's certainly in a very good place as far as trusting the CLR to "just work". Hilariously I also foun…

Userspace scheduling of Goroutines, virtual stack and non-deterministic pointer type allocation in Go are as much magic if not more, the syntactic sugar of C# is there to get the language out of your way and usually comes at no cost :) If you do not like the aesthetics of C# and find Elixir or OCaml family tolerable - perhaps try F#? If you use task CEs there you end up with roughly the same performance profile and g…

> Userspace scheduling of Goroutines, virtual stack and non-deterministic pointer type allocation in Go are as much magic if not more, the syntactic sugar of C# is there to get the language out of your way and usually comes at no cost :)

I don't think C# does it at no cost. I think it's "attachment" to Clean Code makes most C# code bases horrible messes after a while. I know this is a preference thing and that many people will disagree, but I've seen C# code bases that were so complicated to work with that they were actively hindering the development teams ability to meet the business needs. You don't have to write C# that way, but that's what happens in almost every company where I live.

> If you do not like the aesthetics of C# and find Elixir or OCaml family tolerable - perhaps try F#? If you use task CEs there you end up with roughly the same performance profile and get to access huge ecosystem making it one of the few FP languages that can be used in production with minimal risk.

I mean, I don't think I'll ever have to work within the dotnet ecosystem. The way things are going in the green energy and finance sector which is where my career have taken me I'll mostly get to work with Python (with C/Zig) or Go and possibly Java. C# and dotnet is almost exclusively used at stagnant small-medium sized companies and in the consultance business servicing these companies. This is not because of C# or dotnet but more because of the developer landscape. Java is big in "older" organisations because it's what was taught in universities and because it was always good, Go is replacing C#/Java in a lot of newer companies because there are a lot of success stories around it and a lot of the Java developers are retiring. Python is growing really big because a lot of non-swe engineers and accountant types are using it as well as how it's used in ML/AI/Datawarehouse. PHP is big in the web-shop industry and so on. C# manly made it's way into business at places which ran a lot of windows servers. Since organisations rarely change tech stacks in the more "boring" parts of the world, it's not likely to change much.

I don't think dotnet or C# are bad. I write some powershell for azure automation to help IT operations from time to time, but I really don't like working with C# (or Java). I would personally like to work with Rust or more Zig at some point, but it's not like anyone is adopting Rust around here and while Zig can be used for some things in place of C it's not really "production ready" for most things.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#204

Earlier quoted context omitted.

You know what I mean. If this was a real world program where those million tasks actually performed work, then this stack space is available for the application to do that work. It’s not memory that’s consumed by the runtime, it’s memory the runtime expects the program to use - it’s just that this program does no useful work.

I am not u/masklinn - but I don't know what you mean. Doesn't the runtime consume memory by setting it aside for future use? Like what else does "using" ram mean other than claiming it for a time?

Allocating virtual memory is distinct from actually consuming physical memory (RAM).

If a process allocates many pages of virtual memory, but never actually reads or writes to that memory, then it's unlikely that any physical memory backs those pages. In this sense, allocating memory is really just bookkeeping in the operating system. It's when you try to read or write that memory that the operating system will actually allocate physical memory for you.

When you first try to access the virtual memory you've allocated, there will be a page fault, causing the OS to determine if you're actually allowed to read or write to it. If you've previously allocated it, then all is good, and the OS will allocate some physical memory for you, and make your virtual memory pointers point to that physical memory. If not, well, then that's a segfault. It's not until you first try to use the memory that you actually consume RAM.

Re: How much memory do you need in 2024 to run 1M concurrent tasks?

#205

Earlier quoted context omitted.

You obviously missed the chart. The memory usage difference isn't small; it's astronomical as far as scaling goes.

In what world is the cost of 2.5 GB of RAM per 1 million connections an issue? Are you telling me there's a service in this world handling, for example, 100 million active connections, and they can't afford 250 GB of RAM? It's not the 90's anymore. And we're talking about a naive microbenchmark. If you were actually building a service like that in Go (millions of active connections) and you were very concerned about…

> If you were actually building a service like that in Go (millions of active connections) and you were very concerned about memory usage

Or, much more likely, pay a few tens of dollars per month for a second server and start scaling out.

Post reply on HN