Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What async promised and what it delivered

causality.blog

Post reply on HN
Print view View original

131–140 of 317 posts

Re: What async promised and what it delivered

#131

Post by cmrdporcupine »

ibraheemdev wrote: post #80

> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space Why is reserving a megabyte of stack space "expensive"? > and takes roughly a millisecond to create I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.

There's much ridiculous hatred for OS threads based on people's biases of operating systems and hardware from 20 years ago.

So much so that they'll sign themselves up for async frameworks that thread steal at will and bounce things all over cores causing cache line bouncing and associated memory stalls, not understanding what this is doing to their performance profile.

And endure complexity, etc. through awkward async call chains and function colouring.

Most people's applications would be totally fine just spawning OS threads and using them without fear and dropping into a futex when waiting on I/O; or using the kernel's own async completion frameworks. The OS scheduler is highly efficient, and it is very good at managing multiple cores and even being aware asymmetrical CPU hierarchies, etc.. Likely more efficient than half the async runtimes out there.

Re: What async promised and what it delivered

#132

Post by delusional »

ysleepy wrote: post #109
tekacs wrote: post #96

Earlier quoted context omitted.

I mean Java's Loom feels like the 'ultimate' example of the latter for the _ordinary_ programmer, in that it effectively leaves you just doing what looks like completely normal threads however you so please, and it all 'just works'.

Java has gone full circle. Java had green threads in 1997, removed them in 2000 and brought them back properly now as virtual threads. I'm kinda glad they've sat out the async mania, with virtual threads/goroutines, the async stuff just feels like lipstick on a pig. Debugging, stacktrackes etc. are just jumbled.

Java didn't really "sit it out". It launched CompletableFutures, CompletionStages, Sources and Sinks, arguably even streams. All of those are standard library forms of async programming. People tried to make it catch on, but the experience of using it, The runtime wrapping all your errors in completion exceptions, destroying your callstacks, just made it completely useless.

Re: What async promised and what it delivered

#133

Post by gf000 »

josephg wrote: post #98
rstuart4133 wrote: post #33

Async is a Javascript hack that inexplicably got ported to other languages that didn't need it. The issue arose because Javascript didn't have threads, and processing events from the DOM is naturally event driven. To be fair, it's a rare person who can deal with the concurrency issues threads introduce, but the separate stacks threads provide a huge boon. They allow you to turn event driven code into sequential code.…

> At any step in that sequence, the language could have introduced green threads and the job would have been done. The job wouldn’t have been done. They would have needed threads. And mutexes. And spin locks. And atomics. And semaphores. And message queues. And - in my opinion - the result would have been a much worse language. Multithreaded code is often much harder to reason about than async code, because threads c…

Now you are comparing single threaded code with multi threaded, which is a completely different axis to async vs sync. Just take a look at C#'s async, where you have both async and multi threading, with all the possible combinations of concurrency bugs you can imagine.

Re: What async promised and what it delivered

#134

Post by com2kid »

lelanthran wrote: post #122
n_e wrote: post #106

Earlier quoted context omitted.

> Why is reserving a megabyte of stack space "expensive"? Because if you use one thread for each of your 10,000 idle sockets you will use 10GB to do nothing. So you'll want to use a better architecture such as a thread pool. And if you want your better architecture to be generic and ergonomic, you'll end up with async or green threads.

> Because if you use one thread for each of your 10,000 idle sockets you will use 10GB to do nothing. 1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed. 2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used.

> 1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed

I've written massively concurrent systems where each connection only handled maybe a few kilobytes of data.

Async io is a massive win in those situations.

This describes many rest endpoints. Fetch a few rows from a DB, return some JSON.

Re: What async promised and what it delivered

#135

Post by cmrdporcupine »

eklitzke wrote: post #108
ibraheemdev wrote: post #80

> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space Why is reserving a megabyte of stack space "expensive"? > and takes roughly a millisecond to create I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.

Yeah, none of this makes sense to me. Allocating memory for stack space is not expensive (and the default isn't even 1MB??) because you're just creating a VMA and probably faulting in one or two pages. They also say: >The system spends time managing threads that could be better spent doing useful work. What do they think the async runtime in their language is doing? It's literally doing the same thing the kernel woul…

It's a cargo cult and a bias I see all over the place.

I feel like we're now, what, 20, 25 years on and people still haven't adjusted themselves to the fact that the machines we have now are multicore, have boatloads of cache, or how that cache is shared (or not) between cores.

Nor is there apparently a real understanding of the difference between VSS and RSS.

Nor of the fact that modern machines are really really fast if you can keep stuff in cache. And so you really should be focused on how you can make that happen.

Re: What async promised and what it delivered

#136

Post by cmrdporcupine »

n_e wrote: post #129
lelanthran wrote: post #122

Earlier quoted context omitted.

> Because if you use one thread for each of your 10,000 idle sockets you will use 10GB to do nothing. 1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed. 2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used.

> 1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed. My example (and the c10k problem) is 10k concurrent connections, not 10k concurrent requests. > 2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used. Yes, and that's both memory and cpu usage that isn't needed when using a better co…

> Yes, and that's both memory and cpu usage that isn't needed

No, it literally is not. The "memory" is just entries in a page table in the kernel and MMU. It shouldn't worry you at all.

Nor is the CPU used by the kernel to manage those threads going to be necessarily less efficient than someone's handrolled async runtime. In fact given it gets more eyes... likely more.

The sole argument I can see is just avoiding a handful of syscalls and excessive crossing of the kerneluserspace brain blood barrier too much.

Re: What async promised and what it delivered

#137

Post by gf000 »

Kinrany wrote: post #126
gf000 wrote: post #124

Earlier quoted context omitted.

> This simplifies access to shared state without race conditions But in ordinary JS there just can't be a race condition, everything is single threaded.

And it doesn't actually prevent concurrency.

Sure, but concurrent != parallel. You can't have data races with a single thread of execution - a while loop writing i=0 or i=1 on each iteration is not a data race.

Two async functions doing so is not a data race either.

Re: What async promised and what it delivered

#139

Post by YZF »

layer8 wrote: post #112
YZF wrote: post #100

Earlier quoted context omitted.

Pretty much anything that needs performance and has a lot of relatively light operations is not a candidate for spawning a thread. Context switching and the cost of threads is going to kill performance. A server spawning a thread per request for relatively lightweight request is going to be extremely slow. But sure, if every REST call results in a 10s database query then that's not your bottleneck. A query to a datab…

Server applications don’t spawn threads per request, they use thread pools. The extra context switching due to threads waiting for I/O is negligible in practice for most applications. Asynchronous I/O becomes important when the number of simultaneous requests approaches the number of threads you can have on your system. Many applications don’t come close to that in practice. There’s a benefit in being able to code th…

Thread pools are another variation on the theme. But if your threads block then your pool saturates and you can't process any more requests. So thread pools still need non-blocking operations to be efficient or you need more threads. If you have thread pools you also need a way of communicating with that pool. Maybe that exists in the framework and you don't worry about it as a developer. If you are managing a pool of threads then there's a fair amount of complexity to deal with.

I totally agree there are applications for which this is overkill and adds complexity. It's just a tool in the toolbox. Video games famously are just a single thread/main loop kind of application.

Re: What async promised and what it delivered

#140

Post by delusional »

ibraheemdev wrote: post #80

> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space Why is reserving a megabyte of stack space "expensive"? > and takes roughly a millisecond to create I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.

> Why is reserving a megabyte of stack space "expensive"?

Equally, if a megabyte of stack is a lot for your usecase, can't you just ask pthreads to reserve less? I believe it goes down to like 16k

Post reply on HN