Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goroutines Under the Hood (2020)

osmh.dev

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Goroutines Under the Hood (2020)

#11

Post by sumy23 »

aaronbwebber wrote: post #3

I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…

My experience is that whatever you’re doing with the go routine is usually a bottleneck before the go routine itself. E.g. if you make a network request, you become network bound before memory bound from go routines.

Re: Goroutines Under the Hood (2020)

#13

Post by masklinn »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

> And indeed, Go's scheduler is cooperative.

It hasn't been cooperative for a few versions now, the scheduler became preemptive in 1.14. And before that there were yield points at every function prolog (as well as all IO primitives) so there were relatively few situations where cooperation was necessary.

> Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair.

Any context switch (to the kernel) is expensive, and way more than a single operation. The kernel also has a ton of stuff to do, it's not just "picks the thread to run", you have to restore the ip and sp, but also may have to restore FPU/SSE/AVX state (AVX512 is over 2KB of state), traps state.

Kernel-level context switching costs on the order of 10x what userland context switching does: https://eli.thegreenplace.net/2018/measuring-context-switchi...

> LOAD THREAD

There is no load thread instruction

Re: Goroutines Under the Hood (2020)

#14

Post by asien »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

>I would assume any modern processor would make a context switch a one instruction affair.

Has been the historic assumption, has been proven to be wrong by every possible benchmark.

Consider tech empower[0] for raw stack performance , runtime level threads outperform IO threads since OS thread were designed to be mapped on physicals cores.

This is very expensive and inefficient.

Creating one thread for every request you have ( Apache + PHP ) will exhaust the hardware after a few thousands/qps target.

Runtime can indeed have millions of those “lightweight threads” without killing your machine since they create a pool from physical threads and tap into IO events to efficiently switch or resume contexts. This is by far much faster.

[0] https://www.techempower.com/benchmarks/#section=data-r20&hw=...

Re: Goroutines Under the Hood (2020)

#16

Post by jaytaylor »

masklinn wrote: post #13
captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

> And indeed, Go's scheduler is cooperative. It hasn't been cooperative for a few versions now, the scheduler became preemptive in 1.14. And before that there were yield points at every function prolog (as well as all IO primitives) so there were relatively few situations where cooperation was necessary. > Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruc…

> It hasn't been cooperative for a few versions now, the scheduler became preemptive in 1.14. And before that there were yield points at every function prolog (as well as all IO primitives) so there were relatively few situations where cooperation was necessary.

Since co-op was most unnecessary, do you know why it was changed to preemptive or what the specific cases were that are resolved with preemptive scheduling?

Re: Goroutines Under the Hood (2020)

#17

Post by barsonme »

aaronbwebber wrote: post #3

I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…

In addition to the other comments about memory usage, I’ll mention that there is a proposal (that’s either going to make it into Go 1.19 or 1.20?) that uses heuristics to determine a good starting stack size for goroutines.

Re: Goroutines Under the Hood (2020)

#18

Post by duped »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

> I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction pointer.

It can't be a single instruction, since the details of what a "context" contains depends on the OS and ABI. For example on Linux, the signal mask is a part of the OS thread context (but usually not user thread contexts) which requires a syscall to retrieve it from kernel memory before saving it in the context.

The reason why user threads are so much faster than OS threads is precisely because it can be reduced to a handful of instructions without caring about all the details that OS threads need to care about.

> Which only works if really every blocking IO is modified to include yielding behavior. If you call a blocking OS function, I assume something bad will happen.

That's exactly what Go does, they introduce yield points into function prologues and i/o ops. You don't have direct FFI calls in Go so it's not as big of an issue. It's roughly the same problem as GC safepoints in multithreaded interpreters that support FFI.

Re: Goroutines Under the Hood (2020)

#19

Post by pron »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

There are a few reasons why context switching in user mode could be faster, but that has little if anything to do with the performance benefits of usermode threads. The performance benefit of usermode threads is a result of their quantity, due to Little's law. They're not "faster", just more numerous, and that's what you need for higher throughput. More precisely, OS threads, because of their scarcity, introduce an artificial bound on throughput that's lower than what the hardware can support, and usermode threads remove that bound.

More here: https://inside.java/2020/08/07/loom-performance/

As to why it's hard for the OS to allow that many threads, the OS would need to keep thread stacks small and resizable, and that is hard to do if you don't know the specifics of how the language uses the stack. For example, to accommodate low-level languages that allow pointers into the stack you would need to manipulate virtual memory (to keep addresses valid), but that only works at a page granularity, or to introduce split stacks, which would require a new kind of ABI known to compilers (and would probably have a cost to performance).

Re: Goroutines Under the Hood (2020)

#20

Post by YZF »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

There is a ton of context associated with OS/kernel threads. Virtual memory, security, I/O. While there is some hardware acceleration for those in modern processors there isn't anything like LOAD THREAD and even with CPU support it's still very expensive.

You get an interrupt, then the kernel needs to load its own context (the tables it needs to access), then the kernel needs to do the expensive switch.

In user space you have a lot less context. The actual switching is pretty much the cost of a function call. If you need preemption that's a different story and mostly depends on what facilities are available for that. Inserting preemption checks is a little hacky (hello Go ;) ) but what can you do.

EDIT: It's worthwhile noting there's indirect costs like caches being stale. Lightweight/green threads will often work on shared data structures so the caches are more likely to have something useful in them. They may even share the code.

Post reply on HN