Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goroutines Under the Hood (2020)

osmh.dev

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Goroutines Under the Hood (2020)

#31

Post by morelisp »

scottlamb wrote: post #7
aaronbwebber wrote: post #3

I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…

> So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to want to stick to a few thousand goroutines. I can almost guarantee you that you have something better to do with those GBs of memory than store goroutine stacks. You lost me in a couple pla…

I agree that GBs for 100Ks of go routines is not in some sense "a lot", in that you might still be using memory pretty effectively. But I don't see that a "6GB vs 1 core" tradeoff makes any sense to talk about.

We have HTTP ingress that needs ~100 cores but could theoretically all fit in 1GB. We have k/v stores that need only 16 cores but would like 500GB. And we have data points at most places in-between. We can't give the ingress 600GB instead, and we can't give the k/v stores 100 cores. So the fact they're financially interchangeable is meaningless for capacity planning.

Arguably, for most code and especially in a GCd language, using less memory and less CPU go hand-in-hand.

Re: Goroutines Under the Hood (2020)

#32

Post by ibraheemdev »

pron wrote: post #19
captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

There are a few reasons why context switching in user mode could be faster, but that has little if anything to do with the performance benefits of usermode threads. The performance benefit of usermode threads is a result of their quantity , due to Little's law. They're not "faster", just more numerous, and that's what you need for higher throughput. More precisely, OS threads, because of their scarcity, introduce an…

> More precisely, OS threads, because of their scarcity, introduce an artificial bound on throughput that's lower than what the hardware can support, and usermode threads remove that bound.

Why are OS threads scarce? The OS allocates thread stacks lazily. Given a kernel stack of ~8kb (two pages) and a user stack of ~4kb, one could spawn 100k threads with just over 1GB. A userspace runtime will allow you to bring that number down, but if you're at the scale of concurrency it is unlikely to matter much.

Re: Goroutines Under the Hood (2020)

#33

Post by bogomipz »

In the conclusion the author states:

>"Go run-time scheduler multiplexes goroutines onto threads and when a thread blocks, the run-time moves the blocked goroutines to another runnable kernel thread to achieve the highest efficiency possible."

Why would the Go run-time move the blocked goroutines to another runnable kernel thread? If it is currently blocked it won't be schedulable regardless no?

Re: Goroutines Under the Hood (2020)

#34

Post by scottlamb »

morelisp wrote: post #31
scottlamb wrote: post #7

Earlier quoted context omitted.

> So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to want to stick to a few thousand goroutines. I can almost guarantee you that you have something better to do with those GBs of memory than store goroutine stacks. You lost me in a couple pla…

I agree that GBs for 100Ks of go routines is not in some sense "a lot", in that you might still be using memory pretty effectively. But I don't see that a "6GB vs 1 core" tradeoff makes any sense to talk about. We have HTTP ingress that needs ~100 cores but could theoretically all fit in 1GB. We have k/v stores that need only 16 cores but would like 500GB. And we have data points at most places in-between. We can't g…

If you are in aggregate making good use of all the dimensions of the available machines/VMs, great. I think often people either leave one dimension unused or (when buying their own hardware / selecting a VM shape) could be adding more RAM cheaply.

> Arguably, for most code and especially in a GCd language, using less memory and less CPU go hand-in-hand.

Agreed in general. Even in a non-GC language, less dense data structures means worse CPU cache utilization. But on the other hand, memoization and the like can provide a real trade-off.

In this case, I don't think it's costing much CPU. The GC isn't traversing beyond the bounds of the stack, and it mostly shouldn't end up in the CPU cache either. (Just a partial cache line at the boundary, and some more after a goroutine's stack shrinks or the goroutine exits.)

Re: Goroutines Under the Hood (2020)

#35

Post by guenthert »

uluyol wrote: post #6
aaronbwebber wrote: post #3

I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…

Why is spending GB on stack space a bad thing? Ultimately, in a server, you need to store state for each request. Whether that's on the stack or heap, it's still memory that necessarily has to be used.

Despite popular belief, not everything is a (web) server. I can imagine many threads to be appealing in e.g. simulations.

Re: Goroutines Under the Hood (2020)

#37

Post by jjnoakes »

uluyol wrote: post #6
aaronbwebber wrote: post #3

I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…

Why is spending GB on stack space a bad thing? Ultimately, in a server, you need to store state for each request. Whether that's on the stack or heap, it's still memory that necessarily has to be used.

If you need the stack space then there is no difference. The difference arises because if you preallocate all that stack space using worst case stack sizes and don't use most of it, you've wasted lots of memory.

Also there is a ton of nuance here like overcommitted pages and large address spaces which mitigate some of those downsides.

Re: Goroutines Under the Hood (2020)

#38

Post by heavyset_go »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

I don't have any specific recommendations to give you, but skim through an operating systems text book, or college course that puts its slides and whatnot online, when it comes to kernel context switching. It'll give you an idea of what kind of work a kernel must do when context switching between threads and processes, and why userspace multitasking can be more efficient.

Re: Goroutines Under the Hood (2020)

#39

Post by danachow »

captainmuon wrote: post #4

One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…

> modern processor would make a context switch a one instruction affair.

Reduced instruction set (complexity) is a hallmark of modern processor designs, not the other way around.

You might want to read about what is involved in a task switch (either "thread" with same memory mapping, or "process") but it is not something that is conducive to reasonably carry out in one instruction.

Re: Goroutines Under the Hood (2020)

#40

Post by kangda123 »

bogomipz wrote: post #33

In the conclusion the author states: >"Go run-time scheduler multiplexes goroutines onto threads and when a thread blocks, the run-time moves the blocked goroutines to another runnable kernel thread to achieve the highest efficiency possible." Why would the Go run-time move the blocked goroutines to another runnable kernel thread? If it is currently blocked it won't be schedulable regardless no?

I haven't read the article but, generally, main thread pool is designed to utilise the whole processor. So on a 12-core system there will be 12 OS threads. No point in having more. But, if the program opens just 12 big files all threads become blocked and that's obviously tragic: other routines are starved, network sessions time out, timers don't run - chaos! Thus, whenever a thread in the main pool is blocked, you move the thread outside and spawn a fresh new one that can keep going through the compute.

There's also a subtler benefit. Each user thread has a context, e.g. its local run queue. Now, if thread blocks, others need to help it out and steal its work. Go improves that by having a nice handover system, so no random stealing is necessary. Further, by taking context off blocked threads, it keeps all the tasks more centralised. There is probably at most a few tens of processors on any common hardware but there could be thousands threads. It's better to tie runqueues to the former rather than the latter.

Post reply on HN