Goroutines Under the Hood (2020)
osmh.dev
Goroutines Under the Hood (2020)
1–10 of 55 posts
Re: Goroutines Under the Hood (2020)
#2I mean it has many diagrams and logical explanation of Goroutines and concurrency concepts in general but it is definitely not under the hood descriptions.
Re: Goroutines Under the Hood (2020)
#3I love Go and goroutines, but...
> A newly minted goroutine is given a few kilobytes
a line later
> It is practical to create hundreds of thousands of goroutines in the same address space
So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to want to stick to a few thousand goroutines. I can almost guarantee you that you have something better to do with those GBs of memory than store goroutine stacks.
Asking the scheduler to handle scheduling 100s of Ks of goroutines is also not a great idea in my experience either.
Re: Goroutines Under the Hood (2020)
#4One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction pointer.
You probably can't beat that in user space, especially if you want to preempt threads yourself. You'd have to check after every step, or profile your own process or something like that. And indeed, Go's scheduler is cooperative.
But then, why can't you get the performance of Goroutines with OS threads? Is it just because of legacy issues? Or does it only work with cooperative threading, which requires language support?
One thing I'm missing from that article is how the cooperativeness is implemented. I think in Go (and in Java's Project Loom), you have "normal code", but then deep down in network and IO functions, you have magic "yield" instructions. So all the layers above can pretend they are running on regular threads, and you avoid the "colored function problem", but you get runtime behavior similar to coroutines. Which only works if really every blocking IO is modified to include yielding behavior. If you call a blocking OS function, I assume something bad will happen.
Re: Goroutines Under the Hood (2020)
#5One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…
Just the interrupt itself is going to cost a couple of order of magnitude more than the whole userspace context switch.
Re: Goroutines Under the Hood (2020)
#6I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…
Why is spending GB on stack space a bad thing? Ultimately, in a server, you need to store state for each request. Whether that's on the stack or heap, it's still memory that necessarily has to be used.
Re: Goroutines Under the Hood (2020)
#7I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…
> So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to want to stick to a few thousand goroutines. I can almost guarantee you that you have something better to do with those GBs of memory than store goroutine stacks.
You lost me in a couple places:
1) "GBs of memory overhead" being a lot. A rule of thumb I've seen in a datacenter situation is that (iirc) 1 hyperthread and 6 GiB of RAM are roughly equivalent in cost. (I'm sure it varies over processor/RAM generations, so you should probably check this on your platform rather than take my word for it.) I think most engineers are way too stingy with RAM. It often makes sense to use more of it to reduce CPU, and to just spend it on developer convenience. Additionally, often one goroutine matches up to one incoming or outgoing socket connection (see below). How much RAM are you spending per connection on socket buffers? Probably a lot more than a few kilobytes...
2) The idea that you target a certain number of goroutines. They model some activity, often a connection or request. I don't target a certain number of those; I target filling the machine. (Either the most constrained resource of CPU/RAM/SSD/disk/network if you're the only thing running there, or a decent chunk of it with Kubernetes or whatever, bin-packing to use all dimensions of the machine as best as possible.) Unless the goroutines' work is exclusively CPU-bound, of course, then you want them to match the number of CPUs available, so thousands is too much already.
Re: Goroutines Under the Hood (2020)
#8I mean it has many diagrams and logical explanation of Goroutines and concurrency concepts in general but it is definitely not under the hood descriptions.
I came here to write the same thing. Things I had hoped this would go into is how goroutines grow their stacks, and how they are preempted.
Re: Goroutines Under the Hood (2020)
#9One thing that really goes against my intuition is that user space threads (lightweight treads, goroutines) are faster than kernel threads. Without knowing too much assembly, I would assume any modern processor would make a context switch a one instruction affair. Interrupt -> small scheduler code picks the thread to run -> LOAD THREAD instruction and the processor swaps in all the registers and the instruction point…
One of the issues is that OS schedulers are complex, and actually much more expensive than context switch itself. You can mitigate this with user-mode scheduling of kernel threads: https://www.youtube.com/watch?v=KXuZi9aeGTw
Re: Goroutines Under the Hood (2020)
#10I love Go and goroutines, but... > A newly minted goroutine is given a few kilobytes a line later > It is practical to create hundreds of thousands of goroutines in the same address space So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to wan…
> So it's not practical to create 100s of Ks of goroutines - it's possible, sure, but because you incur GBs of memory overhead if you are actually creating that many goroutines means that for any practical problem you are going to want to stick to a few thousand goroutines. I can almost guarantee you that you have something better to do with those GBs of memory than store goroutine stacks. You lost me in a couple pla…
> I think most engineers are way too stingy with RAM. It often makes sense to use more of it to reduce CPU, and to just spend it on developer convenience.
Hey! That's Java's argument!