Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coroutines for Go

research.swtch.com

Post reply on HN
Print view View original

31–40 of 191 posts

Re: Coroutines for Go

#31

Post by yakubin »

klabb3 wrote: post #26
masklinn wrote: post #9

Earlier quoted context omitted.

Goroutines + channels add an enormous amount of overhead. Using them as iterators is basically insane.

It’s not insane at all. How did you come to that conclusion? * Mutex lock+unlock: 10ns * Chan send buffered: 21ns * Try send (select with default): 3.5ns Missing from here is context switches. In either case, the overhead is proportional to how fast each iteration is. I have channels of byte slices of 64k and the channel ops don’t even make a dent compared to other ops, like IO. You should absolutely use channels if…

> Missing from here is context switches.

Exactly. From rsc's previous post[1]:

> On my laptop, a C thread switch takes a few microseconds. A channel operation and goroutine switch is an order of magnitude cheaper: a couple hundred nanoseconds. An optimized coroutine system can reduce the cost to tens of nanoseconds or less.

[1]: https://research.swtch.com/pcdata>

Re: Coroutines for Go

#32

Post by ketchupdebugger »

I'm not sure why author is advocating for single threaded patterns in a multithreaded environment. Not sure why he's trying to limit himself like this. The magic of goroutines is that you can use all of your cores easily not just one. Python and Lua has no choice.

Re: Coroutines for Go

#33

Post by yakubin »

ketchupdebugger wrote: post #32

I'm not sure why author is advocating for single threaded patterns in a multithreaded environment. Not sure why he's trying to limit himself like this. The magic of goroutines is that you can use all of your cores easily not just one. Python and Lua has no choice.

Coroutines are a control-flow mechanism. They're a single-threaded pattern in as much as for loops are a single-threaded pattern. Ability to write multithreaded programs does not exclude the need for good single-threaded tools.

Re: Coroutines for Go

#34

Post by djha-skin »

alphazard wrote: post #22

It looks like a lot of people are missing the point here. Yes a coroutine library would be a worse/more cumbersome way to do concurrency than the go keyword. The use case motivating all the complexity is function iterators, where `range` can be used on functions of type `func() (T, bool)`. That has been discussed in the Go community for a long time, and the semantics would be intuitive/obvious to most Go programmers.…

Still a kitchen sink move, though, isn't it?

Like, no careful thinking and good 80/20 solution this time. Just "huh, we'd need coroutines to do this `right` so let's just do that"

When they added generics, they really, really thought long and hard, and came up with a compromise that was brilliant and innovative in the balance it struck.

I would have hoped to see something like that here, like "we're adding this one feature to goroutines to have control in specific situations" feels like something that would be better than "we're going full Rust on this problem and just adding it."

Re: Coroutines for Go

#35

Post by rcme »

alphazard wrote: post #22

It looks like a lot of people are missing the point here. Yes a coroutine library would be a worse/more cumbersome way to do concurrency than the go keyword. The use case motivating all the complexity is function iterators, where `range` can be used on functions of type `func() (T, bool)`. That has been discussed in the Go community for a long time, and the semantics would be intuitive/obvious to most Go programmers.…

What is wrong with:

    for {
        next := getNext()
        ...
    }
What is the advantage of writing this as:

    for next := range getNext {
        ...
    }

Re: Coroutines for Go

#36

Post by bouk »

rcme wrote: post #35
alphazard wrote: post #22

It looks like a lot of people are missing the point here. Yes a coroutine library would be a worse/more cumbersome way to do concurrency than the go keyword. The use case motivating all the complexity is function iterators, where `range` can be used on functions of type `func() (T, bool)`. That has been discussed in the Go community for a long time, and the semantics would be intuitive/obvious to most Go programmers.…

What is wrong with: for { next := getNext() ... } What is the advantage of writing this as: for next := range getNext { ... }

The first doesn't do control flow

Re: Coroutines for Go

#37

Post by rcme »

bouk wrote: post #36
rcme wrote: post #35

Earlier quoted context omitted.

What is wrong with: for { next := getNext() ... } What is the advantage of writing this as: for next := range getNext { ... }

The first doesn't do control flow

Depends how getNext is defined.

Re: Coroutines for Go

#38

Post by cookiengineer »

Zach_the_Lizard wrote: post #8

I have written Go professionally for many years now and don't want to see it become something like the Python Twisted / Tornado / whatever frameworks. The go keyword nicely prevents the annoying function coloring problem, which causes quite a bit of pain. Sometimes in high performance contexts I'd like to be able to do something like e.g. per CPU core data sharding, but this proposal doesn't scratch those kinds of it…

Can you maybe share hints to better channel management patterns/frameworks?

Usually my goroutines related code feels like a mess because of them, and I don't know how to make them "more clean" (whatever that means). Any hints proven to be maintainable patterns would be highly appreciated.

Re: Coroutines for Go

#39

Post by adrusi »

jerf wrote: post #25

I'm not 100% sure this is the case, but I believe the context of this goes something like this. As Go has added generics, there are proposals to add generic data structures like a Set. Generics solve almost every problem with that, but there is one conspicuous issue that remains for a data structure: You can iterate over a slice or a map with the "range" keyword, and that yields special iteration behavior, but there…

A goroutine really is implemented as a coroutine though. It's just that without this runtime optimization, there hasn't been a way to interact with them without letting a multithreaded scheduler manage their execution.

I don't remember where I came across this, but many years ago I saw python-style generators termed "semi-coroutines" which I'm a fan of. Python (frustratingly) doesn't implement them this way, but the beauty of generators is that by limiting the scope where execution can be suspended to the highest-level stack frame, you can statically determine how much memory is needed for the (semi-)coroutine, so it doesn't require a single allocation.

Zig takes that a step farther by considering recursive functions second-class, which lets the compiler keep track of the maximum stack depth of any function, and thereby allow you to allocate a stack frame for any non-recursive function inside of another stack frame, enabling zero-allocation full coroutines, as long as the function isn't recursive.

That would... probably be overkill for Go, since marginal allocations are very cheap, and you're already paying the runtime cost for dynamic stack size, so the initial allocation can be tiny.

I would love to see full proper coroutine support make it to Go, freeing users of the overhead of the multithreaded scheduler on the occasions where coroutine patterns work best. I remember back in 2012 or so, looking at examples of Go code that showed off goroutine's utility as control flow primitives even when parallelism wasn't desired and being disappointed that those patterns would likely be rare on account of runtime overhead, and sure enough I hardly ever see them.

Re: Coroutines for Go

#40

Post by morelisp »

djha-skin wrote: post #34
alphazard wrote: post #22

It looks like a lot of people are missing the point here. Yes a coroutine library would be a worse/more cumbersome way to do concurrency than the go keyword. The use case motivating all the complexity is function iterators, where `range` can be used on functions of type `func() (T, bool)`. That has been discussed in the Go community for a long time, and the semantics would be intuitive/obvious to most Go programmers.…

Still a kitchen sink move, though, isn't it? Like, no careful thinking and good 80/20 solution this time. Just "huh, we'd need coroutines to do this `right` so let's just do that" When they added generics, they really , really thought long and hard, and came up with a compromise that was brilliant and innovative in the balance it struck. I would have hoped to see something like that here, like "we're adding this one…

This has been under discussion for a long time.

https://github.com/golang/go/issues/43557

https://github.com/golang/go/discussions/56413

I'm not sure Russ's personal blog is any kind of official statement "this is what we're doing" yet?

Post reply on HN