Live data from Hacker News

Error handling with errgroups

dolthub.com

11–18 of 18 posts

Re: Error handling with errgroups

#11
post #2

While we’re at it, can someone explain Go contexts for me? I suspect they are a way to keep thread safe data that would otherwise be global or, in the C world, static, but I’m not quite sure. The go documentation just describes how to use them, not why they should be used.

Contexts are a workaround for the inability to terminate a Goroutine, by canceling tasks in the Goroutine.

Suppose that you spawn a Goroutine for handling an HTTP request from client A. While the request is being processed (e.g. calling DB, external services, etc.), client A drops the TCP connection. With contexts we can notify the handler to cancel any ongoing/pending task. Otherwise, the handler will be still running the remaining tasks, hence wasting resources.

Note that if we pass a context to a function, it entirely depends on the function as to when or whether it will cancel its tasks. It's a cooperative multitasking after all.

In Rust, it's easy to cancel a future: Just drop it. It's not that the Rust's approach is perfect. Rust has the opposite problem: Since a future can be dropped anytime (in any await point), dropping a future in the middle of an execution might lead to an inconsistent state, if the future isn't properly implemented.

Re: Error handling with errgroups

#12

> When we Wait() on the errgroup, it will cancel its context even when every spawned go routine has returned nil. I wonder why. It's documented [1] but seems strange to me. It enforces that errgroup is single-use (rather than cyclical), but I might do that by outright panicking if Go is called after Wait returns. Goroutines in the group might launch others in the group, but presumably not after they've finished, so I…

The context derived from WithContext is scoped to the lifetime of errgroup (after they all return, or any return with an error). This way the function passed to Group.Go can spawn other go routines that will get properly cleaned up. Why those other go routines would spawn processes that they don't clean up otherwise? They probably shouldn't, but canceling the context should always be safe, and is more likely to safel…

> canceling the context should always be safe, and is more likely to safely handle what otherwise would be a bug, than it is to cause a bug.

They gave an example of a bug it caused, and I haven't seen any examples of bugs it would have prevented, so I'm not sure I agree.

Re: Error handling with errgroups

#13

> When we Wait() on the errgroup, it will cancel its context even when every spawned go routine has returned nil. I wonder why. It's documented [1] but seems strange to me. It enforces that errgroup is single-use (rather than cyclical), but I might do that by outright panicking if Go is called after Wait returns. Goroutines in the group might launch others in the group, but presumably not after they've finished, so I…

Contexts spawn a goroutine to block on the Done channel. You always need to call cancel otherwise you'll leak that goroutine. It's an annoying thing that comes from the fact that contexts are implemented in user space and there's no way to block on a channel without block a goroutine.

Re: Error handling with errgroups

#14

Earlier quoted context omitted.

The context derived from WithContext is scoped to the lifetime of errgroup (after they all return, or any return with an error). This way the function passed to Group.Go can spawn other go routines that will get properly cleaned up. Why those other go routines would spawn processes that they don't clean up otherwise? They probably shouldn't, but canceling the context should always be safe, and is more likely to safel…

> canceling the context should always be safe, and is more likely to safely handle what otherwise would be a bug, than it is to cause a bug. They gave an example of a bug it caused, and I haven't seen any examples of bugs it would have prevented, so I'm not sure I agree.

I think the contract in the context package is that WithCancel may leak resources if cancel is not called and the parent context is cancelable but never canceled. At least, that seems to be the behavior I see in the implementation here:

https://github.com/golang/go/blob/master/src/context/context...

So in this case, errgroup probably doesn't have a choice and needs to call cancel() at some point.

Re: Error handling with errgroups

#15
post #14

Earlier quoted context omitted.

> canceling the context should always be safe, and is more likely to safely handle what otherwise would be a bug, than it is to cause a bug. They gave an example of a bug it caused, and I haven't seen any examples of bugs it would have prevented, so I'm not sure I agree.

I think the contract in the context package is that WithCancel may leak resources if cancel is not called and the parent context is cancelable but never canceled. At least, that seems to be the behavior I see in the implementation here: https://github.com/golang/go/blob/master/src/context/context... So in this case, errgroup probably doesn't have a choice and needs to call cancel() at some point.

Ahh. Yeah, that's unfortunate but explains this choice.

btw, I realized I'm not being entirely fair in saying it caused a bug. There would have been one anyway. If conflictsBuilder.Wait() failed, it wouldn't cancel the other two operations. It'd wait them out even though the overall operation was doomed. Maybe not a very noticeable bug under normal conditions but still not right.

Re: Error handling with errgroups

#16
post #3
post #2

While we’re at it, can someone explain Go contexts for me? I suspect they are a way to keep thread safe data that would otherwise be global or, in the C world, static, but I’m not quite sure. The go documentation just describes how to use them, not why they should be used.

They're for cancellation. Commonly, when your goroutine is blocking while waiting for an IO, like a HTTP response, you want to set a deadline. You can do that by creating a suitable context and passing it to the API that makes the request. When the deadline is reached, the API will return an error. You can also cancel contexts yourself, e.g. in response to user input. It can be any blocking API, not just IO, for exam…

> The API has to be designed to support context cancellation, and most of the standard library is.

Important parts of the standard library are not designed for contexts. Let's say I want to write data to a file and then cancel that. How do I do that? `os.File.Write`, `io.Copy`, `ioutil.WriteFile`... none of these let me cancel a write.

The `io.Writer` interface not supporting contexts also means things like the compress and archive stdlib packages don't support contexts, among several others.

Most of the rest of it has had contexts bolted on in nonstandard ways. If you want to use contexts, you can no longer use "net.Dial" or "net.DialTimeout", but must instead use the awkward "(&net.Dialer{}).DialContext" method. The http package has similar issues, including non-idiomatic context usage.

The go stdlib was mostly built before contexts existed, has promised backwards compatibility, and it really shows.

Re: Error handling with errgroups

#17
post #16
post #3

Earlier quoted context omitted.

They're for cancellation. Commonly, when your goroutine is blocking while waiting for an IO, like a HTTP response, you want to set a deadline. You can do that by creating a suitable context and passing it to the API that makes the request. When the deadline is reached, the API will return an error. You can also cancel contexts yourself, e.g. in response to user input. It can be any blocking API, not just IO, for exam…

> The API has to be designed to support context cancellation, and most of the standard library is. Important parts of the standard library are not designed for contexts. Let's say I want to write data to a file and then cancel that. How do I do that? `os.File.Write`, `io.Copy`, `ioutil.WriteFile`... none of these let me cancel a write. The `io.Writer` interface not supporting contexts also means things like the compr…

Good points!

Re: Error handling with errgroups

#18
post #2

While we’re at it, can someone explain Go contexts for me? I suspect they are a way to keep thread safe data that would otherwise be global or, in the C world, static, but I’m not quite sure. The go documentation just describes how to use them, not why they should be used.

Every one of these answers was golden! Thanks to you I became an appreciably better programmer in less than 5 minutes.
Post reply on HN