Conc: Better Structured Concurrency for Go • Hacker News Board

Re: Conc: Better Structured Concurrency for Go

#141

Post by preseinger » Thu, Jan 12, 2023, 11:32 PM UTC

erik_seaberg wrote: post #78

loeg wrote: post #74

Earlier quoted context omitted.

Go panics should not be used for very common errors .

A lot of very common operations can panic: division, dereferencing a pointer, invoking an interface method, indexing/slicing an array/slice/string, asserting the type of an interface, and converting a slice to pointer to array. It’s possible to check, but I’ve never seen a tool that verifies you never use any of these without checking. You also have to check for nil channels, though they block forever (maybe consumin…

Division by zero, dereferencing a nil pointer, invoking methods on a nil interface, invalid indexing of an array, unchecked type assertions -- these are not common operations! These are always easily detectable programmer errors.

Re: Conc: Better Structured Concurrency for Go

#142

Post by preseinger » Thu, Jan 12, 2023, 11:46 PM UTC

Groxx wrote: post #51

preseinger wrote: post #40

Earlier quoted context omitted.

> Index slice out of bounds? panic. Close a channel twice? Panic. Incorrect type assertion? Panic. Dereference nil pointer? Panic. These are all really bad things which should never survive to production code. It is not difficult to detect and prevent them. > I would argue that all of these examples which are the most common in my experience are “goroutine scoped” because the goroutine was aborted before they potenti…

> These are all really bad things which should never survive to production code. It is not difficult to detect and prevent them. This is equivalent to saying "out of bounds memory writes are not difficult to detect and prevent in C code". Like actually equivalent (possibly worse), not just "well if you squint they look similar". Of course it's not hard most of the time . Being perfect is beyond hard though. And if yo…

Go has much stronger memory safety guarantees than C does. They aren't really comparable.

Re: Conc: Better Structured Concurrency for Go

#143

Post by Groxx » Fri, Jan 13, 2023, 1:34 AM UTC

preseinger wrote: post #137

Groxx wrote: post #50

Earlier quoted context omitted.

Since defers run during panics for exactly this reason, no. You can in fact guarantee that is not the case. Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics.

> Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics. https://go.dev/doc/effective_go#panic > The usual way to report an error to a caller is to return an error as an extra return value. . . . But what if the error is unrecoverable? Sometimes the program simply cannot…

That's a style decision, not a correctness issue. You are claiming it is a correctness issue.

Re: Conc: Better Structured Concurrency for Go

#144

Post by erik_seaberg » Fri, Jan 13, 2023, 1:48 AM UTC

preseinger wrote: post #140

erik_seaberg wrote: post #90

Earlier quoted context omitted.

The problem also affects structs that happen to have a private map or cache or callback anywhere within. https://go.dev/play/p/uP-vjpvuhku

Obviously `interface{}` values are not comparable?

The comparison is explicitly allowed in the language spec, there’s no warning for doing it, and it often works depending on the types. It’s a data-dependent runtime error, which is usually hard to guarantee test coverage for.

Re: Conc: Better Structured Concurrency for Go

#145

Post by preseinger » Fri, Jan 13, 2023, 2:02 AM UTC

Groxx wrote: post #143

preseinger wrote: post #137

Earlier quoted context omitted.

> Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics. https://go.dev/doc/effective_go#panic > The usual way to report an error to a caller is to return an error as an extra return value. . . . But what if the error is unrecoverable? Sometimes the program simply cannot…

That's a style decision, not a correctness issue. You are claiming it is a correctness issue.

It is absolutely a correctness issue. Panics do not provide safety guarantees that generalize enough that it is safe to arbitrary recover from them. The statement in the previous sentence is not a subjective opinion, it's a statement of fact. I'm not sure how else to convey this information.

Re: Conc: Better Structured Concurrency for Go

#146

Post by yakaccount4 » Fri, Jan 13, 2023, 2:18 AM UTC

preseinger wrote: post #134

yakaccount4 wrote: post #103

Earlier quoted context omitted.

> I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own. I'd rather have the control to log the panic on a service rather than it forcibly dying and taking down any other connections with it. Kube…

I don't think I'm effectively communicating the impact of handling a panic and continuing program execution. A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program. If the program continues, that change will go undetected, and can have results that make the program completely nondeterministic. This isn't a doom and gloom, sky-is-falling pr…

> A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program ... This isn't a doom and gloom, sky-is-falling prognostication, it's literally what is defined by the spec and memory model of the language.

I do not think you are correct. Go has a class of unrecoverable panics for this specific reason. Go also runs deferred functions after a recoverable panic, so the notion that it's unsafe to handle it, or continue executiona after doesn't hold at all - it is literally a first-class feature of the language.

I have not seen an instance of a recoverable panic that is raised _after_ such a fatal operation. If you have an example of such, I would love to see it.

Re: Conc: Better Structured Concurrency for Go

#147

Post by preseinger » Fri, Jan 13, 2023, 2:29 AM UTC

yakaccount4 wrote: post #146

preseinger wrote: post #134

Earlier quoted context omitted.

I don't think I'm effectively communicating the impact of handling a panic and continuing program execution. A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program. If the program continues, that change will go undetected, and can have results that make the program completely nondeterministic. This isn't a doom and gloom, sky-is-falling pr…

> A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program ... This isn't a doom and gloom, sky-is-falling prognostication, it's literally what is defined by the spec and memory model of the language. I do not think you are correct. Go has a class of unrecoverable panics for this specific reason. Go also runs deferred functions after a recov…

What are unrecoverable panics vs. recoverable panics? Where is that distinction defined?

Re: Conc: Better Structured Concurrency for Go

#148

Post by yakaccount4 » Fri, Jan 13, 2023, 3:00 AM UTC

preseinger wrote: post #147

yakaccount4 wrote: post #146

Earlier quoted context omitted.

> A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program ... This isn't a doom and gloom, sky-is-falling prognostication, it's literally what is defined by the spec and memory model of the language. I do not think you are correct. Go has a class of unrecoverable panics for this specific reason. Go also runs deferred functions after a recov…

What are unrecoverable panics vs. recoverable panics? Where is that distinction defined?

There seems to not be any standard list of unrecoverable panics/aborts, but this Stackoverflow post [1] has a list of a few.

As far as the user/developers are concerned, it doesn't matter too much, since you have no option to recover them, but it would be nice if it was explained if defers are still ran. I'm assuming they are not.

1. https://stackoverflow.com/questions/57486620/are-all-runtime...

Re: Conc: Better Structured Concurrency for Go

#149

Post by preseinger » Fri, Jan 13, 2023, 3:14 AM UTC

yakaccount4 wrote: post #148

preseinger wrote: post #147

Earlier quoted context omitted.

What are unrecoverable panics vs. recoverable panics? Where is that distinction defined?

There seems to not be any standard list of unrecoverable panics/aborts, but this Stackoverflow post [1] has a list of a few. As far as the user/developers are concerned, it doesn't matter too much, since you have no option to recover them, but it would be nice if it was explained if defers are still ran. I'm assuming they are not. 1. https://stackoverflow.com/questions/57486620/are-all-runtime...

If there is no way for callers to reliably distinguish recoverable panics from unrecoverable panics, then this distinction doesn't really exist, does it? Panics are panics.

Re: Conc: Better Structured Concurrency for Go

#150

Post by yakaccount4 » Fri, Jan 13, 2023, 3:32 AM UTC

preseinger wrote: post #149

yakaccount4 wrote: post #148

Earlier quoted context omitted.

There seems to not be any standard list of unrecoverable panics/aborts, but this Stackoverflow post [1] has a list of a few. As far as the user/developers are concerned, it doesn't matter too much, since you have no option to recover them, but it would be nice if it was explained if defers are still ran. I'm assuming they are not. 1. https://stackoverflow.com/questions/57486620/are-all-runtime...

If there is no way for callers to reliably distinguish recoverable panics from unrecoverable panics, then this distinction doesn't really exist, does it? Panics are panics.

I'm not sure what point you are trying to make anymore.

Of course you cannot distinguish between unrecoverable and recoverable panics, because by definition an unrecoverable panic is not recoverable. There is no caller to distinguish between it - it is killed.