Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Conc: Better Structured Concurrency for Go

github.com

Post reply on HN
Print view View original

131–140 of 162 posts

Re: Conc: Better Structured Concurrency for Go

#131

Post by atombender »

vbezhenar wrote: post #95
bit_flipper wrote: post #93

Earlier quoted context omitted.

It's fine to dislike Go's philosophy on error handling, but in order to save you and your co-workers a lot of headache down the road, I'd recommend you just use another language. This is not something you want to do in Go and very few folks who program in Go would be happy to work with that style of code. In case you actually are interested in Go's take on stack traces: You are intended to annotate your errors so you…

Do you suggest me to annotate my errors with "file:line" strings? Do you suggest me to grep source files with error messages to deduce stack trace? Because that's what I'm doing right now when I have to deal with stacktrace-less errors and it's not pleasant.

In the sense that errors are values, you should not normally need to hunt for the origin of errors, and so stack traces should unnecessary.

Many errors don't need metadata. For example, io.EOF signaling the end of a stream is a normal condition. But there are those "true" errors which are unexpected conditions indicative of a bug or a scenario that should be handled differently. To make those easier to find, you can annotate the error values to make it clear what the origin was. This is why there's now a well-established convention to wrap with contextual descriptions at every point in the chain.

In about 8 years of working full-time with Go, the number of situations where I've struggled to hunt down the source of an error value is pretty close to zero. Not zero, but close.

Re: Conc: Better Structured Concurrency for Go

#132

Post by kubanczyk »

tjdevries wrote: post #67
Groxx wrote: post #46

https://github.com/sourcegraph/conc/blob/main/iter/iter.go#L... // Map applies f to each element of input, returning the mapped result. func Map[T, R any](input []T, f func(*T) R) []R { res := make([]R, len(input)) ForEachIdx(input, func(i int, t *T) { res[i] = f(t) }) return res } Seems a little silly to farm that off to a custom func when you could just write the for-loop, but it's probably fine / may be no differe…

I'm not sure what you don't like? Sorry, must just be missing your point.

I'm guessing it's about creating a GOMAXPROCS pool of goroutines which are then (wastefully) competing with each other for an atomic loop counter.

It got me curious what's faster when looping over a slice of million elements: to use a million separate goroutines or to use that pool.

Re: Conc: Better Structured Concurrency for Go

#133

Post by preseinger »

groestl wrote: post #53
preseinger wrote: post #44

Earlier quoted context omitted.

> In practice, panics happen. I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own. > I'm accepting the risk that my application is left in an inconsistent state, Inconsistent state makes it impo…

> assuming panics are normal will take you down some paths that make it basically impossible to write reliable software Na, citation needed. Assuming "panics are normal" is just extrapolating from "errors are normal". It makes reliable software more reliable.

Panics are categorically different than errors. Errors are normal, panics are not normal.

Re: Conc: Better Structured Concurrency for Go

#134

Post by preseinger »

yakaccount4 wrote: post #103
preseinger wrote: post #44

Earlier quoted context omitted.

> In practice, panics happen. I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own. > I'm accepting the risk that my application is left in an inconsistent state, Inconsistent state makes it impo…

> I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own. I'd rather have the control to log the panic on a service rather than it forcibly dying and taking down any other connections with it. Kube…

I don't think I'm effectively communicating the impact of handling a panic and continuing program execution. A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program. If the program continues, that change will go undetected, and can have results that make the program completely nondeterministic. This isn't a doom and gloom, sky-is-falling prognostication, it's literally what is defined by the spec and memory model of the language.

Re: Conc: Better Structured Concurrency for Go

#135

Post by preseinger »

groestl wrote: post #121
bsaul wrote: post #119

Earlier quoted context omitted.

it's pretty obvious that it could influence new developers into the wrong direction though. Saying things like "ha, let's not bother checking this, at worst it'll just panic and i'll simply abort the request". Which would definitely impact the quality of the software overall in a bad way.

I'd not be so sure. Accepting that everything that can fail will fail shaped me as a young developer, and "Exceptional C++" had a huge influence on me. Now my approach for new code I review is this: * Make sure you support properly unrolling the stack * Keep a clean failure boundary, probably somewhere on top of your loop * Fastidiously check your preconditions * Fail brutally if they're not met * Improve from there

Right, all of these are good points, but the problem is that the "failure boundary" of a panic is the entire process. You can't constrain it, or assume that it's scoped to a single goroutine. Errors do not have this property.

Re: Conc: Better Structured Concurrency for Go

#136

Post by preseinger »

jasonhansel wrote: post #64
preseinger wrote: post #44

Earlier quoted context omitted.

> In practice, panics happen. I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own. > I'm accepting the risk that my application is left in an inconsistent state, Inconsistent state makes it impo…

> An account value that previously had balance = 0 may now have balance = 1000. Is this acceptable risk? Your entire web app process crashes due to a panic every time a request triggers an extremely rare edge case. A hacker discovers this and uses it to conduct a DoS attack. Is this acceptable risk?

Yes, definitely preferable! Denial of service is definitely better than invalid state, right?

Re: Conc: Better Structured Concurrency for Go

#137

Post by preseinger »

Groxx wrote: post #50
preseinger wrote: post #44

Earlier quoted context omitted.

> In practice, panics happen. I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own. > I'm accepting the risk that my application is left in an inconsistent state, Inconsistent state makes it impo…

Since defers run during panics for exactly this reason, no. You can in fact guarantee that is not the case. Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics.

> Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics.

https://go.dev/doc/effective_go#panic

> The usual way to report an error to a caller is to return an error as an extra return value. . . . But what if the error is unrecoverable? Sometimes the program simply cannot continue. For this purpose, there is a built-in function panic that in effect creates a run-time error that will stop the program

Panics express unrecoverable failures. This is plainly stated in the language documentation. There are exceptions to this rule, but they are exceptional.

Re: Conc: Better Structured Concurrency for Go

#138

Post by preseinger »

aetherane wrote: post #79
ikiris wrote: post #71

Earlier quoted context omitted.

Why the heck are you writing web apps that panic?

It is pretty easy to have accidental panics in Go, for instance due to a runtime assertion that unexpectedly failed

Runtime assertions without defensive checks are programmer errors that are not difficult to spot in code review and should not be expected to make it to deployed code.

    // RED FLAG
    x := y.(type)

    // good
    x, ok := y.(type)
    if !ok { return an error }

Re: Conc: Better Structured Concurrency for Go

#139

Post by preseinger »

erik_seaberg wrote: post #63
preseinger wrote: post #40

Earlier quoted context omitted.

> Index slice out of bounds? panic. Close a channel twice? Panic. Incorrect type assertion? Panic. Dereference nil pointer? Panic. These are all really bad things which should never survive to production code. It is not difficult to detect and prevent them. > I would argue that all of these examples which are the most common in my experience are “goroutine scoped” because the goroutine was aborted before they potenti…

x == y can panic if interface values contain incomparable fields in unexported nested structs, how would I check for that? Should we let it become a query of death and bet thousands of peers’ jobs on it never happening?

Link to an example? I don't think this is true, unless you're playing stupid games with your code, which wouldn't pass code review.

Re: Conc: Better Structured Concurrency for Go

#140

Post by preseinger »

erik_seaberg wrote: post #90
assbuttbuttass wrote: post #83

Earlier quoted context omitted.

It's true, but you'd never really write code like this https://go.dev/play/p/r9NkQb6bQTx

The problem also affects structs that happen to have a private map or cache or callback anywhere within. https://go.dev/play/p/uP-vjpvuhku

Obviously `interface{}` values are not comparable?
Post reply on HN