Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Conc: Better Structured Concurrency for Go

github.com

Post reply on HN
Print view View original

121–130 of 162 posts

Re: Conc: Better Structured Concurrency for Go

#121

Post by groestl »

bsaul wrote: post #119
groestl wrote: post #53

Earlier quoted context omitted.

> assuming panics are normal will take you down some paths that make it basically impossible to write reliable software Na, citation needed. Assuming "panics are normal" is just extrapolating from "errors are normal". It makes reliable software more reliable.

it's pretty obvious that it could influence new developers into the wrong direction though. Saying things like "ha, let's not bother checking this, at worst it'll just panic and i'll simply abort the request". Which would definitely impact the quality of the software overall in a bad way.

I'd not be so sure. Accepting that everything that can fail will fail shaped me as a young developer, and "Exceptional C++" had a huge influence on me. Now my approach for new code I review is this:

* Make sure you support properly unrolling the stack

* Keep a clean failure boundary, probably somewhere on top of your loop

* Fastidiously check your preconditions

* Fail brutally if they're not met

* Improve from there

Re: Conc: Better Structured Concurrency for Go

#122

Post by tsegratis »

jerf wrote: post #85
klodolph wrote: post #22

Earlier quoted context omitted.

I don’t think this is really a question of whether your code is imperative, since Haskell code will terminate just as surely as Go code if you try to access an array element out of range. (Haskell’s lazy evaluation just makes it a bit harder to catch, since you need to force evaluation of the thunk within the catch statement, and it’s far too easy to end up passing your thunk to somebody who won’t catch the exception…

It is a fundamental problem with the imperative paradigm because of the equivalent of: lock.Take() thingThatMayCrash() lock.Release() The imperative/structured paradigm is that those statements are evaluated in order. While this is not the only way of writing Go, it is legal Go, and the runtime must account for it. Paradigms for which that is not fundamentally true have different options available to them. One of tho…

Nice points

I suppose the core issue is transactions. Erlang could still fail with

  them ! {sub, n}
  crash()
  me ! {add, n}
But typically Erlang's isolated processes means many transactions can fail or rollback in isolation

I suppose instead transactions could be a core language semantic. But hard to do, and erlang actors gives a lot, for very little overhead

Re: Conc: Better Structured Concurrency for Go

#123

Post by Smurfix »

kevmo314 wrote: post #7

The WaitGroup looks suspiciously like errgroup, which even has the .WithMaxGoroutines() functionality: https://pkg.go.dev/golang.org/x/sync/errgroup > A frequent problem with goroutines in long-running applications is handling panics. A goroutine spawned without a panic handler will crash the whole process on panic. This is usually undesirable. In go land, this seems desirable. Recoverable errors should be propagated…

How the heck should a library author know whether an error is recoverable or not? In many situations that depends entirely on the caller.

Not-so-contrived example: a config options contains a value that leads to a division by zero in some library I'm using. I don't want that to crash my whole program, I want to tell the user "hey this part didn't work" but be able to continue with the other parts.

Without writing a ton of boilerplate code.

Re: Conc: Better Structured Concurrency for Go

#124

Post by Smurfix »

philosopher1234 wrote: post #120
masklinn wrote: post #118

Earlier quoted context omitted.

Ah, you’re a fanboy. You could have just put that upfront I would not have wasted my time taking you seriously.

You wrote "That nothing about it’s good". That's stupid. I agree we are wasting our time.

Well, Ok, so the main thing that's good about it is that it's dead easy to start a goroutine. But that's actually bad, just like randomly jumping to any piece of code whatsoever is bad in in a procedural language.

Read the Structured Concurrency notes (https://vorpus.org/blog/notes-on-structured-concurrency-or-g...) for the reasons why.

Re: Conc: Better Structured Concurrency for Go

#125

Post by philosopher1234 »

Smurfix wrote: post #124
philosopher1234 wrote: post #120

Earlier quoted context omitted.

You wrote "That nothing about it’s good". That's stupid. I agree we are wasting our time.

Well, Ok, so the main thing that's good about it is that it's dead easy to start a goroutine. But that's actually bad, just like randomly jumping to any piece of code whatsoever is bad in in a procedural language. Read the Structured Concurrency notes ( https://vorpus.org/blog/notes-on-structured-concurrency-or-g... ) for the reasons why.

I don’t know. I think the culture around go encourages effective/safe strategies for using go routines + channels. I think when channels are used as intended they can be quite effective. They’re not technically more safe than anything else, but I think the fact that they make concurrency more manageable is good.

Re: Conc: Better Structured Concurrency for Go

#126

Post by jaitsu »

erik_seaberg wrote: post #78
loeg wrote: post #74

Earlier quoted context omitted.

Go panics should not be used for very common errors .

A lot of very common operations can panic: division, dereferencing a pointer, invoking an interface method, indexing/slicing an array/slice/string, asserting the type of an interface, and converting a slice to pointer to array. It’s possible to check, but I’ve never seen a tool that verifies you never use any of these without checking. You also have to check for nil channels, though they block forever (maybe consumin…

You can recover from a panic though, so if you are implementing something that may panic you should have some sensible defer/recover in there if you can't afford to have your process crash.

Re: Conc: Better Structured Concurrency for Go

#127

Post by jerf »

tsegratis wrote: post #122
jerf wrote: post #85

Earlier quoted context omitted.

It is a fundamental problem with the imperative paradigm because of the equivalent of: lock.Take() thingThatMayCrash() lock.Release() The imperative/structured paradigm is that those statements are evaluated in order. While this is not the only way of writing Go, it is legal Go, and the runtime must account for it. Paradigms for which that is not fundamentally true have different options available to them. One of tho…

Nice points I suppose the core issue is transactions. Erlang could still fail with them ! {sub, n} crash() me ! {add, n} But typically Erlang's isolated processes means many transactions can fail or rollback in isolation I suppose instead transactions could be a core language semantic. But hard to do, and erlang actors gives a lot, for very little overhead

Erlang qua Erlang doesn't have any transactions, either, so Erlang qua Erlang can't fail that way.

In general, whatever it was the crashing process was doing can still have failed. This obviously doesn't magically fix database transactions, or make it so files are never half written because it crashed in the meantime, or any other "external" action that you can still have failed to protect properly from crashes. It only means that the runtime can't get stuck due to locks failing to be released.

But hey, that's something!

Re: Conc: Better Structured Concurrency for Go

#129

Post by preseinger »

klabb3 wrote: post #54
preseinger wrote: post #38

Earlier quoted context omitted.

It's a good package in general, save for the panic handling. Panics should not be handled in this way. Remove that wart, and it's solid.

Is the alternative to crash and restart the whole process on panic? It would make sense if someone wants to write an in-process supervisor (similar to Erlang?) but this would be basically a main-wrapper - not a per-http-request thing (because Golang http itself would be corrupted). I don’t know enough to say where the crash isolation boundary should best lie, BUT, assuming that you can catch panics at all, it makes a…

> Is the alternative to crash and restart the whole process on panic?

Yes.

> assuming that you can catch panics at all

A panic may happen to be safe to catch and recover from, but this isn't guaranteed, and can't be assumed in general. It's only safe to recover from a panic which you know is benign -- in all cases, across all build and runtime architectures. This is possible in packages that you fully control and which have no external dependencies, or (by fiat) in stdlib packages like net/http. But it's not the case for your service binary with a go.mod that's 100 lines long.

Re: Conc: Better Structured Concurrency for Go

#130

Post by klabb3 »

pkolaczk wrote: post #109
klabb3 wrote: post #107

Earlier quoted context omitted.

If you’re only going to learn one concurrency paradigm, it has to be message passing, because that’s the only one that works between machines. The only reason why people don’t see the similarities when it comes to in-process concurrency is that the request-response paradigm muddies the waters. Anyway, that’s the world we live, so people have to work around the N+1 problem and similar, living in a mixed sync-async wor…

> because that’s the only one that works between machines There is a non-negligible number of problems that don't need multiple machines and are much easier to deal with using other concurrency paradigms. When the language forces only one model onto the developer, things are actually more complex than they need to be.

Yeah, I agree, but Go is allowing traditional mutices that work the same way (except the rwlock discrepancy as pointed out by the paper). Channels aren’t forced, just encouraged. There are cases where locks are preferable.

My main criticism against the paper is that what tipped the scales for concurrency bugs is the “blocking category” which of course is going to be higher with message passing since it’s a blocking paradigm. Locks only block until unlocked, and isn’t used as a general purpose signaling mechanism.

The “non-blocking” bugs are over represented by locks, which are generally more serious (non-deterministic data races vs deterministic deadlocks).

That said, I have criticism against Go as well. The decision to not implement structured concurrency (fire-and-forget goroutines) makes these deadlocks largely go unnoticed. If goroutines had ownership or scope, runtime detection a la ‘-race’ would help detect a lot of these bugs.

Post reply on HN