Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Race Patterns in Go

eng.uber.com

Post reply on HN
Print view View original

151–160 of 205 posts

Re: Data Race Patterns in Go

#151

Post by sethammons »

Groxx wrote: post #141
sethammons wrote: post #123

Earlier quoted context omitted.

What would you expect that code to do? I'd expect to have two different slices, s2 and s3, to contain all the same elements aside from the last. [a, b, c, x] and [a, b, c, y] and s1 remains [a, b, c]

When printing s1, s2, then s3: It does exactly that, yes: https://go.dev/play/p/rs2FeK_QUjs [a b c] [a b c x] [a b c y] But maybe it doesn't: https://go.dev/play/p/Na-eL0sOV9e [a b c] [a b c y] So... maybe they share the same backing array? Lets try setting s2[0] to "z" after appending with the original code: https://go.dev/play/p/mAB-gUb0shB [a b c] [z b c x] [a b c y] Apparently not. But also apparently yes? https:…

Amazing, very interesting

Re: Data Race Patterns in Go

#152

Post by Groxx »

saghm wrote: post #76
Groxx wrote: post #64

Earlier quoted context omitted.

Context for cancellation and replacing thread-local variables (or indeed any way to observe your "current" thread) is one of the things I like tbh. Though Context has abysmal performance implications. But yeah, I want a goroutine handle with a "Wait()" method. Ideally also returning the results. Like most languages. It'd eliminate a ton of manual mutex and channel use that doesn't need to exist. --- Re thread vs task…

Yeah, I generally think of the word "thread" as referring to OS threads and/or "green" threads depending on context (and in this case I thought it was clear what you were referring to!), but since the person who responded to you made the distinction, I figured I'd use their terminology when explaining what I thought you were saying.

I was just leveraging your already-top reply to reply to both of you, sorry about that :) I should've just done two comments. I think you and I are on the same page here.

I think the main reason it doesn't exist is that go had no generics. It'd need to be another custom-generic type (Future[T] basically), and it would make it harder to pass around, just like channels. But since channels are generally intrusively-added, they aren't part of the return signature, so they avoid that generic-return issue. E.g. every "worker pool" accepts a `func()` and callers need to coordinate return values via channels, instead of needing to return a `func[T]()` reference which they have been unable to do until recently (to some degree at least).

Though they probably could've just said "use a Future[interface{}]", like they did for every other generic collection type.

Plus it'd take some of the emphasis off channels, and they seem to really not want to do that. If they were focused on usability instead of channels and select, they'd let us park on multiple mutexes just like channels, just like the runtime does internally a lot to implement all this... but no. Imagine a world where you could `select { case mut.Lock(): ...}`...

Re: Data Race Patterns in Go

#153

Post by dwrodri »

Is it just me, or is Golang's concurrency a very double edged sword? My exposure to goroutines and channels was mind-blowing, but I still really struggle reading through Go code and understanding what's going on in memory. The way that Go "abstracts away" ownership feels more like it's hiding important details rather than unnecessary minutia.

Here's a simple question that's stumped me for some time: if multiple go routines are popping values out of a channel, does the channel need a mutex? Why do the "fan-out, fan-in?" examples in the "Pipelines and Cancellation" post on the Go blog not require mutex locks? Link here: https://go.dev/blog/pipelines

Stuff like that, along with the ambiguity of intializing stuff by value vs using make, the memory semantics of some of the primitives (slices, channels, etc). None of it was like "of course". If something is a reference, I'd rather the language tell me it's a reference. Maybe I'm still too new to the language.

Re: Data Race Patterns in Go

#154

Post by bombela »

dwrodri wrote: post #153

Is it just me, or is Golang's concurrency a very double edged sword? My exposure to goroutines and channels was mind-blowing, but I still really struggle reading through Go code and understanding what's going on in memory. The way that Go "abstracts away" ownership feels more like it's hiding important details rather than unnecessary minutia. Here's a simple question that's stumped me for some time: if multiple go ro…

I am pretty sure the channels are safe to use with multiple producers and multiple consumers (mpmc). But somehow I cannot easily find any official doc clarifying that.

Go doesn't do anything to help you with memory safety around concurrency. And the design of the language is also not helping you avoid logical bugs.

After using Rust, all other imperative languages feel like using an angle grinder with a wood saw blade and no guard. Sure you can do really good if you are careful. But thing will go sideways remarkably quickly. And with the constant urgency of shipping for yesterday. It makes sense most programs look like the aftermath of The Boys show.

Re: Data Race Patterns in Go

#155

Post by gigatexal »

yencabulator wrote: post #149
gigatexal wrote: post #98

In the closure example does declaring a new variable and setting its value to the iterative or the thing being passed in, does that mitigate the pass by reference issue?

Yes. for i := 0; i https://go.dev/play/p/P7TunJCL7RS

Thank you!

The example is brilliant. Thanks.

Re: Data Race Patterns in Go

#156

Post by Groxx »

bombela wrote: post #154
dwrodri wrote: post #153

Is it just me, or is Golang's concurrency a very double edged sword? My exposure to goroutines and channels was mind-blowing, but I still really struggle reading through Go code and understanding what's going on in memory. The way that Go "abstracts away" ownership feels more like it's hiding important details rather than unnecessary minutia. Here's a simple question that's stumped me for some time: if multiple go ro…

I am pretty sure the channels are safe to use with multiple producers and multiple consumers (mpmc). But somehow I cannot easily find any official doc clarifying that. Go doesn't do anything to help you with memory safety around concurrency. And the design of the language is also not helping you avoid logical bugs. After using Rust, all other imperative languages feel like using an angle grinder with a wood saw blade…

They are MPMC except for closing (which panics if anything writes after close), yes.

Re: Data Race Patterns in Go

#157

Post by icedchai »

laerus wrote: post #86
DominoTree wrote: post #59

Seems like Rob Pike and co may have failed "The key point here is our programmers... They’re not capable of understanding a brilliant language... So, the language that we give them has to be easy for them to understand"

Yep, if Google wasn't behind Go the language would have already been history like so many other half baked technologies. It won't be long untill people will talk about Golang like they do about JavaScript.

They said the same thing about JavaScript in the 90's. If Netscape wasn't behind it... Over 25 years later, it's still going strong, even though the entire HTML/CSS/JS model of "app development" is and always was half baked.

Re: Data Race Patterns in Go

#158

Post by icedchai »

voidfunc wrote: post #73
hintymad wrote: post #47

Earlier quoted context omitted.

> I never would've guessed that in 2022, Java would start looking more and more appealing in new ways. I don't quite understand the hatred (to the point of shouting "using Java? Over my dead body), especially in startups, towards Java. I mean, it's a language, big deal. Java's ecosystem more than enough offsets whatever inefficiencies in the language itself, at least for building many of the internal CRUD services. B…

Written a lot of Java, Python, and Go in my career... every single time I see someone take a hardline stance against Java it's always because they had one particular bad experience with it 15-20 years ago and couldn't bend it to their will like Python or Lisp. Or they fought Maven, or some other ancillary tool. Or they rail on the generics and yet the use-cases they come up with for true reified generics are generall…

Mostly Java, Python, and a bit of Go here, also. If you're not sure what language to develop a back end service in, you'll rarely go wrong by picking Java. The JVM absolutely is rock solid. The number of libraries and frameworks available is amazing. If you stay way from heavy weight frameworks and use something leaner like Spring Boot or Dropwizard, you'll be fine.

Re: Data Race Patterns in Go

#159

Post by bombela »

tialaramex wrote: post #124
Thaxll wrote: post #114

Earlier quoted context omitted.

"because there are an absurd amount of races in nearly all of the popular libraries" This is fud, I ran the race detector with a lot of popular lib and I never found issues like that. But since you're claiming there are issues everywhere, do you have examples?

I'd say there's an excellent chance your assumptions about types you got from "popular libraries" is more conservative and that's why you never detected any issues. For example take the JSON decoder. If you have several tasks which can use some data from a JSON blob in parallel, is it OK if they all just share the same JSON decoder? If you're horrified because this seems obviously like a bad idea, that'll be why you…

Replying here to the two siblings comments being confused about the decoder example.

What tialaramex is saying, is that if you have a stream of JSON values, you create a JSON decoder over it. Then every time you call the decode() method, you get the next decoded JSON value.

Then you want to process the JSON values concurrently.

Rephrased, the question was what would happen if you were to have every concurrent task call the decode() method whenever it wants a new value to work on?

It would probably be a data race cluster fuck. But you might find this type of mistakes everywhere in Go. I myself fought things like that in many libraries.

One such occurrence I recall was in the Google Cloud Pub Sub client library. It basically did something similar to this example. Trying to offer concurrency over a stream of messages. It would fail very rarely. And pretty much always passe the race detector. It wasn't fun to debug.

Re: Data Race Patterns in Go

#160

Post by [deleted] »

tapirl wrote: post #140
bumper_crop wrote: post #22

Earlier quoted context omitted.

That's why I opened with "Look at the implementation". Go is unable to store the type and the pointer at the same time, so it warps what "atomic" means. Pretty much every other language has atomic mean "one of these will win, one will lose". Go says "one will win, one will panic and destroy the goroutine. In fact, it's even worse than that. If the Store() caller goes to sleep between setting the type and storing the…

> Pretty much every other language has atomic mean "one of these will win, one will lose" Could you elaborate how "much every other language" implement it?

[deleted]
Post reply on HN