Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Race Patterns in Go

eng.uber.com

Post reply on HN
Print view View original

111–120 of 205 posts

Re: Data Race Patterns in Go

#111

Post by tialaramex »

oconnor663 wrote: post #92
ohazi wrote: post #79

Earlier quoted context omitted.

Hm, maybe I was misremembering. Managed languages like Java do give you memory safety , but I guess data race freedom isn't actually guaranteed. Now that I think about it, this must be the case, right? You have to get `synchronized` right in Java or else you won't get what you expect.

There's a tricky distinction here. I'm pretty sure Java does provide data race freedom, in the specific sense that a data race is "Undefined Behavior caused by a write overlapping with another read or write". The Java standard says that the JVM isn't allowed to trigger this sort of undefined behavior. (Maybe some people say it's technically still a data race? I'm not sure of the right formal definition, but anyway th…

Data Race Freedom is, unsurprisingly, Freedom from Data Races. A Data Race is any time when there's concurrent modification of a memory value, on modern hardware with multiple simultaneous execution contexts those modifications could in some sense happen at the same moment.

[NB: Data Races are a subset of Race Conditions. Race Conditions are sometimes just a fact about the world and you need to write programs that cope with this, but they are not necessarily Data Races, if you copy all the files from folder A to folder B, and then delete folder A, somebody meanwhile adding a file to folder A which you then delete despite not having copied it would be a Race Condition, but it is not a Data Race. ]

The reason you want Data Race Freedom is that it's easy for a programming language to offer Sequential Consistency if you have Data Race Freedom, this guarantee is called SC/DRF.

Why do we want Sequential Consistency? Sequential Consistency is when programs behave as if stuff happened in some sequence. The disk reader gets a block from disk and then the encryptor applies AES/GCM to the block and then the network writer sends the encrypted block to the client. It turns out humans value this very much when trying to reason about any non-trivial program. Get rid of Sequential Consistency and the programmers are just confused and can't solve bugs.

So, we want SC/DRF and in most languages you get that by being very careful to obey the rules to avoid Data Races. If you screw up, you don't have Sequential Consistency. In most languages you lose more than that (in C or C++ you immediately have Undefined Behaviour, game over, all bets are off), but even just losing Sequential Consistency is very bad news.

Safe Rust promises DRF and thus SC. So instead of being very careful you can just write safe Rust.

Re: Data Race Patterns in Go

#112

Post by shp0ngle »

smasher164 wrote: post #48

This definitely matches my experience using Go at my previous organization. 1. Closures and concurrency really don't mix well. The loop variable capture in particular is very pernicious. There's an open issue to change this behavior in the language: https://github.com/golang/go/issues/20733 . 2. Yep. I've seen this problem in our codebase. I've grown to just be very deliberate with data that needs to be shared. Put i…

The first link notes that they don’t plan to change the semantics of the range loop, but they might dis-allow referencing it

which is… uhh ok maybe?

But go is really trying to keep backwards compatibility so they won’t just change this

Re: Data Race Patterns in Go

#113

Post by morelisp »

masklinn wrote: post #102
morelisp wrote: post #101

Earlier quoted context omitted.

If you start worrying about "what if underlying the Go runtime happens?" you'll find a lot worse than realloc. Luckily, in the absence of runtime bugs, you don't have to think about it.

It's not a concern about C, it's a concern about the underlying possibility expressed in C terms: sizeclass arenas are useful to an allocator, that means slack, which means the opportunity for in-place allocation resizing. You advocated the use of a very specific behaviour of `append` as a DID and possibly a correctness requirement of programs. My worry is about whether this behaviour is a hard specification of the G…

Sorry, this is nonsensically mixed up. Even if it reallocs in place, which it may be free to do, the language semantics still guarantee only one observer gets that extended space. Otherwise you would need to worry about this even when dealing with immutable structures like concatenating strings.

Re: Data Race Patterns in Go

#114

Post by Thaxll »

Groxx wrote: post #27
bumper_crop wrote: post #3

Sorry to say, but these hit close to home for me. A lot of the synchronization paradigms in Go are easy to misuse, but lead the author into thinking it's okay. the WaitGroup one is particularly poignant for me, since the race detector doesn't catch it. I'll add one other data race goof: atomic.Value. Look at the implementation. Unlike pretty much every other language I've seen, atomic.Value isn't really atomic, since…

The lack of generics has forced all Go concurrency to be intrusive (i.e. implemented by the person using literally any concurrency ), and yeah. It's horrifyingly error-prone in my experience. It means everyone needs to be an expert, and lol, everyone is not an expert. Generics might save us from the simple, mechanical flaws. Expect to see `Locker ` and `Atomic ` types cropping up. And unbounded buffered thread-safe q…

"because there are an absurd amount of races in nearly all of the popular libraries"

This is fud, I ran the race detector with a lot of popular lib and I never found issues like that.

But since you're claiming there are issues everywhere, do you have examples?

Re: Data Race Patterns in Go

#115

Post by jasonhansel »

I think the root cause of a lot of these data races is that Go has no way of marking variables/fields/pointers/etc. as immutable or constant. This makes it easy to lose track of shared mutable state.

It's not just data races--it's also logical races, which are near-impossible to detect or prevent without something like transactional memory.

Re: Data Race Patterns in Go

#116

Post by jasonhansel »

hintymad wrote: post #47
metadat wrote: post #8

> 2. Slices are confusing types that create subtle and hard-to-diagnose data races The "Slices" example is just nasty! Like, this is just damning for Go's promise of "_relatively_ easy and carefree concurrency" . Think about it for a second or two, >> The reference to the slice was resized in the middle of an append operation from another async routine. What exactly happens in these cases? How can I trust myself, as…

> I never would've guessed that in 2022, Java would start looking more and more appealing in new ways. I don't quite understand the hatred (to the point of shouting "using Java? Over my dead body), especially in startups, towards Java. I mean, it's a language, big deal. Java's ecosystem more than enough offsets whatever inefficiencies in the language itself, at least for building many of the internal CRUD services. B…

I actually think the Java ecosystem is part of why people dislike it. Java seems to attract a lot of extremely heavyweight frameworks (like Spring) that are too complex to fully understand and too heavyweight to make sense for most projects.

Re: Data Race Patterns in Go

#117

Post by josefx »

Groxx wrote: post #27
bumper_crop wrote: post #3

Sorry to say, but these hit close to home for me. A lot of the synchronization paradigms in Go are easy to misuse, but lead the author into thinking it's okay. the WaitGroup one is particularly poignant for me, since the race detector doesn't catch it. I'll add one other data race goof: atomic.Value. Look at the implementation. Unlike pretty much every other language I've seen, atomic.Value isn't really atomic, since…

The lack of generics has forced all Go concurrency to be intrusive (i.e. implemented by the person using literally any concurrency ), and yeah. It's horrifyingly error-prone in my experience. It means everyone needs to be an expert, and lol, everyone is not an expert. Generics might save us from the simple, mechanical flaws. Expect to see `Locker ` and `Atomic ` types cropping up. And unbounded buffered thread-safe q…

> I also really wonder where all these "go makes concurrency a first-class concept" claims come from,

Given that some of the main architects behind Go had K&R C as background I wouldn't be surprised if "first-class" just meant that the language defines both a memory model and primitives for threading. C had neither until it basically adopted both from C++11.

Re: Data Race Patterns in Go

#118

Post by jonathanstrange »

tialaramex wrote: post #111
oconnor663 wrote: post #92

Earlier quoted context omitted.

There's a tricky distinction here. I'm pretty sure Java does provide data race freedom, in the specific sense that a data race is "Undefined Behavior caused by a write overlapping with another read or write". The Java standard says that the JVM isn't allowed to trigger this sort of undefined behavior. (Maybe some people say it's technically still a data race? I'm not sure of the right formal definition, but anyway th…

Data Race Freedom is, unsurprisingly, Freedom from Data Races. A Data Race is any time when there's concurrent modification of a memory value, on modern hardware with multiple simultaneous execution contexts those modifications could in some sense happen at the same moment. [NB: Data Races are a subset of Race Conditions. Race Conditions are sometimes just a fact about the world and you need to write programs that co…

AFAIK, you still have to be very careful since data races based on data dependencies can never be excluded in general, that is theoretically not possible. What you get is a guarantee that your program is not in an undefined state. There are still plenty of ways to shoot yourself in the foot with incorrect synchronization.

Re: Data Race Patterns in Go

#119

Post by crabbygrabby »

ohazi wrote: post #68

A dig against Rust I sometimes hear is "Oh, data race freedom isn't such a big deal, if you really need it, a garbage collected language like Java will give you that guarantee." So now I'm hearing that Go, a garbage collected language, doesn't guarantee data race freedom? I guess it's garbage collected but not "managed" by a runtime or something? Why go to all that effort to get off of C++ just to stop 30% short? The…

Go has some appeal in general. It's super easy to stand up webby glue appy things in go, and it has a solid cloud ecosystem, maybe the best cloud ecosystem.

That said, people ragging on rust pushing that trope are basically just making stuff up to hate on it. Anyone who looks into the language and views programming languages as tools and understands these issues gets why someone might use rust.

But yea, it's ironic... Especially seeing how many times I've seen smart colleagues get go concurrency wrong.

Re: Data Race Patterns in Go

#120

Post by crabbygrabby »

laerus wrote: post #86
DominoTree wrote: post #59

Seems like Rob Pike and co may have failed "The key point here is our programmers... They’re not capable of understanding a brilliant language... So, the language that we give them has to be easy for them to understand"

Yep, if Google wasn't behind Go the language would have already been history like so many other half baked technologies. It won't be long untill people will talk about Golang like they do about JavaScript.

JavaScript is nutty but let's be honest, js is one of the most widely used programming languages. For better and for worse.

I don't think go lang will die out because it does get some things right. Unfortunately, there's still a bit of things going wrong.

Post reply on HN