Data Race Patterns in Go • Hacker News Board

Re: Data Race Patterns in Go

#91

Post by oconnor663 » Sat, Jun 11, 2022, 6:55 AM UTC

sa46 wrote: post #71

ohazi wrote: post #68

A dig against Rust I sometimes hear is "Oh, data race freedom isn't such a big deal, if you really need it, a garbage collected language like Java will give you that guarantee." So now I'm hearing that Go, a garbage collected language, doesn't guarantee data race freedom? I guess it's garbage collected but not "managed" by a runtime or something? Why go to all that effort to get off of C++ just to stop 30% short? The…

A data race and garbage collection are unrelated. A data race occurs when: > two or more threads in a single process access the same memory location concurrently, and at least one of the accesses is for writing. Rust provides compile time protection against data races with the borrow checker. Go provides good but imperfect runtime detection of data races with the race detector. Like most things in engineering, either…

They're unrelated in theory, but in practice a lot of garbage collected languages do try to turn data races into defined behavior. Java requires the JVM to implement some defined semantics for data races, though I think they're still considered terribly confusing in practice. Python prevents data races with the GIL, and JS prevents them by either not having threads at all or not letting them share memory. I think Go is actually somewhat unique among modern, GC'd languages in that data races in Go are true UB (albeit with lots of best-effort checks).

Re: Data Race Patterns in Go

#92

Post by oconnor663 » Sat, Jun 11, 2022, 7:04 AM UTC

ohazi wrote: post #79

beltsazar wrote: post #72

Earlier quoted context omitted.

> "Oh, data race freedom isn't such a big deal, if you really need it, a garbage collected language like Java will give you that guarantee." Java programs aren't guaranteed to be free of data race. Java spec guarantees that if that happens, there will be no undefined behavior (like in C++).

Hm, maybe I was misremembering. Managed languages like Java do give you memory safety , but I guess data race freedom isn't actually guaranteed. Now that I think about it, this must be the case, right? You have to get `synchronized` right in Java or else you won't get what you expect.

There's a tricky distinction here. I'm pretty sure Java does provide data race freedom, in the specific sense that a data race is "Undefined Behavior caused by a write overlapping with another read or write". The Java standard says that the JVM isn't allowed to trigger this sort of undefined behavior. (Maybe some people say it's technically still a data race? I'm not sure of the right formal definition, but anyway the important thing is that in Java the UB doesn't happen here.) However, what happens when you do that can still be extremely tricky, and I think the Java compiler is still allowed to reorder reads and writes in ways that'll be extremely confusing if you have code that looks like a data race. It won't give an attacker arbitrary code execution, but it's very likely still a bug.

Re: Data Race Patterns in Go

#93

Post by jordanbeiber » Sat, Jun 11, 2022, 7:09 AM UTC

hintymad wrote: post #42

> and contains approximately 2,100 unique Go services (and growing). A side topic: this is really not something to be proud of. There used to be more people than quantity of work in Uber and engineers fought for credits by building bogus decomposed services, and the sheer number of services seems indicate it's still so.

If they still use cadence/temporal[0] extensively this kind of blurs the concept of technical ”services”.

We’ve started to use it (temporal) a bit for general automations, and it’s pretty great. Monorepo with a lot of different activities (“microservices”) makes sense.

The activites are orchestrated in workflows (much like DDD “sagas”) and scheduled via temporal. This gives awesome introspection and observability.

[0] https://temporal.io/

Re: Data Race Patterns in Go

#94

Post by oconnor663 » Sat, Jun 11, 2022, 7:23 AM UTC

jchw wrote: post #9

metadat wrote: post #8

> 2. Slices are confusing types that create subtle and hard-to-diagnose data races The "Slices" example is just nasty! Like, this is just damning for Go's promise of "_relatively_ easy and carefree concurrency" . Think about it for a second or two, >> The reference to the slice was resized in the middle of an append operation from another async routine. What exactly happens in these cases? How can I trust myself, as…

Slices may just be one of the best and worst parts of Go. They're cumbersome, their behavior sometimes feels 'inexplicable,' and even as an experienced developer you are likely to eventually fallen into one of the traps where your 'obvious' code isn't so obvious. That said... when programming in programming languages without a slice type, I always want to have one. And though it's confusing at times, the design does…

> it's hard to think of how you would improve on the actual underlying design.

I'm biased as a Rust fan in general, but I think Rust pretty much nails this. Rust distinguishes between a borrowing view of variable length (a slice, spelled &[T]) and an owned allocation of variable length (usually a Vec). Go uses the same type for both, which makes the language smaller, but it leads to confusion about who's pointing to what when a slice is resized.

Re: Data Race Patterns in Go

#95

Post by [deleted] » Sat, Jun 11, 2022, 7:25 AM UTC

jchw wrote: post #65

likeabbas wrote: post #39

Earlier quoted context omitted.

Good point. Right now I kind of see modern programming as two fold: 1) Loosely typed to get you what you want faster, but with some mistakes, and 2) Strongly typed that forces you to try harder, but ultimately better I'm usually happier with the latter. I find I become far more frustrated when I try to write python than I do something like Rust just because I know when I write Python that I will have mistakes I'll ha…

I greatly prefer Rust to Python, but slightly prefer Go to Rust in many situations. Python even with MyPy just feels sloppy, whereas Go does offer decently strong typing at least. As for Go vs Rust, the answer is in the ecosystem, and I blame that on the difficulty of maintaining high quality Rust crates. I know not everyone agrees with this explanation, but I think Rust being complicated makes it hard to offer zero…

[deleted]

Re: Data Race Patterns in Go

#96

Post by dthul » Sat, Jun 11, 2022, 7:25 AM UTC

jchw wrote: post #65

likeabbas wrote: post #39

Earlier quoted context omitted.

Good point. Right now I kind of see modern programming as two fold: 1) Loosely typed to get you what you want faster, but with some mistakes, and 2) Strongly typed that forces you to try harder, but ultimately better I'm usually happier with the latter. I find I become far more frustrated when I try to write python than I do something like Rust just because I know when I write Python that I will have mistakes I'll ha…

I greatly prefer Rust to Python, but slightly prefer Go to Rust in many situations. Python even with MyPy just feels sloppy, whereas Go does offer decently strong typing at least. As for Go vs Rust, the answer is in the ecosystem, and I blame that on the difficulty of maintaining high quality Rust crates. I know not everyone agrees with this explanation, but I think Rust being complicated makes it hard to offer zero…

OT: sqlx tip: you can compile without a DB connection by either using the non-macro version of the queries (at the cost of losing compile time query checking) or you "bake" the query checks using sqlx-cli which produces files that you can check into your repo and allow the macros to compile offline. In that case you only need to re-connect to the DB when the SQL queries change.

Re: Data Race Patterns in Go

#97

Post by fulafel » Sat, Jun 11, 2022, 7:34 AM UTC

ohazi wrote: post #79

beltsazar wrote: post #72

Earlier quoted context omitted.

> "Oh, data race freedom isn't such a big deal, if you really need it, a garbage collected language like Java will give you that guarantee." Java programs aren't guaranteed to be free of data race. Java spec guarantees that if that happens, there will be no undefined behavior (like in C++).

Hm, maybe I was misremembering. Managed languages like Java do give you memory safety , but I guess data race freedom isn't actually guaranteed. Now that I think about it, this must be the case, right? You have to get `synchronized` right in Java or else you won't get what you expect.

There's a specified Java memory model (JMM) and in the case the programmer shooting themself in the foot with concurrency by eg buggy synchronization, it provides just enough guarantees to protect the integrity of the state of the runtime itself.

(I didn't find this integirty of runtime specified in the JMM spec, hopefully it's in the other specs).

In the JMM terminology, the "you're in the clear" term is "well-formed execution". If you break the rules, you're not in "well-formed execution" land any more, and things may fly out of your orifices, but a specific type of C/C++ style dragon won't maybe fly out of your nose.

So there's a weak kind of memory safety, your app data in may still be garbled, possibly in an attacker-controlled way, but the attacker probably won't get remote code execution.

Re: Data Race Patterns in Go

#98

Post by gigatexal » Sat, Jun 11, 2022, 7:37 AM UTC

In the closure example does declaring a new variable and setting its value to the iterative or the thing being passed in, does that mitigate the pass by reference issue?

Re: Data Race Patterns in Go

#99

Post by morelisp » Sat, Jun 11, 2022, 7:39 AM UTC

bumper_crop wrote: post #22

morelisp wrote: post #13

Earlier quoted context omitted.

> atomic.Value isn't really atomic, since the concrete type can't ever change after being set. How does this mean it's non-atomic ? As far as I know you can still never Load() a partial Store(). (Also, even if it was possible, this would never be a good idea...)

That's why I opened with "Look at the implementation". Go is unable to store the type and the pointer at the same time, so it warps what "atomic" means. Pretty much every other language has atomic mean "one of these will win, one will lose". Go says "one will win, one will panic and destroy the goroutine. In fact, it's even worse than that. If the Store() caller goes to sleep between setting the type and storing the…

Abend is a fairly normal and in many ways best way to "lose" in a race. It's fine, it's atomic.

Re: Data Race Patterns in Go

#100

Post by mmarq » Sat, Jun 11, 2022, 7:41 AM UTC

saghm wrote: post #55

hexxagone wrote: post #53

Earlier quoted context omitted.

Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. "go makes concurrency a first-class concept" I think it usually refers to goroutines being built in the language. "Go is abnormally dangerous when it comes to concurrency IMO". Personnally, it has not been my experience with Go concurren…

> Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. This doesn't stop there being "task handles" then, though? I think the point GP was making is that something that in most languages would be simple methods on a handle like "wait for this task to finish" or "stop this task" instead nee…

My (rather horrid) pattern to address this problem is to wrap the goroutine in a function that returns a channel receiver. When the goroutine ends it sends something to the channel and whatever called it can await the result or completion using the receiver.