Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go’s race detector has a mutex blind spot

doublefree.dev

Post reply on HN
Print view View original

31–40 of 60 posts

Re: Go’s race detector has a mutex blind spot

#31

Post by jimbo808 »

Xeoncross wrote: post #15

I'm so glad to be out of the dark ages of parallelism. Complaining about Go's race detector or exactly which types of logical races Rust can't prevent is such a breath of fresh air compared to all those other single-core languages we're paid to write with that had threading, async, or concurrency bolted-on as an afterthought. I can only hope Go and Rust continue to improve until the next language generation comes alo…

My guess is that next the language gen will be languages that AI generates, which are optimized to be readable to humans and writable by AI. Maybe even two layers, one layer that is optimized for human skimming, and another layer that actually compiles, which is optimized for AI to generate and for the computer to compile.

Re: Go’s race detector has a mutex blind spot

#32

Post by CodeBrad »

jimbo808 wrote: post #31
Xeoncross wrote: post #15

I'm so glad to be out of the dark ages of parallelism. Complaining about Go's race detector or exactly which types of logical races Rust can't prevent is such a breath of fresh air compared to all those other single-core languages we're paid to write with that had threading, async, or concurrency bolted-on as an afterthought. I can only hope Go and Rust continue to improve until the next language generation comes alo…

My guess is that next the language gen will be languages that AI generates, which are optimized to be readable to humans and writable by AI. Maybe even two layers, one layer that is optimized for human skimming, and another layer that actually compiles, which is optimized for AI to generate and for the computer to compile.

> which are optimized to be readable to humans and writable by AI

How might a language optimized for AI look different than a language optimized for humans?

Re: Go’s race detector has a mutex blind spot

#33

Post by empath75 »

klabb3 wrote: post #4
TheDong wrote: post #2

You're using Go's race detector wrong if you expect it to actually catch all races. It doesn't, it can't, it's a best effort thing. The right way to use the go race detector is: 1. Only turn it on in testing. It's too slow to run in prod to be worth it, so only in testing. If your testing does not cover a use-case, tough luck, you won't catch the race until it breaks prod. 2. Have a nightly job that runs unit and int…

> Rewrite your code in rust, and get something better than the go race detector every time you compile. Congrats, rustc forced you to wrap all your types in Arc >, and you no longer have data races. As a gift, you will get logical race conditions instead, that are even more difficult to detect, while being equally difficult to reproduce reliably in unit tests and patch. Don’t get me wrong, Rust has done a ton for saf…

> Congrats, rustc forced you to wrap all your types in Arc>, and you no longer have data races.

Or you can just avoid shared mutable state, or use channels, or many of the other patterns for avoiding data races in Rust. The fun thing is that you can be sure that no matter what you do, as long as it's not unsafe, it will not cause a data race.

Re: Go’s race detector has a mutex blind spot

#34

Post by toast0 »

Xeoncross wrote: post #15

I'm so glad to be out of the dark ages of parallelism. Complaining about Go's race detector or exactly which types of logical races Rust can't prevent is such a breath of fresh air compared to all those other single-core languages we're paid to write with that had threading, async, or concurrency bolted-on as an afterthought. I can only hope Go and Rust continue to improve until the next language generation comes alo…

IMHO, shared memory parallelism as the norm, means we're still in the dark ages.

Yes, shared memory is useful sometimes, but I don't think it should be the norm. But I've done parallel stuff in lots of languages, most recently Erlang and Rust... Message passing is so much nicer than having threads all mucking about in the same data if you don't need them to. You can write message passing parallel code in Rust, but it's not the norm, and you'll have to do a lot of the plumbing.

Re: Go’s race detector has a mutex blind spot

#35

Post by tialaramex »

ViewTrick1002 wrote: post #23
jerf wrote: post #20

Earlier quoted context omitted.

OK, but you're not in "Go"-specific problems any more, that's just concurrency issues. There isn't any approach to concurrency that will rigorously prevent programmers from writing code that doesn't progress sufficiently, not even going to the extremes of Erlang or Haskell. Even when there are no locks qua locks to be seen in the system at all I've written code that starved the system for resources by doing things li…

I would say it is a Go specific problem with how mutexes and defer are used together. In rust you would just throw a block around the mutex access changing the scoping and ensuring it is dropped before the slow function is called. Call it a minimally intrusive manual unlock.

In Rust you can also explicitly drop the guard.

    drop(foo); // Now foo doesn't exist, it was dropped, thus unlocking anything which was kept locked while foo exists
If you feel that the name drop isn't helpful you can write your own function which consumes the guard, it needn't actually "do" anything with it - the whole point is that we moved the guard into this function, so, if the function doesn't return it or store it somewhere it's gone. This is why Destructive Move is the correct semantic and C++ "move" was a mistake.

Re: Go’s race detector has a mutex blind spot

#36

Post by pkolaczk »

tialaramex wrote: post #30
Xeoncross wrote: post #15

I'm so glad to be out of the dark ages of parallelism. Complaining about Go's race detector or exactly which types of logical races Rust can't prevent is such a breath of fresh air compared to all those other single-core languages we're paid to write with that had threading, async, or concurrency bolted-on as an afterthought. I can only hope Go and Rust continue to improve until the next language generation comes alo…

You know how a modern language like Rust doesn't have the unstructured control flow with features like "goto"† but only a set of structured control flow features, such as pattern matching, conditionals, loops and functions? Structured Concurrency is the same idea, but for concurrency. Instead of that code to create an appropriate number of threads, parcel out work, and so on, you just express high level goals like "D…

Rust async streams or rayon come very close to what you describe as structured concurrency. Actually much closer than anything I saw in other mainstream languages eg Java or Go.

Re: Go’s race detector has a mutex blind spot

#37

Post by Lvl999Noob »

jimbo808 wrote: post #31
Xeoncross wrote: post #15

I'm so glad to be out of the dark ages of parallelism. Complaining about Go's race detector or exactly which types of logical races Rust can't prevent is such a breath of fresh air compared to all those other single-core languages we're paid to write with that had threading, async, or concurrency bolted-on as an afterthought. I can only hope Go and Rust continue to improve until the next language generation comes alo…

My guess is that next the language gen will be languages that AI generates, which are optimized to be readable to humans and writable by AI. Maybe even two layers, one layer that is optimized for human skimming, and another layer that actually compiles, which is optimized for AI to generate and for the computer to compile.

For the current category of LLM based AI, "AI optimised" means "old and popular". Even if you add a layer that has much more details but may be a lot more verbose or whatever, that layer would not be "AI optimised".

Re: Go’s race detector has a mutex blind spot

#38

Post by khuey »

tialaramex wrote: post #30
Xeoncross wrote: post #15

I'm so glad to be out of the dark ages of parallelism. Complaining about Go's race detector or exactly which types of logical races Rust can't prevent is such a breath of fresh air compared to all those other single-core languages we're paid to write with that had threading, async, or concurrency bolted-on as an afterthought. I can only hope Go and Rust continue to improve until the next language generation comes alo…

You know how a modern language like Rust doesn't have the unstructured control flow with features like "goto"† but only a set of structured control flow features, such as pattern matching, conditionals, loops and functions? Structured Concurrency is the same idea, but for concurrency. Instead of that code to create an appropriate number of threads, parcel out work, and so on, you just express high level goals like "D…

> Nothing as close to the metal as Rust has that baked in today

Rust's futures/streams are basically what you're asking for. You need a crate rather than just the bare language but I don't think that's a particularly important distinction.

Re: Go’s race detector has a mutex blind spot

#39

Post by pkolaczk »

klabb3 wrote: post #4
TheDong wrote: post #2

You're using Go's race detector wrong if you expect it to actually catch all races. It doesn't, it can't, it's a best effort thing. The right way to use the go race detector is: 1. Only turn it on in testing. It's too slow to run in prod to be worth it, so only in testing. If your testing does not cover a use-case, tough luck, you won't catch the race until it breaks prod. 2. Have a nightly job that runs unit and int…

> Rewrite your code in rust, and get something better than the go race detector every time you compile. Congrats, rustc forced you to wrap all your types in Arc >, and you no longer have data races. As a gift, you will get logical race conditions instead, that are even more difficult to detect, while being equally difficult to reproduce reliably in unit tests and patch. Don’t get me wrong, Rust has done a ton for saf…

I wrote plenty of concurrent Rust code and the number of times I had to use Arc is extremely low (maybe a few times per thousands lines).

As for your statement that concurrency is generally hard - yes it is. But it is even harder with data races.

Re: Go’s race detector has a mutex blind spot

#40

Post by empath75 »

pkolaczk wrote: post #36
tialaramex wrote: post #30

Earlier quoted context omitted.

You know how a modern language like Rust doesn't have the unstructured control flow with features like "goto"† but only a set of structured control flow features, such as pattern matching, conditionals, loops and functions? Structured Concurrency is the same idea, but for concurrency. Instead of that code to create an appropriate number of threads, parcel out work, and so on, you just express high level goals like "D…

Rust async streams or rayon come very close to what you describe as structured concurrency. Actually much closer than anything I saw in other mainstream languages eg Java or Go.

Rayon is about as pure an example of it as you can imagine. In a lot of cases you just need to replace iter() with par_iter() and it just works.
Post reply on HN