Live data from Hacker News

A million ways to die from a data race in Go

gaultier.github.io

121–130 of 146 posts

Re: A million ways to die from a data race in Go

#121

Earlier quoted context omitted.

> Go is remarkably easy to be productive in which is what the label on the tin can claims. To feel productive in.

It feels productive because you're not waiting ages for it to compile again after every change.

I would say all the boiler plate and extra typing, while the language not preventing you from shooting yourself in the foot.

Re: A million ways to die from a data race in Go

#122
post #115

> I have been writing production applications in Go for a few years now. sorry, what? https://gaultier.github.io/blog/a_million_ways_to_data_race_... this code is obviously wrong, fractally wrong why would you create a new PricingService for every request? what makes you think a mutex in each of those (obviously unique) PricingService values would somehow protect the (inexplicably shared) PricingInfo value?? > the fi…

I had a similar reaction to that, glad it's not just me.

Meanwhile with the 4th item, this whole example is gross, repeatedly polling a buffer every 100ms is a massive red flag. And as for the data race in that item, the idiomatic fix is to just use io.Pipe, which solves the entire problem far more cleanly than inventing a SyncWriter.

The author's last comment regarding "It would also be nice if more types have a 'sync' version, e.g. SyncWriter, SyncReader, etc" probably indicates there's some fundamental confusion here about idiomatic Go.

Re: A million ways to die from a data race in Go

#123
post #107
post #97

The first Go proverb Rob Pike listed in his talk "Go Proverbs" was, "Don't communicate by sharing memory, share memory by communicating." Go was designed from the beginning to use Tony Hoare's idea of communicating sequential processes for designing concurrent programs. However, like any professional tool, Go allows you to do the dangerous thing when you absolutely need to, but it's disappointing when people insist o…

> people insist on using the dangerous way and then blame it on the language Can you blame them when the dangerous way uses 0 syntax while the safe way uses non-0 syntax? I think it's fine to criticize unsafe defaults, though of course it would not be fair to treat it like it's the only option

They're not using the dangerous way because of syntax, they're using it because they think they're "optimizing" their code. They should write correct code first, measure, and then optimize if necessary.

Re: A million ways to die from a data race in Go

#124

Earlier quoted context omitted.

Rust's `const` is an actual constant, like 4 + 1 is a constant, it's 5, it's never anything else, we don't need to store it anywhere - it's just 5. In C++ `const` is a type qualifier and that keyword stands for constant but really means immutable not constant. This results in things like you can "cast away" C++ const and modify that variable anyway, whereas obviously we can't try to modify a constant because that's n…

Right, I forgot that 'const' in Rust is 'constexpr'/'consteval' in C++, while absence of 'mut' is probably closer to C++ 'const', my apologies. C++ 'constexpr' and Rust 'const' is more about compile-time execution than marking something immutable. In Rust, it is probably also possible to do a cast like &T to *mut T. Though that might require unsafe and might cause UB if not used properly. I recall some people hoping…

AFAICT Although C++ now has const, constexpr. consteval and constinit, none of those mean an actual constant. In particular constexpr is largely just boilerplate left over from an earlier idea about true compile time constants, and so it means almost nothing today.

Yes, the C++ compile time execution could certainly be considered more powerful than Rust's and Zig's even more powerful than that. It is expected that Rust will some day ship compile time constant trait evaluations, which will mean you don't have to write awkward code that avoids e.g. iterators -- so with that change it's probably in the same ballpark as C++ 17 (maybe a little more powerful). However C++ 20 does compile-time dynamic allocation†, and I don't think that's on the horizon for Rust.

† In C++ 20 you must free these allocations inside the same compile-time expression, but that's still a lot of power compared to not being allowed to allocate. It is definitely possible that a future C++ language will find a way to sort of "grandfather in" these allocations so that somehow they can survive to runtime rather than needing to free them.

Rust does give you the option to break out the big guns by writing "procedural" aka "proc" macros which are essentially Rust that is run inside your compiler. Obviously these are arbitrarily powerful, but far too dangerous - there's a (serious) proc macro to run Python from inside your Rust program and (joke, in that you shouldn't use it even though it would work) proc macro which will try out different syntax until it finds which of several options results in a valid program...

Re: A million ways to die from a data race in Go

#125
post #99
post #97

The first Go proverb Rob Pike listed in his talk "Go Proverbs" was, "Don't communicate by sharing memory, share memory by communicating." Go was designed from the beginning to use Tony Hoare's idea of communicating sequential processes for designing concurrent programs. However, like any professional tool, Go allows you to do the dangerous thing when you absolutely need to, but it's disappointing when people insist o…

Meaning similar to Erlang style message passing?

Not quite. Erlang uses the Actor model which delivers messages asynchronously to named processes. In Go, messages are passed between goroutines via channels, which provide a synchronization mechanism (when un-buffered). The ability to synchronize allow one to setup a "rhythm" to computation that the Actor model is explicitly not designed to do. Also, note that a process must know its consumer in the Actor model, but goroutines do not need to know their consumer in the CSP model. Channels can even be passed around to other goroutines!

Each have their own pros and cons. You can see some of the legends who invented different methods of concurrency here: https://www.youtube.com/watch?v=37wFVVVZlVU

There's also a nice talk Rob Pike gave that illustrated some very useful concurrency patterns that can be built using the CSP model: https://www.youtube.com/watch?v=f6kdp27TYZs

Re: A million ways to die from a data race in Go

#126

Does Elixir have any footguns like this? As it is immutable I don't think any of these are possible.

Sorry, this is going to be a slightly longer reply since this is a really interesting question to ask!

Elixir (and anything that runs on the BEAM) takes an entirely different perspective on concurrency than almost everything else out there. It still has concurrency gotchas, but at worst they result in logic bugs, not violations of the memory model.

Stuff like:

  - forgetting to update a state return value in a genserver
  - reusing an old conn value and/or not using the latest conn value in Plug/Phoenix
  - in ETS, making the assumption nothing else writes to your key after doing a read (I wrote a library to do this safely with compare-and-swap: https://github.com/ckampfe/cas)
  - same as the ETS example, but in a process: but doing a write after doing a read and assuming nothing else has altered the process state in the interim
  - leaking processes (and things like sockets/ports), either by not supervising them, monitoring them, or forgetting to shut them down, etc. This can lead to things like OOMs, etc.
  - deadlocking processes by getting them into a state where they each expect a reply from the other process (OTP timeouts fix this, try to always use OTP)
  - logical race conditions in a genserver init callback, where the process performs some action in the init that cannot complete until the init has returned, but the init has not returned yet, so you end up with a race or an invalid state
  - your classic resource exhaustion issues, where you have a ton of processes attempting to use some resource and that resource not being designed to be accessed by 1,000,000 things concurrently
  - OOMing the VM by overfilling the mailbox of a process that can't process messages fast enough
Elixir doesn't really have locks in the same sense as a C-like language, so you don't really have lock lifetime issues, and Elixir datastructures cannot be modified at all (you can only return new, updated instances of them) so you can't modify them concurrently. Elixir has closures that can capture values from their environment, but since all values in Elixir are immutable, the closure can't modify values that it closes over.

Elixir really is designed for this stuff down to its core, and (in my opinion) it's evident how much better Elixir's design is for this problem space than Go's is if you spend an hour with each. The tradeoff Elixir makes is that Elixir isn't really what I'd call a general purpose language. It's not amazing for CLIs, not amazing for number crunching code, not amazing for throughput-bound problems. But it is a tremendous fit for the stuff most of us are doing: web services, job pipelines, etc. Basically anything where the primary interface is a network boundary.

Edited for formatting.

Re: A million ways to die from a data race in Go

#127
post #125
post #99

Earlier quoted context omitted.

Meaning similar to Erlang style message passing?

Not quite. Erlang uses the Actor model which delivers messages asynchronously to named processes. In Go, messages are passed between goroutines via channels, which provide a synchronization mechanism (when un-buffered). The ability to synchronize allow one to setup a "rhythm" to computation that the Actor model is explicitly not designed to do. Also, note that a process must know its consumer in the Actor model, but…

It's true that message sends with Erlang processes do not perform rendezvous synchronization (i.e., sends are nonblocking), but they can be used in a similar way by having process A send a message to process B and then blocking on a reply from process B. This is not the same as unbuffered channel blocking in Go or Clojure, but it's somewhat similar.

For example, in Erlang, `receive` _is_ a blocking operation that you have to attach a timeout to if you want to unblock it.

You're correct about identity/names: the "queue" part of processes (the part that is most analogous to a channel) is their mailbox, which cannot be interacted with except via message sends to a known pid. However, you can again mimic some of the channel-like functionality by sending around pids, as they are first class values, and can be sent, stored, etc.

I agree with all of your points, just adding a little additional color.

Re: A million ways to die from a data race in Go

#128
post #82
post #81

Earlier quoted context omitted.

Under most circumstances function local variables aren't passed to other threads, or passed at all.

And? That's a small, optional optimization done by e.g. Swift. Also, I don't know how it's relevant to Go which uses a tracing GC.

It's not "small" if it accounts for most of the allocations :)

Re: A million ways to die from a data race in Go

#129
post #115

> I have been writing production applications in Go for a few years now. sorry, what? https://gaultier.github.io/blog/a_million_ways_to_data_race_... this code is obviously wrong, fractally wrong why would you create a new PricingService for every request? what makes you think a mutex in each of those (obviously unique) PricingService values would somehow protect the (inexplicably shared) PricingInfo value?? > the fi…

Yeah this whole section of the article threw me all the way off. What even is this code? There’s so many things wrong with it, it blows my mind.

About the only code example I saw in here and thought “yeah it sucks when that happens” is the accidental closure example. Accidentally shadowing something you’re trying to assign to in a branch because you need to handle an error or accidentally reassigning something can be subtle. But it’s pretty 101 go.

The rest is… questionable at best.

Re: A million ways to die from a data race in Go

#130

TL;DR. Author with “years of experience of shipping to prod” mutates globals without a mutex and is surprised enough to write a blog.

There’s an example of a mutex too…

An example where they’re creating a new mutex every time they call a function and then surprised when multiple goroutines that called that function and got entirely different mutexes somehow couldn’t coordinate the locks together.

That isn’t a core misunderstanding of Go, that’s a core misunderstanding of programming.

Post reply on HN