Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are Go maps sensitive to data races?

dave.cheney.net

Post reply on HN
Print view View original

31–40 of 81 posts

Re: Are Go maps sensitive to data races?

#31

Post by lazyjones »

alkonaut wrote: post #21
grabcocque wrote: post #17

I'm increasingly convinced that Go is the new PHP, designed and used by the kind of people who know just enough to be incredibly dangerous to themselves and everyone else they touch.

I don't understand why someone would write a blog post about a collection type not being thread safe? Why not just look at the documentation and simply confirm that it isn't thread safe? Edit: Ok one reason could be that the documentation sucks and there are no mentions of the concurrency guarantees of the standard library types...

Possibly because the Go "authorities" are now trying to attract the kind of programmers who need "trigger warnings" and "safe spaces", for some reason. Just look at the new community CoC. I apologize if this constitutes a micro- or nano-aggression.

More seriously, it's probably just a frequent mistake and they try to educate people proactively to help code quality in the long run.

Edit: the non-atomic map updates are mentioned in the FAQ, but you're right, the documentation could be improved in that regard (possibly triggering many criticisms of the language from people who've never used it, just browsed the docs for reasons not to use it).

Re: Are Go maps sensitive to data races?

#32

Post by _ak »

kasey_junk wrote: post #24

A concurrent map is the top of my wants list for go by a long stretch. The next thing on that list is a high performance concurrent queue. That these things aren't available is a direct result of the decision to not support user defined generics in go.

> The next thing on that list is a high performance concurrent queue.

That's called a channel in Go.

Re: Are Go maps sensitive to data races?

#33

Post by goykasi »

grabcocque wrote: post #2

"no, there is nothing wrong with Go’s map implementation." "Getting your locking wrong will corrupt the internal structure of the map." Ahem. If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong.

I see nothing wrong with how Go's map implementation was described. Go has never claimed to be completely safe for concurrent operations. Deadlocks, inconsistent state, etc are more than easy to accomplish. But, it still stands up to a "good for concurrency" claim by offering nice primitives for writing concurrent code, a race detector and explicitly stating in the stdlib documentation which libraries are thread-safe -- libraries are to be considered not safe unless explicitly said to be.

Re: Are Go maps sensitive to data races?

#34

Post by chrisseaton »

alkonaut wrote: post #13

Any data structure that isn't explicitly designed for concurrent access will not work when used concurrently. Since a concurrent map (aka. Dictionary aka. Hashtable) is a lot more complex and slow than a non concurrent one, it's very very unlikely that Go would ship with its standard map type being a concurrent one. It would be a huge waste. More likely there is a separate concurrent map type. It's exactly the same i…

> Since a concurrent map (aka. Dictionary aka. Hashtable) is a lot more complex and slow than a non concurrent one, it's very very unlikely that Go would ship with its standard map type being a concurrent one. It would be a huge waste.

We should develop a system where maps are unsynchronised by default, but when you access them from a second thread they are then converted transparently by the runtime to a thread-safe implementation.

Re: Are Go maps sensitive to data races?

#35

Post by lazyjones »

grabcocque wrote: post #6
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

I apologise for expecting better of language designers. After all, the solutions to these sorts of issues has only been around for a couple of decades. Far too soon.

The explanation for this design decision is reasonable IMHO: https://golang.org/doc/faq#atomic_maps

Re: Are Go maps sensitive to data races?

#36

Post by seanmcdirmid »

4ad wrote: post #4
grabcocque wrote: post #2

"no, there is nothing wrong with Go’s map implementation." "Getting your locking wrong will corrupt the internal structure of the map." Ahem. If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

32 bit data types are read and written atomically on the CLR. Pointers are read/written atomicly on most ANY platform, or you couldn't build any kind of higher level concurrency abstraction. If not, the language could never provide memory safety.

Re: Are Go maps sensitive to data races?

#37

Post by kasey_junk »

_ak wrote: post #32
kasey_junk wrote: post #24

A concurrent map is the top of my wants list for go by a long stretch. The next thing on that list is a high performance concurrent queue. That these things aren't available is a direct result of the decision to not support user defined generics in go.

> The next thing on that list is a high performance concurrent queue. That's called a channel in Go.

A channel is a simple abstraction around a giant mutex. It does not scale throughput very well and has terrible latency performance to boot.

Something like the disruptor (https://lmax-exchange.github.io/disruptor/) is more along the lines of what I was thinking of. Without generics support you have to write it over and over again for each use case.

Re: Are Go maps sensitive to data races?

#38

Post by JulianMorrison »

The other reason: things inside maps are often pointer types with mutable data. Even if access to the map is properly locked, if two threads take a pointer out of it and directly or indirectly keep a reference (for example, re-slicing a slice without copying it), then one can stomp on the data the other is reading.

The fixes for this are subtle and annoying. Either you have to lock around your data, make it somehow immutable, or force copying rather than referencing (by making it "value" data).

Go would really benefit from a port of Clojure's data structures.

Re: Are Go maps sensitive to data races?

#40

Post by lambda_cube »

4ad wrote: post #15
coldtea wrote: post #7

Earlier quoted context omitted.

Yeah, but the parent's phrase was: "If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong". If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is no…

First, that statement was not present in grandparent's post when the parent posted his message. Second, that premise is completely untrue. Go, and most other languages give you more-or-less orthogonal primitives and let you combine them. It's okay for your types not to be thread safe, if by combining them with some other primitive you can make them thread safe. Every other language does this. Some offer you a concurr…

> no languages with mutable state that I know of offer concurrent integer and concurrent structs

Java does for integer http://docs.oracle.com/javase/8/docs/api/java/util/concurren...

You can find similar classes in the java.util.concurrent.atomic package. http://docs.oracle.com/javase/8/docs/api/java/util/concurren...

I'm not sure what requirements you have for a concurrent struct, but Java has classes to atomically manipulate int and long fields of a class.

I'm not arguing your general point (in fact, I agree with it), I'm just supplying some extra information of a language that you apparently don't know.

Post reply on HN