Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are Go maps sensitive to data races?

dave.cheney.net

Post reply on HN
Print view View original

11–20 of 81 posts

Re: Are Go maps sensitive to data races?

#11

Post by jrockway »

grabcocque wrote: post #6
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

I apologise for expecting better of language designers. After all, the solutions to these sorts of issues has only been around for a couple of decades. Far too soon.

All the language designer can do is throw in a lock that you may or may not need if you're already locking properly. ConcurrentModificationException is nice, but comes at a performance cost.

Re: Are Go maps sensitive to data races?

#12

Post by alkonaut »

grabcocque wrote: post #6
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

I apologise for expecting better of language designers. After all, the solutions to these sorts of issues has only been around for a couple of decades. Far too soon.

This is a library thing, not a language thing. And it's a performance tradeoff: they could have shipped with a concurrent map only, that is 1/10 as fast as the current one, but never causes a data race. However everyone that isn't doing anything concurrently would be paying for it. Not very clever. I trust the class library designers thought this through. They reached the same conclusion as pretty much every other library designer in every other language. For the same reasons.

Re: Are Go maps sensitive to data races?

#13

Post by alkonaut »

Any data structure that isn't explicitly designed for concurrent access will not work when used concurrently.

Since a concurrent map (aka. Dictionary aka. Hashtable) is a lot more complex and slow than a non concurrent one, it's very very unlikely that Go would ship with its standard map type being a concurrent one. It would be a huge waste.

More likely there is a separate concurrent map type. It's exactly the same in other languages with mutable collections (Java, .NET, C++, ...).

Re: Are Go maps sensitive to data races?

#14

Post by arethuza »

coldtea wrote: post #7
iand wrote: post #3

Earlier quoted context omitted.

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

Yeah, but the parent's phrase was: "If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong". If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is no…

I think there is a big difference between "being good for concurrency" and preventing all concurrent programming problems - concurrency is hard and while go does seem to provide a lot of nice structures to make it more straightforward it can't, and doesn't appear to claim to, hide all the complexities of concurrency from developers.

Re: Are Go maps sensitive to data races?

#15

Post by 4ad »

coldtea wrote: post #7
iand wrote: post #3

Earlier quoted context omitted.

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

Yeah, but the parent's phrase was: "If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong". If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is no…

First, that statement was not present in grandparent's post when the parent posted his message.

Second, that premise is completely untrue. Go, and most other languages give you more-or-less orthogonal primitives and let you combine them. It's okay for your types not to be thread safe, if by combining them with some other primitive you can make them thread safe.

Every other language does this. Some offer you a concurrent map along with a non-concurrent map. For many reasons I won't get into, Go only has one kind of map and lets the user do the rest (which is an extremely easy idiom in Go). Note that even though many languages conveniently offer concurrent maps, no languages with mutable state that I know of offer concurrent integer and concurrent structs. The user is still responsible for ensuring the safety of those.

This is perfectly fine because the grandparent's premise is wrong. In concurrent language, by far the most common case is by data to be owned by a goroutine/thread/whatever. It's very easy to reason about code this way, and it's the way you are encouraged to write code.

Only when you need to share data you need to worry about concurrency-safety, and in that scenario you have available all the tools to ensure it.

Re: Are Go maps sensitive to data races?

#16

Post by alkonaut »

coldtea wrote: post #7
iand wrote: post #3

Earlier quoted context omitted.

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

Yeah, but the parent's phrase was: "If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong". If we accept his premise, then the fact that "they wasn't designed for that" is no excuse. It's like selling a kids toy that has tiny choke-inducing parts. In that case, just saying: "It wasn't designed to be eaten by kids" is no…

A lot of comments that just seem to ignore the fact that noone would actually want standard collections to be thread safe! Concurrent collections are slow. That's why class libraries use different types for a therad safe collection and a regular non-thread safe collection. If you are using concurrency you have to use concurrent collections.

You could argue that if most Go code is concurrent then the regular collections should be the thread safe ones, and for code that is known to be single threaded there would be simpler/faster non-concurrent collections. That is a completely valid argument to make, but I think that design would be too confusing for new Go developers, since in most other languages the standard collections are the non-concurrent ones, and the concurrent ones are special, and it's an explicit design goal of Go to be easy to adopt coming from another language.

Re: Are Go maps sensitive to data races?

#18

Post by netheril96 »

grabcocque wrote: post #6
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

I apologise for expecting better of language designers. After all, the solutions to these sorts of issues has only been around for a couple of decades. Far too soon.

You are not expecting better. You are expecting worse. Plenty of use cases involve maps without concurrent modification, e.g., when used as a local variable. Your "better" version of maps will slow down all of them.

Re: Are Go maps sensitive to data races?

#19

Post by titzer »

4ad wrote: post #4
grabcocque wrote: post #2

"no, there is nothing wrong with Go’s map implementation." "Getting your locking wrong will corrupt the internal structure of the map." Ahem. If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

Actually Java guarantees atomic accesses for integer-sized fields and references at the language level. They aren't race-free, but they are atomic. And really, atomicity is the main property that the OP had in mind when posting. Since you mentioned C++, it also does guarantee certain-sized certain-alignment loads and stores are atomic.

In Java there is also the plethora of thread safe data structures available in java.util.concurrent. They are written by experts and are pretty darn fast (TM). It might have been reasonable to use one of these as the basis for Go map implementations, but the language designers chose to assume unsynchronized access to shared maps is a design bug in applications.

Re: Are Go maps sensitive to data races?

#20

Post by titzer »

alkonaut wrote: post #12
grabcocque wrote: post #6

Earlier quoted context omitted.

I apologise for expecting better of language designers. After all, the solutions to these sorts of issues has only been around for a couple of decades. Far too soon.

This is a library thing, not a language thing. And it's a performance tradeoff: they could have shipped with a concurrent map only, that is 1/10 as fast as the current one, but never causes a data race. However everyone that isn't doing anything concurrently would be paying for it. Not very clever. I trust the class library designers thought this through. They reached the same conclusion as pretty much every other li…

Data races are not the same thing as non-atomic accesses.
Post reply on HN