Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are Go maps sensitive to data races?

dave.cheney.net

Post reply on HN
Print view View original

21–30 of 81 posts

Re: Are Go maps sensitive to data races?

#21

Post by alkonaut »

grabcocque wrote: post #17

I'm increasingly convinced that Go is the new PHP, designed and used by the kind of people who know just enough to be incredibly dangerous to themselves and everyone else they touch.

I don't understand why someone would write a blog post about a collection type not being thread safe? Why not just look at the documentation and simply confirm that it isn't thread safe?

Edit: Ok one reason could be that the documentation sucks and there are no mentions of the concurrency guarantees of the standard library types...

Re: Are Go maps sensitive to data races?

#22

Post by AYBABTME »

grabcocque wrote: post #17

I'm increasingly convinced that Go is the new PHP, designed and used by the kind of people who know just enough to be incredibly dangerous to themselves and everyone else they touch.

You say that, while also suggesting that Java's best effort ConcurrentModificationException (something that will be thrown if you're lucky) is better. Suggests to me that you know just enough to be dangerous.

Re: Are Go maps sensitive to data races?

#23

Post by shellac »

alkonaut wrote: post #10
grabcocque wrote: post #5

Earlier quoted context omitted.

Java's (pre-concurrent) HashMaps aren't designed for concurrent access. Do you know what they do? They throw a ConcurrentModificationException. Do you know what they don't do? Silently corrupt themselves.

That check probably comes at a performance cost that the Go standard library designers found unacceptable. You don't even need to be modifying it concurrently to read inconsistent state, it's enough to write on one thread and read on another, while the map is updating internally. So in order to prevent inconsistent reads you would have to check for concurrent access on all reads. The .NET BCL Dictionary is just like…

The check is actually very cheap -- you can check the source yourself -- but as you might expect it won't catch all unsafe use.

Re: Are Go maps sensitive to data races?

#24

Post by kasey_junk »

A concurrent map is the top of my wants list for go by a long stretch. The next thing on that list is a high performance concurrent queue. That these things aren't available is a direct result of the decision to not support user defined generics in go.

Re: Are Go maps sensitive to data races?

#25

Post by viraptor »

4ad wrote: post #4
grabcocque wrote: post #2

"no, there is nothing wrong with Go’s map implementation." "Getting your locking wrong will corrupt the internal structure of the map." Ahem. If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

C got atomic values in C11: http://en.cppreference.com/w/c/language/atomic

Not that I disagree with your comment in general, but atomic values do exist in many languages.

Re: Are Go maps sensitive to data races?

#26

Post by JulianMorrison »

grabcocque wrote: post #2

"no, there is nothing wrong with Go’s map implementation." "Getting your locking wrong will corrupt the internal structure of the map." Ahem. If you have a non-concurrency-safe map implementation in a language that's advertised as being good for concurrency, there is definitely something wrong.

Go's idiomatic way to handle access to maps is: at any one time exactly one thread owns the map. Either pass it around, or have a gatekeeper thread.

Re: Are Go maps sensitive to data races?

#27

Post by alkonaut »

titzer wrote: post #20
alkonaut wrote: post #12

Earlier quoted context omitted.

This is a library thing, not a language thing. And it's a performance tradeoff: they could have shipped with a concurrent map only, that is 1/10 as fast as the current one, but never causes a data race. However everyone that isn't doing anything concurrently would be paying for it. Not very clever. I trust the class library designers thought this through. They reached the same conclusion as pretty much every other li…

Data races are not the same thing as non-atomic accesses.

Sorry, I interpreted the statement as being about concurrency in maps, not about atomicity.

Re: Are Go maps sensitive to data races?

#28

Post by jeremyjh »

titzer wrote: post #19
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

Actually Java guarantees atomic accesses for integer-sized fields and references at the language level. They aren't race-free, but they are atomic. And really, atomicity is the main property that the OP had in mind when posting. Since you mentioned C++, it also does guarantee certain-sized certain-alignment loads and stores are atomic. In Java there is also the plethora of thread safe data structures available in jav…

Java chooses safety over performance. It is not free to check for concurrent access in a non-concurrent structure. In some languages, we let the programmer take on the choice of which costs to bear. There is not very much that I like about Go, but there is nothing wrong with having non-concurrent data structures that are documented as such and fail in non-defined ways when there is shared data access.

Re: Are Go maps sensitive to data races?

#29

Post by AYBABTME »

viraptor wrote: post #25
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

C got atomic values in C11: http://en.cppreference.com/w/c/language/atomic Not that I disagree with your comment in general, but atomic values do exist in many languages.

Also in Go https://golang.org/pkg/sync/atomic/

Re: Are Go maps sensitive to data races?

#30

Post by alkonaut »

shellac wrote: post #23
alkonaut wrote: post #10

Earlier quoted context omitted.

That check probably comes at a performance cost that the Go standard library designers found unacceptable. You don't even need to be modifying it concurrently to read inconsistent state, it's enough to write on one thread and read on another, while the map is updating internally. So in order to prevent inconsistent reads you would have to check for concurrent access on all reads. The .NET BCL Dictionary is just like…

The check is actually very cheap -- you can check the source yourself -- but as you might expect it won't catch all unsafe use.

Like you suggest you can make a cheap check that sometimes triggers an exception. Or an expensive check that always triggers on all concurrent access. As far as I can see in the java HashMap source, the concurrent modification check is actually only done for the enumerator (?) so the problem of multiple concurrent insertions, or even reading-while-inserting will never be caught anyway. Just like Go, or .NET.

A check for single thread access (which is even stricter than non-concurrent access which allows several threads as long as they aren't used concurrently) would be pretty cheap: store the thread ID on creation and then verify on reads and writes.

Failing on concurrent access otherwise on all reads and writes would basically mean that you add a lock to the write operation, and fail any reads while anyone is writing. This however is close enough to making it a full concurrent map, so it isn't worth it.

Post reply on HN