Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are Go maps sensitive to data races?

dave.cheney.net

Post reply on HN
Print view View original

51–60 of 81 posts

Re: Are Go maps sensitive to data races?

#51

Post by 4ad »

viraptor wrote: post #25
4ad wrote: post #4

Earlier quoted context omitted.

> Ahem Maps are not concurrency-safe, you know what else isn't? Integers, floats, pointers, structs. And not only in Go, but in pretty much almost all languages with mutable state, like C and C++. The map implementation in Go is consistent with the rest of the language, plus it allows for great performance under some rather common scenarios. But I guess it's fashionable to snark on message boards than rather understa…

C got atomic values in C11: http://en.cppreference.com/w/c/language/atomic Not that I disagree with your comment in general, but atomic values do exist in many languages.

Thanks for the reference, I knew about C11 atomic types, and, I should have use a better phrasing. Note that C11 atomics are still not applicable to multi-word aggregate types, like structs. You'd use atomics in C11 (in the type system), for what you'd use sync/atomic (like other poster mentioned) in Go (outside the type system).

Re: Are Go maps sensitive to data races?

#53

Post by alkonaut »

grabcocque wrote: post #47
alkonaut wrote: post #16

Earlier quoted context omitted.

A lot of comments that just seem to ignore the fact that noone would actually want standard collections to be thread safe! Concurrent collections are slow . That's why class libraries use different types for a therad safe collection and a regular non-thread safe collection. If you are using concurrency you have to use concurrent collections. You could argue that if most Go code is concurrent then the regular collecti…

They're really not though. Java has ConcurrentHashMap and isn't slow. Clojure has its persistent HashMap and it isn't slow. Scala has, TWO (?) concurrent hash maps in its core and neither is slow. You know, two thirds of the way through the second decade of the 21st century, it's okay to spend a few cycles not corrupting your data structures.

There are both memory and performance considerations. For a server/db web application this may not be a problem, but for e.g. some logic in a game engine that runs every frame on a mobile device, then using 16 extra locks per map for example isn't a good idea. Nor is having 5% performance overhead.

I can agree with an argument that thread-safe should be default and specialized/fast collections should be optional though.

Re: Are Go maps sensitive to data races?

#54

Post by kasey_junk »

chrisseaton wrote: post #45
kasey_junk wrote: post #43

Earlier quoted context omitted.

https://golang.org/src/runtime/chan.go Lines 28, 152, 401

I stand corrected. I guess it must have some more complex functionality that I thought, needing a lock.

That is the charitable way to interpret it.

Re: Are Go maps sensitive to data races?

#55

Post by noselasd »

grabcocque wrote: post #5
iand wrote: post #3

Earlier quoted context omitted.

The two sentences are consistent once you read that Go maps are not designed to be safe for concurrent access.

Java's (pre-concurrent) HashMaps aren't designed for concurrent access. Do you know what they do? They throw a ConcurrentModificationException. Do you know what they don't do? Silently corrupt themselves.

A Java Hastable is threadsafe, and is documented to be so

A Java HashMap is not threadsafe and is documented to be so, and can silently corrupt data, can go into an infinite loop etc. if you mess with it from several threads.

Both leaves you with a lot of potential for races in application logic though.

Re: Are Go maps sensitive to data races?

#56

Post by JulianMorrison »

vinceyuan wrote: post #52

> Go maps are not goroutine safe Which types in Go are goroutine safe? Do we need to use sync.Mutex for all global variables which are read and written in goroutines? My understanding is only read-only variables are goroutine safe.

If you copy value data inside a mutex, access to the copy you made is goroutine safe.

Re: Are Go maps sensitive to data races?

#57

Post by JulianMorrison »

SixSigma wrote: post #49

Share memory by communicating; don't communicate by sharing memory. https://golang.org/doc/codewalk/sharemem/

Easy to say, hard to do.

You need to make sure that once you've done with the item and passed it on, you don't have any overt, tacit, or deeply-buried pointers into the item you just let go of.

Re: Are Go maps sensitive to data races?

#58

Post by lmm »

vbezhenar wrote: post #8
grabcocque wrote: post #5

Earlier quoted context omitted.

Java's (pre-concurrent) HashMaps aren't designed for concurrent access. Do you know what they do? They throw a ConcurrentModificationException. Do you know what they don't do? Silently corrupt themselves.

From HashMap JavaDoc: Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast beha…

Indeed it doesn't promise it. But in practice it's very good at it - the reason they have to put that note in there is because if you made a program whose correctness depended on it, it would be a long time before you saw the bug.

Re: Are Go maps sensitive to data races?

#59

Post by lmm »

chrisseaton wrote: post #34
alkonaut wrote: post #13

Any data structure that isn't explicitly designed for concurrent access will not work when used concurrently. Since a concurrent map (aka. Dictionary aka. Hashtable) is a lot more complex and slow than a non concurrent one, it's very very unlikely that Go would ship with its standard map type being a concurrent one. It would be a huge waste. More likely there is a separate concurrent map type. It's exactly the same i…

> Since a concurrent map (aka. Dictionary aka. Hashtable) is a lot more complex and slow than a non concurrent one, it's very very unlikely that Go would ship with its standard map type being a concurrent one. It would be a huge waste. We should develop a system where maps are unsynchronised by default, but when you access them from a second thread they are then converted transparently by the runtime to a thread-safe…

You can't do that retroactively. And it would make performance very hard to reason about or measure.

Re: Are Go maps sensitive to data races?

#60

Post by lmm »

AYBABTME wrote: post #22
grabcocque wrote: post #17

I'm increasingly convinced that Go is the new PHP, designed and used by the kind of people who know just enough to be incredibly dangerous to themselves and everyone else they touch.

You say that, while also suggesting that Java's best effort ConcurrentModificationException (something that will be thrown if you're lucky ) is better. Suggests to me that you know just enough to be dangerous.

Java's ConcurrentModificationException shows up very quickly if you test at all (probably 99% of the time). It is very good at helping beginners not make these mistakes.
Post reply on HN