Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Race Patterns in Go

eng.uber.com

Post reply on HN
Print view View original

81–90 of 205 posts

Re: Data Race Patterns in Go

#81

Post by erik_seaberg »

Tomis02 wrote: post #70
fmakunbound wrote: post #54

> contains approximately 2,100 unique Go services (and growing) How is that possible and what do they do???

Sadly, people get rewarded for what they built (even if makes everyone's lives harder in the long run) than for exercising restraint and saying "no".

There’s a perception that big tech pays longtime employees less than they’re willing to offer new candidates, making promotion or job hopping the best ways to earn the current market rate. And if you copy Google’s promo process, you get promo-driven development, because there aren’t enough projects that actually need that level of complexity.

Re: Data Race Patterns in Go

#82

Post by silisili »

saghm wrote: post #55
hexxagone wrote: post #53

Earlier quoted context omitted.

Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. "go makes concurrency a first-class concept" I think it usually refers to goroutines being built in the language. "Go is abnormally dangerous when it comes to concurrency IMO". Personnally, it has not been my experience with Go concurren…

> Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. This doesn't stop there being "task handles" then, though? I think the point GP was making is that something that in most languages would be simple methods on a handle like "wait for this task to finish" or "stop this task" instead nee…

I've always thought it would be nice if the go command returned an ID. Doing so would also be completely backwards compatible, of course. Then add a library or few builtins to do things on that ID, at minimum maybe kill it, perhaps get status of it, etc. Maybe not full blown actor model, but having nothing feels powerless.

Re: Data Race Patterns in Go

#83

Post by nindalf »

bumper_crop wrote: post #3

Sorry to say, but these hit close to home for me. A lot of the synchronization paradigms in Go are easy to misuse, but lead the author into thinking it's okay. the WaitGroup one is particularly poignant for me, since the race detector doesn't catch it. I'll add one other data race goof: atomic.Value. Look at the implementation. Unlike pretty much every other language I've seen, atomic.Value isn't really atomic, since…

On atomics in Go, the beta for Go 1.19 was released an hour ago (https://groups.google.com/g/golang-announce/c/SNruPJUSFz0?pl...).

> The sync/atomic package defines new atomic types Bool, Int32, Int64, Uint32, Uint64, Uintptr, and Pointer. These types hide the underlying values so that all accesses are forced to use the atomic APIs. Pointer also avoids the need to convert to unsafe.Pointer at call sites. Int64 and Uint64 are automatically aligned to 64-bit boundaries in structs and allocated data, even on 32-bit systems.

Go 1.19 is expected to release in August.

Re: Data Race Patterns in Go

#85

Post by jasonwatkinspdx »

jchw wrote: post #9
metadat wrote: post #8

> 2. Slices are confusing types that create subtle and hard-to-diagnose data races The "Slices" example is just nasty! Like, this is just damning for Go's promise of "_relatively_ easy and carefree concurrency" . Think about it for a second or two, >> The reference to the slice was resized in the middle of an append operation from another async routine. What exactly happens in these cases? How can I trust myself, as…

Slices may just be one of the best and worst parts of Go. They're cumbersome, their behavior sometimes feels 'inexplicable,' and even as an experienced developer you are likely to eventually fallen into one of the traps where your 'obvious' code isn't so obvious. That said... when programming in programming languages without a slice type, I always want to have one. And though it's confusing at times, the design does…

> it's hard to think of how you would improve on the actual underlying design.

I think ranges are part of D's design they got right, and I think a similar abstraction would be in line with golang's general design ethos, GC design, etc, other than perhaps some folks might pattern match it as "this is like STL therefor bad burn it with fire etc" without actually thinking about it in detail.

Re: Data Race Patterns in Go

#86

Post by laerus »

DominoTree wrote: post #59

Seems like Rob Pike and co may have failed "The key point here is our programmers... They’re not capable of understanding a brilliant language... So, the language that we give them has to be easy for them to understand"

Yep, if Google wasn't behind Go the language would have already been history like so many other half baked technologies. It won't be long untill people will talk about Golang like they do about JavaScript.

Re: Data Race Patterns in Go

#88

Post by hnlmorg »

hexxagone wrote: post #53
Groxx wrote: post #27

Earlier quoted context omitted.

The lack of generics has forced all Go concurrency to be intrusive (i.e. implemented by the person using literally any concurrency ), and yeah. It's horrifyingly error-prone in my experience. It means everyone needs to be an expert, and lol, everyone is not an expert. Generics might save us from the simple, mechanical flaws. Expect to see `Locker ` and `Atomic ` types cropping up. And unbounded buffered thread-safe q…

Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. "go makes concurrency a first-class concept" I think it usually refers to goroutines being built in the language. "Go is abnormally dangerous when it comes to concurrency IMO". Personnally, it has not been my experience with Go concurren…

Go has OS threads and “green threads” (named “go processes”). You create green threads via the go keyword and the Go runtime assigns that to an OS thread. You can have many go processes to a single OS thread and typically have a maximum of 1 OS thread per CPU core (though that is configurable).

The GP is correct that you cannot manage go processes from outside of that green thread. With (for example) POSIX threads, which still leaves a lot to be desired, you can at least manage the thread from other threads.

Go definitely has some rough edges around threading. The idea is you’re supposed to use channels for everything but in my experience channels have so many edge cases for subtle ways to completely lock up your application that it’s often easier to fallback to the classic mutex-style idioms.

I do really like the go keyword, it’s handy. But I have a background in POSIX threads so probably find concurrency in Go easier than most yet even I have to concede that Go under-delivered on its concurrency promises.

Re: Data Race Patterns in Go

#89

Post by shp0ngle »

jeremy_wiebe wrote: post #58
hintymad wrote: post #42

> and contains approximately 2,100 unique Go services (and growing). A side topic: this is really not something to be proud of. There used to be more people than quantity of work in Uber and engineers fought for credits by building bogus decomposed services, and the sheer number of services seems indicate it's still so.

That number really store out to me too. I’d be very curious how they decide what becomes a separate service. I’d also be curious if the 50m lines of code included generated code.

I did not work in Uber, but a similar company, and... it's a very political thing, usually.

Taking an existing service and making it into 2 new microservices is a "thing you did". Suddenly, you have "impact" and can claim the new service as "yours". Everyone wants to be a king of their little kingdom.

Re: Data Race Patterns in Go

#90

Post by eurasiantiger »

”Our Go monorepo consists of about 50 million lines of code (and growing) and contains approximately 2,100 unique Go services (and growing).”

What the hell is that company doing?

Try to imagine an ERD or DFD of their day-to-day operations. 2,100 unique services…

Post reply on HN