Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NilAway: Practical nil panic detection for Go

uber.com

Post reply on HN
Print view View original

31–40 of 262 posts

Re: NilAway: Practical nil panic detection for Go

#31

Post by adtac »

npalli wrote: post #18

"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)" Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?

Imagine a multidimensional matrix with various payment methods, local regulations, cloud providers, third party dependencies, web/mobile platforms, etc. Then also add more dimensions for internal things like accounting, hiring, payroll, promotions, compliance, security, monitoring, etc. Then double it for Uber Eats or whatever.

There's a lot of overlap and some invalid combinations, but you're still left with a huge number of combinations where Uber must simply work. And every time you add a new thing to this list, the total number of combinations grows polynomially.

(Also, Go is slightly more verbose than most languages. I think that's a feature and not a bug, but it's one more reason.)

Re: NilAway: Practical nil panic detection for Go

#32

Post by ianmcgowan »

vrosas wrote: post #28
npalli wrote: post #18

"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)" Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?

Uber is famous for NIH syndrome. You can tell by their open source projects they've basically built every part of their infra from scratch. So it's not just the application code but everything else that helps run it.

Either genius or madness, you be the judge!

Re: NilAway: Practical nil panic detection for Go

#33

Post by [deleted] »

npalli wrote: post #18

"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)" Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?

[deleted]

Re: NilAway: Practical nil panic detection for Go

#34

Post by masklinn »

mutatio wrote: post #14
yencabulator wrote: post #7

Earlier quoted context omitted.

Races between goroutines can corrupt memory. E.g. manipulate a map from two goroutines and you can wreck its internal state.

Can this actually manifest? Even without the -race flag I think maps are a special case which will panic with a concurrent mutation error if access isn't synchronized.

> Can this actually manifest?

Yes. Per rsc (https://research.swtch.com/gorace)

> In the current Go implementations, though, there are two ways to break through these safety mechanisms. The first and more direct way is to use package unsafe, specifically unsafe.Pointer. The second, less direct way is to use a data race in a multithreaded program.

That races undermine memory safety in go has been used in CTFs: https://github.com/netanel01/ctf-writeups/blob/master/google...

These are not idle fancies, there are lots of ways to unwittingly get data races in go: https://www.uber.com/blog/data-race-patterns-in-go.

Re: NilAway: Practical nil panic detection for Go

#35

Post by latchkey »

vrosas wrote: post #28
npalli wrote: post #18

"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)" Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?

Uber is famous for NIH syndrome. You can tell by their open source projects they've basically built every part of their infra from scratch. So it's not just the application code but everything else that helps run it.

I personally find uberFX to be a fantastic project. It isn't necessary for you to write golang with it, but it certainly does provide a great framework for organizing code so that you can ensure that writing tests is as easy as it can be.

Re: NilAway: Practical nil panic detection for Go

#36

Post by insanitybit »

skybrian wrote: post #30
insanitybit wrote: post #3

> Nil panics are found to be an especially pervasive form of runtime errors in Go programs. Uber’s Go monorepo is no exception to this, and has witnessed several runtime errors in production because of nil panics, with effects ranging from incorrect program behavior to app outages, affecting Uber customers. Insane that Go had decades of programming mistakes to learn from but it chose this path. Anyway, at least Uber…

I don’t see this as a band-aid. It’s doing proper type checking (static analysis) and that seems quite promising? Getting good type errors without requiring type annotations seems like a win over languages that are annotation-heavy. Normally I’d be skeptical about relying on type inference too much over explicit type declarations, but maybe it’s okay for this problem? This is speculative, but I could see this becomin…

I guess you could call it a bandage? The point wasn't to bad-mouth it or undersell it - bandaids are awesome. The point is that we need some external thing to patch holes in the underlying system.

Re: NilAway: Practical nil panic detection for Go

#39

Post by __turbobrew__ »

I don’t really buy the usefulness of trying to statically detect possible nil panics. In their example of a service panicing 3000+ times a day why didn’t they just check the logs to get the stack trace of the panic and fix it there? I don’t see why static analysis was needed to fix that panic in runtime.

What I would really like golang to have is way to send a “last gasp” packet to notify some other system that the runtime is panicing. Ideally at large scales it would be really nice to see what is panicing where and at what time with also stack traces and maybe core dumps. I think that would be much more useful for fixing panics in production.

There was a proposal to add this to the runtime, but it got turned down: https://github.com/golang/go/issues/32333 Most of the arguments against the proposal seem to be that it is hard to determine what is safe to run in a global panic handler. I think the more reasonable option is to tell the go runtime that you want it to send a UDP packet to some address when it panics. That allows the runtime to not support calling arbitrary functions during panicing as it only has to send a UDP packet and then crash.

I could see the static analyzer being useful for helping prevent the introduction of new panics, but I would much rather have better runtime detection.

Re: NilAway: Practical nil panic detection for Go

#40

Post by lopkeny12ko »

npalli wrote: post #18

"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)" Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?

When you have thousands and thousands of engineers, and they are evaluated by how much code they produce, and they need to justify their job and continued employmwment, you end up with a 90M line codebase.
Post reply on HN