Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NilAway: Practical nil panic detection for Go

uber.com

Post reply on HN
Print view View original

11–20 of 262 posts

Re: NilAway: Practical nil panic detection for Go

#11

Post by marsavar »

insanitybit wrote: post #3

> Nil panics are found to be an especially pervasive form of runtime errors in Go programs. Uber’s Go monorepo is no exception to this, and has witnessed several runtime errors in production because of nil panics, with effects ranging from incorrect program behavior to app outages, affecting Uber customers. Insane that Go had decades of programming mistakes to learn from but it chose this path. Anyway, at least Uber…

While this is a great piece of engineering, and will certainly deliver a huge amount of value to any project, the fact that a whole new tool had to be built (and will have to be maintained) to address serious, fundamental shortcomings in the language is really quite sad.

Re: NilAway: Practical nil panic detection for Go

#12

Post by mrkeen »

aatd86 wrote: post #9

Very interesting work. I wonder what were the difficulties encountered. Aliasing? Variable reassignment wrt short declaration shadowing? Hopefully with time, when exploring union types and perhaps a limited form of generalized subtyping (currently it's only interface types) we'll be able to deal with nil for good. Nil is useful, as long as correctly reined in.

> Nil is useful, as long as correctly reined in.

A good way to rein in behaviour is with types. If you need Nil in your domain, great! Give it type 'Nil'.

Re: NilAway: Practical nil panic detection for Go

#13

Post by aatd86 »

mrkeen wrote: post #12
aatd86 wrote: post #9

Very interesting work. I wonder what were the difficulties encountered. Aliasing? Variable reassignment wrt short declaration shadowing? Hopefully with time, when exploring union types and perhaps a limited form of generalized subtyping (currently it's only interface types) we'll be able to deal with nil for good. Nil is useful, as long as correctly reined in.

> Nil is useful, as long as correctly reined in. A good way to rein in behaviour is with types. If you need Nil in your domain, great! Give it type 'Nil'.

Yes that's part of it. It will probably require a nil type which is currently untyped nil when found in interfaces.

The untyped nil type is just not a first-class citizen nowadays.

But with type sets, we could probably have ways to track nillables at the type system level through type assertions.

And where nillables are required such as map values it would be feasible to create some from non nillables then ( interface{T | nil})

But that's way ahead still.

Re: NilAway: Practical nil panic detection for Go

#14

Post by mutatio »

yencabulator wrote: post #7
mseepgood wrote: post #6

Earlier quoted context omitted.

I don't think a pure Go program can core dump, unless you use Cgo (wrongly) or unsafe. It can only panic.

Races between goroutines can corrupt memory. E.g. manipulate a map from two goroutines and you can wreck its internal state.

Can this actually manifest? Even without the -race flag I think maps are a special case which will panic with a concurrent mutation error if access isn't synchronized.

Re: NilAway: Practical nil panic detection for Go

#15

Post by foldr »

insanitybit wrote: post #3

> Nil panics are found to be an especially pervasive form of runtime errors in Go programs. Uber’s Go monorepo is no exception to this, and has witnessed several runtime errors in production because of nil panics, with effects ranging from incorrect program behavior to app outages, affecting Uber customers. Insane that Go had decades of programming mistakes to learn from but it chose this path. Anyway, at least Uber…

It's worth bearing in mind that some of these runtime panics would have happened anyway even if the code had been implemented in (e.g.) Rust. Ugly real world code tends to make quite frequent use of unwrap() or equivalents. For example: https://github.com/search?q=repo%3Arust-lang%2Frust+.unwrap%...

Re: NilAway: Practical nil panic detection for Go

#16

Post by yencabulator »

mutatio wrote: post #14
yencabulator wrote: post #7

Earlier quoted context omitted.

Races between goroutines can corrupt memory. E.g. manipulate a map from two goroutines and you can wreck its internal state.

Can this actually manifest? Even without the -race flag I think maps are a special case which will panic with a concurrent mutation error if access isn't synchronized.

Another example: thread A toggles an interface variable between two types, thread B calls a method on it. You can get the method of type X called with a receiver of type Y.

Re: NilAway: Practical nil panic detection for Go

#17

Post by usrbinbash »

tuetuopay wrote: post #8
insanitybit wrote: post #3

> Nil panics are found to be an especially pervasive form of runtime errors in Go programs. Uber’s Go monorepo is no exception to this, and has witnessed several runtime errors in production because of nil panics, with effects ranging from incorrect program behavior to app outages, affecting Uber customers. Insane that Go had decades of programming mistakes to learn from but it chose this path. Anyway, at least Uber…

> Insane that Go had decades of programming mistakes to learn from but it chose this path. Yup, every time I write some Go I feel like it's been made in a vaccum, ignoring decades of programming language. null/nil is a solved problem by languages with sum types like haskell and rust, or with quasi-sums like zig. It always feels like a regression when switching from rust to go. Kudos to Uber for the tool, it looks ama…

> ignoring decades of programming language

True, and because of this, the language can be learned over a weekend or during onboarding, new hires can rapidly digest codebases and be productive for the company, code is straightforward and easy to read, libraries can be quickly forked and adapted to suit project needs, and working in large teams on the same project is a lot easier than in many other languages, the compiler is blazing fast, and it's concurrency model is probably the most convenient I have ever seen.

Or to put this in less words: Go trades "being-modern" for amazing productivity.

> It always feels like a regression when switching from rust to go.

It really does, and that's what I love about Go. Don't get me wrong I like Rust. I like what it tries to do. But I also love the simplicity, and sheer productiveness of Go. If I have to deal with the odd nil-based error here and there, I consider that a small price to pay.

And judging by the absolute success Go has (measured by contributions to Github), many many many many many developers agree with me on this.

Re: NilAway: Practical nil panic detection for Go

#18

Post by npalli »

"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)"

Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?

Re: NilAway: Practical nil panic detection for Go

#19

Post by insanitybit »

usrbinbash wrote: post #17
tuetuopay wrote: post #8

Earlier quoted context omitted.

> Insane that Go had decades of programming mistakes to learn from but it chose this path. Yup, every time I write some Go I feel like it's been made in a vaccum, ignoring decades of programming language. null/nil is a solved problem by languages with sum types like haskell and rust, or with quasi-sums like zig. It always feels like a regression when switching from rust to go. Kudos to Uber for the tool, it looks ama…

> ignoring decades of programming language True, and because of this, the language can be learned over a weekend or during onboarding, new hires can rapidly digest codebases and be productive for the company, code is straightforward and easy to read, libraries can be quickly forked and adapted to suit project needs, and working in large teams on the same project is a lot easier than in many other languages, the compi…

> And judging by the absolute success Go has (measured by contributions to Github), many many many many many developers agree with me on this.

Yeah, I truly hate this field

Re: NilAway: Practical nil panic detection for Go

#20

Post by insanitybit »

foldr wrote: post #15
insanitybit wrote: post #3

> Nil panics are found to be an especially pervasive form of runtime errors in Go programs. Uber’s Go monorepo is no exception to this, and has witnessed several runtime errors in production because of nil panics, with effects ranging from incorrect program behavior to app outages, affecting Uber customers. Insane that Go had decades of programming mistakes to learn from but it chose this path. Anyway, at least Uber…

It's worth bearing in mind that some of these runtime panics would have happened anyway even if the code had been implemented in (e.g.) Rust. Ugly real world code tends to make quite frequent use of unwrap() or equivalents. For example: https://github.com/search?q=repo%3Arust-lang%2Frust+.unwrap%...

You can `grep unwrap` for Rust, you need an entire SAT solver for Go.
Post reply on HN