Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taking Go modules for a spin

dave.cheney.net

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Taking Go modules for a spin

#61

Post by slrz »

dragonwriter wrote: post #53
eikenberry wrote: post #46

Earlier quoted context omitted.

> unintentional backwards incompatibilities You mean a bug? Because that's what that is and it is no different from any other bug, and like any other bug they are outside the scope of dependency specifications as they are unintended.

> like any other bug they are outside the scope of dependency specifications Known relevant bugs in particular versions of dependencies are not outside the scope of what non-vgo dependency management solutions address.

Maybe they should be. If you can make it work, fixing the bug seems like the obviously superior solution compared to letting it fester and working around it locally with incompatibility declarations, slowly degrading the ecosystem up to the point where you have lots of little islands that can't be used together anymore in a sane manner.

Re: Taking Go modules for a spin

#62

Post by Lazare »

slrz wrote: post #60
Lazare wrote: post #56

Earlier quoted context omitted.

The counterarguments are, basically: 1. vgo focuses on the wrong issue (if you're spending a ton of time resolving and re-resolving your dependency graph, maybe the issue is your build process). 2. vgo will get the wrong answers and/or make development much harder There's a long writeup of some of the ways vgo can go wrong here: https://sdboyer.io/vgo/failure-modes/ and some background here: https://sdboyer.io/vgo/in…

> There are some quirks in the golang ecosystem which mean you end up analyzing your dependency tree way, way, more than you do in basically any other common language. What are these?

This comment from an earlier discussion here covers some of them: https://news.ycombinator.com/item?id=17185335

Basically, dep/glide do a bunch of stuff, including recursively parsing import statements because of How Go Works (tm). Other package managers don't, because they have lock files, and central repositories. Go expects you to just be able to throw a ton of raw code into your GOPATH and have it all magically fetched from github, which is super cool, but also very hard to do quickly, and not really something other languages are clamouring to support.

(A lot of attention has been focused on vgo's solver, and it is much faster, but the solver isn't what takes up all the time; the speedup from dep/glide to vgo seems to be almost entirely related to the changes in how dependencies are declared. Saving 10ms on a more efficient solver algorithm means nothing if the overall process is spending 12s grinding through slow disc and network access.)

And when you survey the language ecosystem, you see a lot of languages very enthusiastically committed to traditional package managers (with lock files) and centralised repositories. Cargo, composer, npm/yarn, bundler/ruby gems - recent history is full of languages happily moving in that direction. Go is an exception, and I don't see anyone actively copying that quirk any time soon.

Re: Taking Go modules for a spin

#63

Post by dilap »

bbatha wrote: post #52
greenhouse_gas wrote: post #49

Earlier quoted context omitted.

>Actually solving a constraint graph is an NP-hard problem How big of a deal is this IRL? Assuming you have 1000 modules, how long should it take to solve the graph?

Not very the parts that make it NP hard are allowing libraries to specify maximum versions (and other more complex version ranges). Most of the time libraries use minimum constraints (~) or (^) which allows the heuristic to work like go's algorithm. For rust, node, and other languages libraries can be imported twice as different versions (without requiring a major version renaming like go) this also allows the heuris…

does rust have mutable package-level state like go?

Re: Taking Go modules for a spin

#64

Post by bbatha »

dilap wrote: post #63
bbatha wrote: post #52

Earlier quoted context omitted.

Not very the parts that make it NP hard are allowing libraries to specify maximum versions (and other more complex version ranges). Most of the time libraries use minimum constraints (~) or (^) which allows the heuristic to work like go's algorithm. For rust, node, and other languages libraries can be imported twice as different versions (without requiring a major version renaming like go) this also allows the heuris…

does rust have mutable package-level state like go?

It does. There are ways to mark a package as "only once" in the dep graph. For instance, C libraries are required to be marked in this way.

The only once constraint also has a nice out for the SAT solver, if you reach a conflict or something that can't be solved cheaply you just make the user select a version that may not be compatible with the constraints. Bower, dep, and maven work that way.

Re: Taking Go modules for a spin

#65

Post by steveklabnik »

dilap wrote: post #63
bbatha wrote: post #52

Earlier quoted context omitted.

Not very the parts that make it NP hard are allowing libraries to specify maximum versions (and other more complex version ranges). Most of the time libraries use minimum constraints (~) or (^) which allows the heuristic to work like go's algorithm. For rust, node, and other languages libraries can be imported twice as different versions (without requiring a major version renaming like go) this also allows the heuris…

does rust have mutable package-level state like go?

Packages are made up of modules, and modules can have global state. But doing so directly is unsafe, specifically because it can introduce a data race. Rust also does not have “life before main”, so it doesn’t get used in the same way as languages that do. I’m not sure if Go does?

Re: Taking Go modules for a spin

#66

Post by dragonwriter »

slrz wrote: post #61
dragonwriter wrote: post #53

Earlier quoted context omitted.

> like any other bug they are outside the scope of dependency specifications Known relevant bugs in particular versions of dependencies are not outside the scope of what non-vgo dependency management solutions address.

Maybe they should be. If you can make it work, fixing the bug seems like the obviously superior solution compared to letting it fester and working around it locally with incompatibility declarations, slowly degrading the ecosystem up to the point where you have lots of little islands that can't be used together anymore in a sane manner.

> If you can make it work, fixing the bug seems like the obviously superior solution compared to letting it fester and working around it locally with incompatibility declarations

Fixing the bug creates a new version. Unless you are going to create the mess of unpublishing packages or replacing packages with new different ones with the same identified version (both of which are problematic in a public package ecosystem), the fact that maintainers should fix bugs that occur in published versions doesn't , at all, address the issue for downstream projects that is addressed by incompatibility declarations in a dependency management system, even before considering that downstream maintainers can't force upstream maintainers to fix bugs in the first place.

Re: Taking Go modules for a spin

#67

Post by dilap »

bbatha wrote: post #64
dilap wrote: post #63

Earlier quoted context omitted.

does rust have mutable package-level state like go?

It does. There are ways to mark a package as "only once" in the dep graph. For instance, C libraries are required to be marked in this way. The only once constraint also has a nice out for the SAT solver, if you reach a conflict or something that can't be solved cheaply you just make the user select a version that may not be compatible with the constraints. Bower, dep, and maven work that way.

You anticipated where I was going, which is mutable state + multiple copies of packages seems like a recipe for trouble.

So, I'm not sure how happy I would be as a user if my package installer bailed out and asked me to choose!

Out of curiosity, how do you mark your package as "only once" in cargo? I tried googling, and didn't find an answer, but did find a bug where people couldn't build because they ended up depending on two different versions of C libraries!

It does make wonder if MVS will solve real pain in practice. :-)

Re: Taking Go modules for a spin

#68

Post by dilap »

steveklabnik wrote: post #65
dilap wrote: post #63

Earlier quoted context omitted.

does rust have mutable package-level state like go?

Packages are made up of modules, and modules can have global state. But doing so directly is unsafe, specifically because it can introduce a data race. Rust also does not have “life before main”, so it doesn’t get used in the same way as languages that do. I’m not sure if Go does?

(I replied but the reply vanished. If it reappears, apologies for the dup.)

Yeah, go has a magic function `func init()` which gets called before main. (You can actually have as many init's as you want, and they all get called.)

Probably evil, though so far it hasn't hurt me in the same way as, e.g., c++ constructors have. Maybe because it's more explicit and thus you're less likely to use it in practice.

Re: Taking Go modules for a spin

#69

Post by steveklabnik »

dilap wrote: post #68
steveklabnik wrote: post #65

Earlier quoted context omitted.

Packages are made up of modules, and modules can have global state. But doing so directly is unsafe, specifically because it can introduce a data race. Rust also does not have “life before main”, so it doesn’t get used in the same way as languages that do. I’m not sure if Go does?

(I replied but the reply vanished. If it reappears, apologies for the dup.) Yeah, go has a magic function `func init()` which gets called before main. (You can actually have as many init's as you want, and they all get called.) Probably evil, though so far it hasn't hurt me in the same way as, e.g., c++ constructors have. Maybe because it's more explicit and thus you're less likely to use it in practice.

Cool, thanks!

Re: Taking Go modules for a spin

#70

Post by atombender »

akavel wrote: post #55
atombender wrote: post #17

Earlier quoted context omitted.

The Go team, for years, actually insisted that package management was something the Go community had to go and figure out. As you say, Google uses a monorepo where they check everything into a single tree. One of the core Go developers -- I forget who, unfortunately -- actually claimed at one point that they didn't want to design a package management system because they didn't know how ; since Google used a monorepo,…

Personally, I actually never thought Dep was blessed by the Go team to the point that it "was going to be official". So I wouldn't be so fast to say "pretty much everyone". Notably, some loud people "thought" so; I've noticed that many quiet people quietly did not (as can be seen e.g. by the numerous voiced agreements and endorsements of the vgo prototype on the mailing list). Again, I personally expected exactly som…

What impression you had probably depends on whether you read the Go mailing lists or not. All of my colleagues, myself included, had gotten the impression, who knows how, that Dep was official. We don't follow the Go lists.

The Dep situation is very similar to that of Eric S. Raymond's attempt at replacing the Linux kernel's config tool. Instead of presenting a design proposal and discussing it in public, he pretty much finished the project on his own, perhaps thinking that a working version would lead to adoption by users and thus forcing the kernel team to accept something users liked. Or perhaps he assumed he had clout in the Linux community, which of course he didn't. Either way, this kind of brute-forcing just leads to wasted work and resentment.

Post reply on HN