An Analysis of vgo
sdboyer.io
An Analysis of vgo
1–10 of 69 posts
Re: An Analysis of vgo
#2Could anyone distill this down for the casual reader? The author has a fairly laborious style and seems to be writing for an audience that is intimately familiar with the internals of dep and vgo.
Re: An Analysis of vgo
#3The basic upshot of all this is that vgo's attempt to avoid NP-completeness in the dependency resolution algorithm is solving a problem that doesn't matter in practice, and is simultaneously creating downsides that do matter in practice.
This is consistent with what I've seen with systems like Cargo. If I had to make a list of top 10 issues I run into with Cargo, theoretical scalability of the core dependency resolution algorithm wouldn't make the cut.
Re: An Analysis of vgo
#4I think the current approach of announcing the death of `dep` before even having a stable alternative (`vgo`) is deeply flawed. With `dep` the ecosystem finally had something most of the developers agreed on, despite whatever shortcomings it had. The model was working. They could have adopted dep and improved on it. The packaging story for Golang has gone from worse to OK to worse again.
After all this progress, we still don't have a stable alternative and no clear date as to when it might be ready or even worse, the community is still not sure vgo will be the final one. Golang should be about pragmatic, practical decisions. The packaging situation is just the opposite.
Re: An Analysis of vgo
#5The basic upshot of all this is that vgo's attempt to avoid NP-completeness in the dependency resolution algorithm is solving a problem that doesn't matter in practice, and is simultaneously creating downsides that do matter in practice. This is consistent with what I've seen with systems like Cargo. If I had to make a list of top 10 issues I run into with Cargo, theoretical scalability of the core dependency resolut…
On the other hand, git became successful by being fast and by not solving certain of the hard theoretical problems that specialists were obsessed with solving. It's hard to know before hand what the healthy compromise might be.
Re: An Analysis of vgo
#6Could anyone distill this down for the casual reader? The author has a fairly laborious style and seems to be writing for an audience that is intimately familiar with the internals of dep and vgo.
The article is about vgo, a proposal (with working prototype) to build package management into Go itself.
vgo bundles several improvements, a major one being the introduction of "modules", which are versioned collections of packages. Modules, among other features, will let us decouple the import path from the file system structure and source repository, making the awkward and much-maligned $GOPATH finally obsolete.
vgo also introduces a new dependency resolution algorithm, minimal version selection (MVS), which is different from traditional dependency resolution algorithms in that it will at any time choose the oldest allowed version that satisfy the minimum version constraints specified in the list of imported packages, where other systems will choose the newest. As MVS does not require (as with dep) a complex boolean SAT solver with potentially pathological, NP-complete cases, it is much simpler and faster to compute. However, it also has downsides. (Edited, thanks to pa7ch for correction.)
The author likes pretty much everything about vgo except the MVS algorithm, which has been deeply controversial since the proposal was first published. The author goes into explanations of why he thinks MVS is a bad fit, but this is a complicated topic, so you really have read both the vgo proposal and this rebuttal to understand what it's all about.
The article was written by Sam Boyer, one of the designers/authors of dep. While he has been courteous about it, there are obviously emotions at play; during its development, dep seemed to have the blessing of the official Go team, and this proposal came out of the blue and probably felt like an ambush.
Re: An Analysis of vgo
#7This badly needs a TL;DR. If there's something seriously wrong with vgo's dependency versioning then it should be possible to explain it in less than 100k words.
Re: An Analysis of vgo
#8The basic upshot of all this is that vgo's attempt to avoid NP-completeness in the dependency resolution algorithm is solving a problem that doesn't matter in practice, and is simultaneously creating downsides that do matter in practice. This is consistent with what I've seen with systems like Cargo. If I had to make a list of top 10 issues I run into with Cargo, theoretical scalability of the core dependency resolut…
I'm curious to hear what problems you've had with cargo. I've found it to be incredibly reliable and easy to use compared to most other similar tools in other languages like java, python, c++ (if its fair to pick on c++ here...)
Re: An Analysis of vgo
#9I think the current approach of announcing the death of `dep` before even having a stable alternative (`vgo`) is deeply flawed. With `dep` the ecosystem finally had something most of the developers agreed on, despite whatever shortcomings it had. The model was working. They could have adopted dep and improved on it. The packaging story for Golang has gone from worse to OK to worse again. After all this progress, we s…
I don't disagree — dep has been a huge improvement on Go's package management story. Until dep was stable, we were using Glide, which is extremely buggy.
That said, if you ignore the doubts and arguments about the sanity of its dependency resolution algorithm, I think vgo's introduction of modules is its real contribution.
Go packages have been problematic since the start since their design, naively, conflates a bunch of concepts that most developers keep separate: File location, file structure and source repository. That is, when you do 'import "github.com/foo/bar"', there's a whole bunch of conventions at play that dictate where it can be fetched from and where it should be in the file system. Aside from the fact that the $GOPATH convention is maddeningly annoying to many developers, the design has several issues, such as that an entire git repository has to be fetched in order to use a nested package, or that a package can have multiple import paths. But getting rid of $GOPATH is in itself a huge deal.
I'm personally less concerned with the new MVS algorithm (though I personally prefer the more traditional lock file approach such as that used by dep, Cargo, Bundler, NPM, etc.), as long as I can get modules. While getting modules retrofitted into depwithout all the other stuff in the proposal would be lovely, I don't know if the module design is too dependent (heh) on the rest of vgo's versioning thesis to be separated out.
Re: An Analysis of vgo
#10Could anyone distill this down for the casual reader? The author has a fairly laborious style and seems to be writing for an audience that is intimately familiar with the internals of dep and vgo.
The article is about vgo, a proposal (with working prototype) to build package management into Go itself. vgo bundles several improvements, a major one being the introduction of "modules", which are versioned collections of packages. Modules, among other features, will let us decouple the import path from the file system structure and source repository, making the awkward and much-maligned $GOPATH finally obsolete. v…
Minor correction: MVS requires modules to specify the minimum version to use of a module. The algorithm will then select the newest among those based on the idea that semver is forward compatible among minor versions. The name minimal version selection throws people off.
Other algorithms would select the newest release within a major version even if no module has ever tested it and it was released 5 seconds ago.
When authors do make incompatible changes and MVS selects a broken dependency, as expected, there are manual escape hatches. More complicated NP complete algo's would have more of an automated answer here by allowing dependencies to have boolean constraints (greater then X, less then Y, not Z etc.) on version so that there is this collaborative summation of constraints to work around authors violating semver or bugs. To summerize poorly: It seems Sam Boyer regrets that dep didn't do what vgo does by default, but believes that a SAT solver could kinda do MVS for the happy path but still solve against the introduction of boolean constraints.
Personally, I'd like to let vgo play out a bit, I'm not convinced boolean constraints are something I really want imposed on me by my dependencies.
More info for those who are looking for more context: https://research.swtch.com/vgo https://github.com/golang/go/issues/24301
Whether you agree or disagree, its novel research in this field and excellent technical writing worth reading in its entirety over a cup of coffee.