A response about dep and vgo • Hacker News Board

Re: A response about dep and vgo

#101

Post by Nullabillity » Sat, Jul 28, 2018, 4:45 AM UTC

kjksf wrote: post #63

necubi wrote: post #58

Earlier quoted context omitted.

Other package managers (cargo, gemfile, etc.) also give reproducible builds by using a lockfile. Upgrading dependencies to their maximum version is an explicit action (cargo update). It's not clear to me why the go team things lockfiles are a huge issue.

Lock files also remove the "silent security fix" that was touted as the benefit of other systems. You can't have it both ways. You either get reproducible builds (by default with vgo, by adding lock files in other systems) or you get silent upgrades that potentially fix security issues (but also potentially introduce security issues). You also mis-characterize the position of vgo creator. He doesn't think that lock f…

Lockfiles have other benefits, such as locking down dependencies' hashes to prevent future tampering.

Re: A response about dep and vgo

#102

Post by uluyol » Sat, Jul 28, 2018, 4:46 AM UTC

One thing that I think is being missed is that code/a product is not the only thing that matters. In fact, it's one of the less important things here.

Once a design is set, implementing it is "just work". I'm not sure trying to undervalue that, but the important part is first investigating the design. The package management committee came up with the tried and tested design while Russ was trying to understand the true requirements. Both have their merits, and time will tell if Russ is right here. The thing to do keep in mind is that investigative work is inherently "wasteful".

Re: A response about dep and vgo

#103

Post by EdiX » Sat, Jul 28, 2018, 4:47 AM UTC

lilyball wrote: post #73

Athas wrote: post #38

Earlier quoted context omitted.

There are lots of small differences, but I would say there are two major: * Semantic Import Versioning. A package is identified by some name. I think vgo calls it an "import path", but I have also seen "package path" used. Every version of that package must remain compatible with previous versions. You may not break compatibility (i.e. increment the major semver number) without also renaming the package. There is som…

> but also gives you reproducible builds without using a lockfile (which is nice) I find the claims that vgo removes the lockfile disingenuous. Yes it's technically correct in that there's no lockfile, but it's incorrect in that you've basically turned the file that declares dependencies into the equivalent of a lockfile. With something like cargo, I can `cargo update` and it will fetch new versions of my dependencie…

go get will update a dependency and edit go.mod for you automatically. And using go mod -fix will fix misleading versions specified in go.mod (i.e. if you require version X but are getting version Y because of a transitive dependency the require will be updated to say Y).

Re: A response about dep and vgo

#104

Post by mseepgood » Sat, Jul 28, 2018, 5:39 AM UTC

cweagans wrote: post #14

zeeboo wrote: post #4

Earlier quoted context omitted.

Additionally, dep is, and always was, meant as an experiment to learn from. Russ gave multiple concrete examples of things he learned from dep. That the dep authors don't take this as a win is telling. Pitching dep as the eventual Go dependency management system and having that not be the case is a problem they invented for themselves.

Throwing away dep means also throwing away the nontechnical groundwork that dep was built on - for instance, user research. Seems pretty careless to me, especially when the alternative is the product of one person's thinking on the subject done in a vacuum without the input of an entire committee of smart, reasonable people who have literally spent years diving into this specific domain.

> for instance, user research

"If I had asked people what they wanted, they would have said faster horses."

(or yet another Bundler/npm/Cargo clone.)

Re: A response about dep and vgo

#105

Post by pcwalton » Sat, Jul 28, 2018, 5:52 AM UTC

zeeboo wrote: post #39

Athas wrote: post #17

Vgo has certainly been a disaster from a communication and community point of view. From an outsider perspective, it looks like a case of maintainer arrogance, and it is hard to believe that Go really has needs that are significantly different from those of e.g. Rust or Java. However, I am also in the unusual position that I had to design and implement a package manager for my own little language recently. My origina…

For what it's worth, I analyzed as many Gopkg.{lock,toml} files I could find in the wild, and found that vgo's algorithms would service all of them. I've done similar analysis for a smaller number of Rust projects using Cargo and had the same findings. I've found it hard to find projects in the wild that actually do any non-trivial version selection. I think this is some good evidence that it will work out in practic…

The problem with minimal version selection isn't that it can't support nontrivial version selection but rather that it doesn't automatically select the newest versions of packages.

Re: A response about dep and vgo

#106

Post by pcwalton » Sat, Jul 28, 2018, 5:54 AM UTC

claydavisss wrote: post #35

stcredzero wrote: post #11

Earlier quoted context omitted.

Ya, I've been following all this drama since the vgo proposal and I also find it somewhat rich...On another note I hope this is the last we hear about all this, please lets move on. I only became aware of this by going to a meetup and seeing a talk. I just want something that's workable and not broken. The reason BDFLs are in the position of making that decision is because they have proven through the accumulation of…

A competition still has to be judged by someone. The only curious element here is that a (the?) judge also had the winning entry. Russ claims he got buy-in from the other core members and so far no one has contradicted that. AND...vgo is better than dep. Let's not forget that. Peter can hem and haw about politics but in the end the best horse won. Go developers want the tools to be as good as possible, 99% of them do…

> AND...vgo is better than dep. Let's not forget that. Peter can hem and haw about politics but in the end the best horse won.

Nobody has worked with minimum version selection and compared it to the traditional approach enough to know. I have serious reservations as to how it will work in practice.

Re: A response about dep and vgo

#107

Post by pcwalton » Sat, Jul 28, 2018, 5:59 AM UTC

DannyBee wrote: post #95

munificent wrote: post #18

Earlier quoted context omitted.

> dep is, and always was, meant as an experiment to learn from. My impression is that the dep folks understand that too. The problem is that there is no consensus on what was learned from it. The dep folks seem to have come away convinced that a SAT-solver approach is the better approach. rsc is clearly convinced of the opposite. Everyone knows it is ultimately rsc's call, so I don't think talking about the power dyn…

I'll present a counterpoint of a different kind, since everyone is arguing about what current package managers do and don't do. Satisfiability problems of this kind appear in a ridiculous number of fields and applications (and not just by reduction). The vast majority of them, in practice, are approximated rather than exactly solved. Most of the ones that are exactly solved are in software verification, model checkin…

> The idea that "package management" is one of those areas that absolutely must be exactly solved to generate acceptable results seems to me to be ... probably wrong.

The logical conclusion of your statement is that minimal version selection is the wrong approach! Minimal version selection is an "exact" solution, in contrast to the traditional one. You would only arrive at MVS if you considered the problem of selecting precise dependencies to be so important that it's worth making it the user's problem instead of having the tool solve it. The philosophy of the traditional package management solution is that it's best to have the tool do the right thing--select most recent versions that satisfy constraints--so that the user is free to worry about more important things.

> There are much more critical and harder things we've been approximating for years and everyone is just fine with it.

Yes! That's why MVS, and by extension vgo, is oriented around solving a non-problem!

> It's also a very complex one that often fails in interesting ways in both this, and other, domains.

I cannot name one example of a single time SAT solving has failed in Cargo.

Re: A response about dep and vgo

#108

Post by pcwalton » Sat, Jul 28, 2018, 6:12 AM UTC

Athas wrote: post #49

jessaustin wrote: post #46

Earlier quoted context omitted.

Minimal Version Selection. That's really interesting. Obviously this would lead to fewer regressions, but it seems opposed to the "security problems fixed for free" scenario you hear about with other package dependency schemes. Were we all just imagining that scenario? Is there another way we should be handling vulnerabilities, i.e. by "repudiating" old versions rather than simply releasing new ones? Then you could h…

I think reproducible builds are more important than having security problems fixed behind your back. It's not like vgo makes it hard to upgrade all dependencies to their most recent version - it's done automatically with a single command, if you want it.

> It's not like vgo makes it hard to upgrade all dependencies to their most recent version - it's done automatically with a single command, if you want it.

And how does it guarantee that this actually results in anything that works, if upper bounds are not supported?

Re: A response about dep and vgo

#109

Post by samat » Sat, Jul 28, 2018, 6:32 AM UTC

I don’t have extensive knowledge of the matter in hand, but this narrative seems much more coherent. And sad for Go/Google. So much for “open”. Just like Android/Google play services thing.

Calling something “open” a hundred times a day does not remove one bit of corporate control.

Re: A response about dep and vgo

#110

Post by kaeshiwaza » Sat, Jul 28, 2018, 7:26 AM UTC

pcwalton wrote: post #105

zeeboo wrote: post #39

Earlier quoted context omitted.

For what it's worth, I analyzed as many Gopkg.{lock,toml} files I could find in the wild, and found that vgo's algorithms would service all of them. I've done similar analysis for a smaller number of Rust projects using Cargo and had the same findings. I've found it hard to find projects in the wild that actually do any non-trivial version selection. I think this is some good evidence that it will work out in practic…

The problem with minimal version selection isn't that it can't support nontrivial version selection but rather that it doesn't automatically select the newest versions of packages.

It's a feature not a problem. It select the only know working solution. You're still free to update to newest versions.