Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A response about dep and vgo

peter.bourgon.org

Post reply on HN
Print view View original

41–50 of 139 posts

Re: A response about dep and vgo

#41

Post by jessaustin »

"Dep does not support using multiple major versions of a program in a single build. This alone is a complete showstopper. Go is meant for large-scale work, and in a large-scale program different parts will inevitably need different versions of some isolated dependency."

Russ has asserted this from day one, and has brought several examples out in evidence, but has simply not convinced the committee that it’s true. Calling it a serious problem, let alone a showstopper, is a significant overstatement. The strongest claim that can be made on this point is that it’s a matter of opinion.

OK that's enough for me. "BDFL" clearly correct, random critics totally unwilling to consider the proposition. More language developers are realizing that hierarchical imports with directory name conventions are a good practical thing that just works. Multiple versions come for free with this arrangement. If your tool can't handle that, your tool sucks.

Re: A response about dep and vgo

#42

Post by zeeboo »

munificent wrote: post #18
zeeboo wrote: post #4

Earlier quoted context omitted.

Additionally, dep is, and always was, meant as an experiment to learn from. Russ gave multiple concrete examples of things he learned from dep. That the dep authors don't take this as a win is telling. Pitching dep as the eventual Go dependency management system and having that not be the case is a problem they invented for themselves.

> dep is, and always was, meant as an experiment to learn from. My impression is that the dep folks understand that too. The problem is that there is no consensus on what was learned from it. The dep folks seem to have come away convinced that a SAT-solver approach is the better approach. rsc is clearly convinced of the opposite. Everyone knows it is ultimately rsc's call, so I don't think talking about the power dyn…

I think there's empirical evidence that the SAT-solver approach is not necessary. I have done an analysis on as many Gopkg.{lock,toml} files as I could find, and in no instances did it ever do any non-trivial version selection: the maximal available version at the time was always selected. Additionally, Russ has stated that ~93% of the top 1000 Go packages in the wild build successfully with no changes. I appreciate that they may be convinced, but I think they need to ask themselves what evidence would change their mind. They have had months coming up on a year to figure this out.

https://github.com/zeebo/dep-analysis

Re: A response about dep and vgo

#43

Post by coldtea »

geodel wrote: post #34
coldtea wrote: post #21

> Community consensus is not always possible. If we don’t get there, then the core Go team decides. Technically I am the final decider but what actually happens is that a bunch of long-time Go team members talk through the decision to get to a consensus among ourselves. Was anything major ever decided based on a "community consensus" in Go? What I've read about, and this case is no exception, is that it's always "a b…

> something like Swift (which despite Apple's thing, goes out of its way to engage the community in feature roadmap) Good you say that. Not too long ago solutions from swift-server-group is discarded and Apple simply developed swift-nio in-house. I read that server group was blindsided by this development and now more or less disbanded.

Not sure about the swift-server-group situation -- some where blindsided, but here's e.g. the head of the closely related Vapor (server side web framework) project:

"I've seen a surprising amount of people thinking that the server-side Swift community or the Swift Server working group was somehow blind-sided by SwiftNIO. That couldn't be farther from the truth. We had known about SwiftNIO before the first line of Vapor 3 code was even written."

And in any case, even if the server group was blindsided it's not representative of a general tendency, as there are a very active features discussions e.g. in https://forums.swift.org/c/evolution with the community and public members involved and shaping changes.

Re: A response about dep and vgo

#44

Post by Athas »

zeeboo wrote: post #39
Athas wrote: post #17

Vgo has certainly been a disaster from a communication and community point of view. From an outsider perspective, it looks like a case of maintainer arrogance, and it is hard to believe that Go really has needs that are significantly different from those of e.g. Rust or Java. However, I am also in the unusual position that I had to design and implement a package manager for my own little language recently. My origina…

For what it's worth, I analyzed as many Gopkg.{lock,toml} files I could find in the wild, and found that vgo's algorithms would service all of them. I've done similar analysis for a smaller number of Rust projects using Cargo and had the same findings. I've found it hard to find projects in the wild that actually do any non-trivial version selection. I think this is some good evidence that it will work out in practic…

> I think it's less that Go has significantly different needs, but it's more that people overestimate what their actual needs are.

I think you are right. And I doubt it hurts that SAT solving is a fun problem!

My main package management experience has been with Haskell, which has used the cabal tool for many years. Cabal was a traditional solver-based tool (with the added pain of a global mutable package database, although that is going away), and it frequently broke down in confusing ways. Cabal hell was a widely used term. A few years ago, another tool arrived on the scene, Stack, which used the same package format and such as cabal, but snapshotted the central package list (Hackage) by gathering subsets of packages and versions that were guaranteed to work together (or at least do not have conflicting bounds). It works well[0], and although it does in principle result in a major loss in flexibility, it's rarely something I miss. Importantly, the improvement in reliability was nothing short of astounding. That certainly helped convince me that flexibility may not be a needed feature for a (language) package manager.

[0]: There are all sorts of socio-political stack/cabal conflicts in the Haskell community now, but I'm not sure they are founded in technical issues.

Re: A response about dep and vgo

#45

Post by lizmat »

claydavisss wrote: post #37
munificent wrote: post #22

Earlier quoted context omitted.

> Rails has stayed on the course DHH wants it and hasn't become a mess mainly because DHH remains the BDFL. \* and DHH happens to make good decisions frequently enough. For every successful BDFL example, history is littered with a hundred dead languages and frameworks designed by a single visionary leader who made the wrong decisions.

The only meaningful example of this I can think of is Larry Wall and Perl...but his mistake was in loosening his grip instead of tightening it. Perl6 started off as a utopia wherein Larry was only considered slightly more powerful than any other contributor. This resulted in a revolving door of clowns taking the reigns and rapidly resigning as Larry looked on. Fortunately Perl5 still does the Pumpking thing which is…

> This resulted in a revolving door of clowns taking the reigns and rapidly resigning as Larry looked on.

I would say that this hasn't been the case for the past 10 years at least. Patrick Michaud has been Perl 6 pumpking since then, to make room for Jonathan Worthington about 2 years ago. Hardly a revolving door and hardly clowns.

Re: A response about dep and vgo

#46

Post by jessaustin »

Athas wrote: post #38
pas wrote: post #33

Earlier quoted context omitted.

Could you contrast Cargo and vgo for those who only know the former?

There are lots of small differences, but I would say there are two major: * Semantic Import Versioning. A package is identified by some name. I think vgo calls it an "import path", but I have also seen "package path" used. Every version of that package must remain compatible with previous versions. You may not break compatibility (i.e. increment the major semver number) without also renaming the package. There is som…

Minimal Version Selection.

That's really interesting. Obviously this would lead to fewer regressions, but it seems opposed to the "security problems fixed for free" scenario you hear about with other package dependency schemes. Were we all just imagining that scenario? Is there another way we should be handling vulnerabilities, i.e. by "repudiating" old versions rather than simply releasing new ones? Then you could have something like "Minimal Unrepudiated Version Selection".

Re: A response about dep and vgo

#47

Post by kenhwang »

Athas wrote: post #38
pas wrote: post #33

Earlier quoted context omitted.

Could you contrast Cargo and vgo for those who only know the former?

There are lots of small differences, but I would say there are two major: * Semantic Import Versioning. A package is identified by some name. I think vgo calls it an "import path", but I have also seen "package path" used. Every version of that package must remain compatible with previous versions. You may not break compatibility (i.e. increment the major semver number) without also renaming the package. There is som…

I'm still mind-boggled by the crusade against the lockfile. It's a completely unambiguous source for everything that costs essentially nothing, why is there so much time and energy devoted to getting rid of it? Especially with all the drawbacks minimal version selection comes with; exact version selection with the lockfile seems better in just about every case.

Re: A response about dep and vgo

#48

Post by nathancahill »

Athas wrote: post #38
pas wrote: post #33

Earlier quoted context omitted.

Could you contrast Cargo and vgo for those who only know the former?

There are lots of small differences, but I would say there are two major: * Semantic Import Versioning. A package is identified by some name. I think vgo calls it an "import path", but I have also seen "package path" used. Every version of that package must remain compatible with previous versions. You may not break compatibility (i.e. increment the major semver number) without also renaming the package. There is som…

> vgo tries to install the oldest version of a packages that still satisfies any lower bounds

This alone would make npm 100x less painful to use.

Re: A response about dep and vgo

#49

Post by Athas »

jessaustin wrote: post #46
Athas wrote: post #38

Earlier quoted context omitted.

There are lots of small differences, but I would say there are two major: * Semantic Import Versioning. A package is identified by some name. I think vgo calls it an "import path", but I have also seen "package path" used. Every version of that package must remain compatible with previous versions. You may not break compatibility (i.e. increment the major semver number) without also renaming the package. There is som…

Minimal Version Selection. That's really interesting. Obviously this would lead to fewer regressions, but it seems opposed to the "security problems fixed for free" scenario you hear about with other package dependency schemes. Were we all just imagining that scenario? Is there another way we should be handling vulnerabilities, i.e. by "repudiating" old versions rather than simply releasing new ones? Then you could h…

I think reproducible builds are more important than having security problems fixed behind your back. It's not like vgo makes it hard to upgrade all dependencies to their most recent version - it's done automatically with a single command, if you want it.

Re: A response about dep and vgo

#50

Post by jonstewart »

munificent wrote: post #18
zeeboo wrote: post #4

Earlier quoted context omitted.

Additionally, dep is, and always was, meant as an experiment to learn from. Russ gave multiple concrete examples of things he learned from dep. That the dep authors don't take this as a win is telling. Pitching dep as the eventual Go dependency management system and having that not be the case is a problem they invented for themselves.

> dep is, and always was, meant as an experiment to learn from. My impression is that the dep folks understand that too. The problem is that there is no consensus on what was learned from it. The dep folks seem to have come away convinced that a SAT-solver approach is the better approach. rsc is clearly convinced of the opposite. Everyone knows it is ultimately rsc's call, so I don't think talking about the power dyn…

It also ignores rsc's clear algorithmic preferences. No, RE2 does not support back-references, because they require a back-tracking implementation with exponential worst-case behavior. RE2 uses an NFA with O(nm) worst-case behavior. It's a classical Unix approach where a simple implementation is preferable to having more features, especially if there are algorithmic considerations.
Post reply on HN