The vgo proposal is accepted. Now what? • Hacker News Board

Re: The vgo proposal is accepted. Now what?

#51

Post by jimmy1 » Wed, May 30, 2018, 2:27 AM UTC

throwaway243425 wrote: post #33

To an outsider this may sound like there is some sort of process that resulted in this solution, there actually isn't any. The committee is nothing more than a simple bureaucracy to the point that it is almost a joke how Russ makes a proposal, community is against it, then it gets accepted by the Committee. It is all just a funny joke.

> To an outsider

> Russ

> throwaway

Something tells me you aren't really an outsider.

Anyways,

> community is against it, then it gets accepted by the Committee

Is this your primary gripe? Because this has happened pretty consistently in the history of almost all open source languages. Think of how many JSRs were hotly contested, only to be accepted. Open source does not mean democratic development, and thankfully so because you might have ended up with this https://i.redd.it/7t1p88ct13ez.jpg

Re: The vgo proposal is accepted. Now what?

#52

Post by amasad » Wed, May 30, 2018, 2:50 AM UTC

eberkund wrote: post #20

I wish they had just copied Rust/Cargo. I remember reading a comment on GitHub somewhere from one of the Go maintainers who responded to someone expressing a similar sentiment and his reply was basically that Go is somehow different than every other language and they need to explore and find a unique custom solution for their particular use case. Has it ever been addressed anywhere why the tried and true "list of pac…

There are a lot of suboptimal design decisions that's shared across the state of the art in package management (cargo, pipenv, npm, yarn, etc) that the Go team is not taking for granted.

One is reproducible builds. The standard answer is lock files which is extra bloat and leads to merge conflicts. vgo is trying to avoid it with the "minimal versions" strategy (https://research.swtch.com/vgo-repro).

Another is shared (or recursive) dependencies. The semver answer to this is that if the versions match then they should be shared if not then they should be duplicated. But what do you do if it's a singleton, listens on a port, or exposes a port. Right now, with npm for example, good luck with that, you're in for a world of pain. On the other hand with vgo's "semantic import versioning" they're trying to make version interop more explicit.

Just because 90% of the time mainstream package mangers work doesn't mean it's a solved problem. Kudos for the Go team for trying to advance the state of the art.

Re: The vgo proposal is accepted. Now what?

#53

Post by lobster_johnson » Wed, May 30, 2018, 2:53 AM UTC

pcwalton wrote: post #50

lobster_johnson wrote: post #28

Earlier quoted context omitted.

Rust/Cargo had the luxury of being a greenfield project that could adopt semver from the beginning, whereas Go made the mistake of starting out, and then going years , without any official package management solution. As a result, Go has a swathe of applications and libraries that use specific workflows as well as a mélange of community-developed package management tools such as godep, Glide and dep. The semver stand…

Rust was actually around for a good while before Cargo was adopted. (In fact, there were two Cargos, the older of which bore very little resemblance to the Cargo of today.) Of course, Go was stable for longer. I have to admit I'm a bit confused as to why the dependency resolution algorithm in dep is seen as slow. The speed of the solver is not a problem in any other package management system I've seen. If it is indee…

Here [1] is the "dep ensure -v" output for a project of mine. It takes almost 12 seconds even when there are no changes to the actual file. I don't know why, or whether it's actually the solver (though the output seems to indicate it).

[1] https://gist.github.com/atombender/7c28f1d371fcb139e1e742a08...

Re: The vgo proposal is accepted. Now what?

#54

Post by lobster_johnson » Wed, May 30, 2018, 2:58 AM UTC

skybrian wrote: post #47

adwhit wrote: post #2

For those who missed it, Sam Boyer (maintainer of dep) wrote a detailed post about why he thinks vgo (or rather Minimum Version Selection) is inadequate[0]. The key argument is With dep, it’s usually easy to point to failures - they’re explicit, verbose (and, currently, often difficult to understand, and printed out at the end of a dep ensure run. The primary failure mode in vgo, however, is silent false positives -…

It's worth pointing out that when working in a monorepo, there are no version numbers and no package system to tell you that upstream broke you, or you broke something downstream. So how do you find bugs? You run tests. The same can be true if you're working with package management. Most bugs aren't found by the dependency tracking system anyway. Having good tests will tell you about incompatibilities that upstream d…

Testing and package management aren't mutually exclusive. If you're in a monorepo, all your code evolves in lockstep. A core tenet of versioned package management is to get reproducible builds that only need to break once you choose to evolve forward in time (i.e. upgrade).

Historically, most languages (C, C++, pre-Maven Java) haven't had package management at all, and so dependencies have typically been managed by vendoring the code (or JAR files). JAR files worked okay, but vendoring incurs maintenance overhead that isn't acceptable in today's environment. git submodules are theoretically a solution, but also high-maintenance.

Re: The vgo proposal is accepted. Now what?

#55

Post by kibwen » Wed, May 30, 2018, 2:59 AM UTC

amasad wrote: post #52

eberkund wrote: post #20

I wish they had just copied Rust/Cargo. I remember reading a comment on GitHub somewhere from one of the Go maintainers who responded to someone expressing a similar sentiment and his reply was basically that Go is somehow different than every other language and they need to explore and find a unique custom solution for their particular use case. Has it ever been addressed anywhere why the tried and true "list of pac…

There are a lot of suboptimal design decisions that's shared across the state of the art in package management (cargo, pipenv, npm, yarn, etc) that the Go team is not taking for granted. One is reproducible builds. The standard answer is lock files which is extra bloat and leads to merge conflicts. vgo is trying to avoid it with the "minimal versions" strategy ( https://research.swtch.com/vgo-repro ). Another is shar…

> The standard answer is lock files which is extra bloat and leads to merge conflicts.

Bloat how? A lockfile is a handful of bytes. A thousand lockfiles could fit in the space of a single Go hello world binary. And I've never heard of a merge conflict from a lockfile. What are the actual arguments against lockfiles?

Re: The vgo proposal is accepted. Now what?

#56

Post by lobster_johnson » Wed, May 30, 2018, 3:15 AM UTC

_pfxa wrote: post #4

I still wonder why solving a solved problem took so long to solve for the Go community, given they already solved it anyways? That is, in more proper words, first of all, language specific package management is mostly a solved problem. There are possible improvements, and maybe vgo realises some of them, but that's mostly a bikeshedding problem. What users need is to be able to declare what packages they need, in wha…

> Now Cox's solution might indeed be better (though I think it's an overkill ...

vgo is actually much, much simpler than dep. The sheer number of words in Russ Cox's series of blog posts belies its simplicity. vgo doesn't need a SAT solver. If you look at many of the issues dep is struggling with, they're related to solving N libraries with transitive dependencies up the wazoo.

Cox's long treatise reflects the complexity of the problem space. Developers tend to brush off package management as being simple. But once you include range-based version constraints and transitive dependencies, it gets a bit messier. Look at NPM and Yarn; they're still struggling to get all the details right. On the other hand, there's Ruby's Bundler. It came out in 2009, RubyGems in 2004, and I've never had a single issue with the toolchain (other than messing up my own constraints). I don't know what kind of magic elixir they were drinking, but somehow those guys managed to nail it from day one.

Re: The vgo proposal is accepted. Now what?

#57

Post by amasad » Wed, May 30, 2018, 3:16 AM UTC

kibwen wrote: post #55

amasad wrote: post #52

Earlier quoted context omitted.

There are a lot of suboptimal design decisions that's shared across the state of the art in package management (cargo, pipenv, npm, yarn, etc) that the Go team is not taking for granted. One is reproducible builds. The standard answer is lock files which is extra bloat and leads to merge conflicts. vgo is trying to avoid it with the "minimal versions" strategy ( https://research.swtch.com/vgo-repro ). Another is shar…

> The standard answer is lock files which is extra bloat and leads to merge conflicts. Bloat how? A lockfile is a handful of bytes. A thousand lockfiles could fit in the space of a single Go hello world binary. And I've never heard of a merge conflict from a lockfile. What are the actual arguments against lockfiles?

More things, more state = bloat. Don't you agree that if you can do something without holding state it's invariably simpler? That's what MVS does.

There's also cases where your lock file and manifest are not in agreement. e.g. https://github.com/rust-lang/cargo/issues/4100

Re: The vgo proposal is accepted. Now what?

#58

Post by skybrian » Wed, May 30, 2018, 3:30 AM UTC

lobster_johnson wrote: post #54

skybrian wrote: post #47

Earlier quoted context omitted.

It's worth pointing out that when working in a monorepo, there are no version numbers and no package system to tell you that upstream broke you, or you broke something downstream. So how do you find bugs? You run tests. The same can be true if you're working with package management. Most bugs aren't found by the dependency tracking system anyway. Having good tests will tell you about incompatibilities that upstream d…

Testing and package management aren't mutually exclusive. If you're in a monorepo, all your code evolves in lockstep. A core tenet of versioned package management is to get reproducible builds that only need to break once you choose to evolve forward in time (i.e. upgrade). Historically, most languages (C, C++, pre-Maven Java) haven't had package management at all, and so dependencies have typically been managed by v…

Yes, you are right. And I believe vgo is supposed to guarantee reproducible builds just like other package systems.

However, when you upgrade a dependency, it's still possible that you're using a particular combination of library versions that have never been tested before.

Some incompatibilities can be prevented by looking at version constraints. But you're not left with no error detection if the package system fails to detect an incompatibility; in the end, what matters is that the code compiles and the tests pass.

Re: The vgo proposal is accepted. Now what?

#59

Post by jasonwatkinspdx » Wed, May 30, 2018, 3:37 AM UTC

kibwen wrote: post #55

amasad wrote: post #52

Earlier quoted context omitted.

There are a lot of suboptimal design decisions that's shared across the state of the art in package management (cargo, pipenv, npm, yarn, etc) that the Go team is not taking for granted. One is reproducible builds. The standard answer is lock files which is extra bloat and leads to merge conflicts. vgo is trying to avoid it with the "minimal versions" strategy ( https://research.swtch.com/vgo-repro ). Another is shar…

> The standard answer is lock files which is extra bloat and leads to merge conflicts. Bloat how? A lockfile is a handful of bytes. A thousand lockfiles could fit in the space of a single Go hello world binary. And I've never heard of a merge conflict from a lockfile. What are the actual arguments against lockfiles?

There are situations where lock files, as implemented in some languages, can generate conflicts. Honestly I consider this a benefit, because I'd like to know about different expectations as early and explicitly as possible.

The bloat argument seems frankly absurd to me.

Re: The vgo proposal is accepted. Now what?

#60

Post by kibwen » Wed, May 30, 2018, 3:39 AM UTC

lobster_johnson wrote: post #53

pcwalton wrote: post #50

Earlier quoted context omitted.

Rust was actually around for a good while before Cargo was adopted. (In fact, there were two Cargos, the older of which bore very little resemblance to the Cargo of today.) Of course, Go was stable for longer. I have to admit I'm a bit confused as to why the dependency resolution algorithm in dep is seen as slow. The speed of the solver is not a problem in any other package management system I've seen. If it is indee…

Here [1] is the "dep ensure -v" output for a project of mine. It takes almost 12 seconds even when there are no changes to the actual file. I don't know why, or whether it's actually the solver (though the output seems to indicate it). [1] https://gist.github.com/atombender/7c28f1d371fcb139e1e742a08...

That's quite weird. When I run `rm Cargo.lock && cargo generate-lockfile` on the Servo repo (test performed on the cheapest VPS that money can buy) it exits near-instantly (after first spending three seconds trying to git-fetch new versions of the dozen custom dependencies that live on Github rather than crates.io). For reference, here's what Servo's dependency graph looked like two years ago (July 2016): https://dirkjan.ochtman.nl/files/servo-graph.svg ; the number of transitive dependencies is quite large and yet the runtime of version selection is negligible.