Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go += Package Versioning

research.swtch.com

Post reply on HN
Print view View original

191–200 of 213 posts

Re: Go += Package Versioning

#191

Post by mCOLlSVIxp6c »

overcyn wrote: post #143
mCOLlSVIxp6c wrote: post #93

I predict that under the proposed minimum version selection system some package will decide it's important and flawless enough that users should always depend on the most recent version. The package will recommend that users depend on version v1.0.0 but the first release will be v1.100.0. Subsequent releases will decrement the minor version to make sure newer releases are selected.

User's wouldn't be able to depend on version v1.0.0 if it doesn't exist. From what I understand the tool isn't downloading the list of all versions from GitHub and selecting the minimum. Its searching for the maximum version number (i.e. minimal required version), from your go.mod file and all your dependencies' go.mod file.

I think you're correct. So my nasty little hack wouldn't work. That's probably a good thing.

Re: Go += Package Versioning

#192

Post by stubish »

munificent wrote: post #33

I'm going to comment mostly on the parts of the proposal that I think are wrong, but don't take this to be an overall negative response. I'm excited to see smart folks working on this, and package management is a really hard problem. There are no silver bullets to code reuse. Context for those who don't know: I along with Natalie Weizenbaum wrote pub[1], the package manager used for Dart. > Instead of concluding from…

Current Go vendoring already allows two versions of a package to be used at the same time, and it is problematic. Both copies of the package will run their initialization, which works if they are completely self contained, but if they interact with any other global state things can start going wrong. You mutexes protecting external global resources don't work, because they are in different namespaces. These and similar problems are why common wisdom is that libraries should not pull in vendored dependencies.

So yes, attempting to allow multiple versions of the same module will cause grief.

You would also need to work around a way to override the choices made by dependencies, eg. to rebuild with security or bug fixes, without requiring me to fork everything.

Re: Go += Package Versioning

#193

Post by pmarreck »

peterevans wrote: post #171
pmarreck wrote: post #168

Earlier quoted context omitted.

How would it be possible to guarantee reproducible/deterministic compatibility without using something like the hash of the entire library as an implicit "version"? (say, the git SHA) I believe that the Nix OS does something along these lines to guarantee deterministic builds. A number of times in at least a couple of languages, I've seen a library keep the same (exact) version number but make some small "bugfix" cha…

The proposal doesn't seek to guarantee reproducible builds; it merely seeks to enable them, through the methods they outline. If you did want to guarantee reproducible builds with SHA-1 hashes, one way would be to introduce those into the .mod files they outlined. But that'd be clunky; it's much easier to reason about a version number than it is a digest hash. Another method would be to introduce a lock file where th…

What are the criticisms of lockfiles? I've used lockfiles more or less successfully in Rails, Elixir, and of late, Node. I thought it was a proven (if imperfect) idea...

Re: Go += Package Versioning

#194

Post by joeshaw »

marcrosoft wrote: post #55

Please don't remove vendoring...

I'm fine with vendoring going away as long as we can check these new modules into version control and have vgo use them. Adding a caching proxy adds too much infrastructure for most folks, and without it you can't have reproducible and available builds.

I like Go's distributed package imports (even if everyone just uses Github), but it means you're distributing single points of failure all over the place. Vendoring (whether source or modules) solves this problem.

Re: Go += Package Versioning

#195

Post by hota_mazi »

LaGrange wrote: post #174
hota_mazi wrote: post #95

Earlier quoted context omitted.

Or adopt a system with a sane versioning system and immutable artifacts, e.g. Maven Central.

First things first, Maven's config files are an elaborate form of torture. Second, there is no such thing as remote artifacts that are immutable. Check your shit into your git repository. And managing that becomes a problem, you've already done goofed up and have too many dependencies, and it's unlikely there's _anything_ reproducible about your software.

> Second, there is no such thing as remote artifacts that are immutable.

And yet, that's exactly what Maven Central artifacts are.

And why Maven Central is rock solid for reproducible builds while supporting versioning.

Re: Go += Package Versioning

#196

Post by peterevans »

pmarreck wrote: post #193
peterevans wrote: post #171

Earlier quoted context omitted.

The proposal doesn't seek to guarantee reproducible builds; it merely seeks to enable them, through the methods they outline. If you did want to guarantee reproducible builds with SHA-1 hashes, one way would be to introduce those into the .mod files they outlined. But that'd be clunky; it's much easier to reason about a version number than it is a digest hash. Another method would be to introduce a lock file where th…

What are the criticisms of lockfiles? I've used lockfiles more or less successfully in Rails, Elixir, and of late, Node. I thought it was a proven (if imperfect) idea...

I should note I am fine with lockfiles myself, so I can only speculate as to what RSC/others feel. It is fair to say that lockfiles are not Go—they would be another set of syntax that has nothing else to do with the language aside from their use in package management. So one might argue that it would be desirable to have a solution to Go's package management that was achieved using Go, which are what the module files are written in.

Re: Go += Package Versioning

#197

Post by coldtea »

steveklabnik wrote: post #69
coldtea wrote: post #60

Earlier quoted context omitted.

> I'm going to comment mostly on the parts of the proposal that I think are wrong, but don't take this to be an overall negative response. I'm excited to see smart folks working on this, and package management is a really hard problem. And as usual Golang ignores progress in the area by package managers such as npm, cargo, et al for what seems like a half-hearted solution. Issues I see: the introduction of modules on…

Having had lengthy conversations with Sam Boyer on this topic, I know that at least he deeply knows how these systems work. So it hasn't felt like "ignored" to me, at least, as an outside observer.

Perhaps not ignored in the "didn't know about them sense", but in the "nevertheless went ahead and did its own thing".

Re: Go += Package Versioning

#198

Post by munificent »

Groxx wrote: post #162
munificent wrote: post #153

Earlier quoted context omitted.

What happens if: 1. I depend on foo with constraint ">1.5.0". The current minimum version of foo that meets that is 1.7.0. 2. Later, foo 1.6.0 is published. 3. I run go get. If I understand the proposal correctly, that go get will now spontaneously downgrade me to foo 1.6.0. That defies the claim that builds are always reproducible.

So, I think you're right... but this is only a flaw if you as a user specify a lower bound that does not exist. The tool won't do this. And it can be prevented by disallowing referring to versions that don't exist. It's entirely valid (and interesting! I hadn't thought of this one), but I'm not sure if this would happen even once IRL, except for people trying to break the system. Which can be fun, but isn't a risk .

My experience from maintainer a package manager and trying to keep the ecosystem healthy — which mirrors my experience on lots of other systems with many users — is that anything your system allows people to do will be done at some point.

Re: Go += Package Versioning

#199

Post by jacobvosmaer »

ithkuil wrote: post #179
XorNot wrote: post #130

Earlier quoted context omitted.

I'm not really enthused by efforts to bring in the same packaging constructs as other languages, which all have the effect of making time-to-compile after git clone longer. Frankly I think we tend to conflate two separate but related tasks in these discussions: communicating updates, and distributing dependencies. vendor/ folders are a totally fine distribution system - optimal even. Time-to-compile is 0 because you…

Yeah, I'm also a bit concerned with vendoring going away. The proposed solution to preserving upstream is imho very elegant (caching proxies) and scales better than vendoring, but it requires a bit more infrastructure. Perhaps vgo could be taught to look in a local directory for an exploded content of what logically is the upstream archive (and that local directory could be checked in git, or preserved as a cache by…

I think vendoring is very useful and it would be a step back if it becomes harder.

Caching proxies for zip downloads sounds nice, but it's more than just "a bit more infrastructure". I think it would be a huge burden to package publishers if each of them has to manage their own dependency zip mirror as a separate piece of infrastructure. You need version control anyway; checking your dependencies into that same version control does not require a new piece of infrastructure.

Coming from Ruby, where rubygems.org is a very painful point of failure, in my eyes the fact that Go dependencies are not a separate download is a big plus.

In fact without a single blessed dependency repository such as rubygems.org, in the Go case you have as many points of failure at build time as there are different code hosting sites in your dependency graph.

Re: Go += Package Versioning

#200

Post by [deleted] »

munificent wrote: post #154
binarycrusader wrote: post #57

Earlier quoted context omitted.

One potential problem with dependency resolution that focuses solely on minimal or maximal-bound resolution is that it usually ignores the untested version combination problems. That is, a given version that satisfies a version bound may not have necessarily been tested with all of the different combinations of versions of components it is used with. It will be interesting to see how minimal version selection interac…

As far as I know, that problem is effectively unsolvable. For most real-world-sized dependency graphs, the set of all valid combinations of dependency versions is heat-death-of-the-universe huge.

[deleted]
Post reply on HN