Live data from Hacker News

Go += Package Versioning

research.swtch.com

161–170 of 213 posts

Re: Go += Package Versioning

#161
post #32

Glad to see this! I'm still digesting the details, but to comment on https://research.swtch.com/cargo-newest.html Cargo does not use the latest version. Saying toml = "0.4.1" is the same as saying toml = "^0.4.1" NOT saying toml = "=0.4.1" which is what rsc would guess. This decision was made because ^ is the most reasonable default; over-use of `=` destroy's Cargo's ability to help you upgrade, given that you basica…

Hi. As I said in that page, I do know that cargo is working as designed, and that "0.4.1" is the same as "^0.4.1". My point is maybe a little more subtle, that cargo takes the newest allowed under the constraints, so given "^0.4.1", it has a choice between 0.4.1, 0.4.2, 0.4.3, 0.4.4, and 0.4.5, and it takes the last. When you're adding a new direct dependency, taking the latest is almost certainly right. But when you…

> I think taking the last is problematic: it seems much safer to me to take the one closest to what the author of that code tested with.

If someone doesn't respect semantic versioning the difference of one minor or few minor versions is irrelevant.

> For what it's worth, I think I do understand what cargo is doing, and why, and I completely respect that approach.

I think understanding SemVer would help a long way.

Re: Go += Package Versioning

#162

Earlier quoted context omitted.

You CAN ship v1.2.1 after v1.3.0 is live. I have tested this with the vgo prototype and it works fine (see github.com/joncalhoun/vgo_main): $ vgo list -m -u MODULE VERSION LATEST github.com/joncalhoun/vgo_main - - github.com/joncalhoun/vgo_demo v1.0.1 (2018-02-20 18:26) v1.1.0 (2018-02-20 18:25) Notice that v1.0.1 was released AFTER v1.1.0 What the minimum version is doing is giving our code a way to automatically re…

What happens if: 1. I depend on foo with constraint ">1.5.0". The current minimum version of foo that meets that is 1.7.0. 2. Later, foo 1.6.0 is published. 3. I run go get. If I understand the proposal correctly, that go get will now spontaneously downgrade me to foo 1.6.0. That defies the claim that builds are always reproducible.

So, I think you're right... but this is only a flaw if you as a user specify a lower bound that does not exist. The tool won't do this. And it can be prevented by disallowing referring to versions that don't exist.

It's entirely valid (and interesting! I hadn't thought of this one), but I'm not sure if this would happen even once IRL, except for people trying to break the system. Which can be fun, but isn't a risk.

Re: Go += Package Versioning

#163
post #14

Earlier quoted context omitted.

You say that because you have infinite filesystem space. I don't.

Though GOPATH avoids duplication, it's entirely possible to avoid duplication without GOPATH.

And downloading the full Git repository for every `go get` is frequently a far greater waste than even storing copies of all the versions you're currently using in all projects you work on.

GOPATH is unambiguously terrible.

Re: Go += Package Versioning

#164
post #10

Earlier quoted context omitted.

I've been programming in Go for a couple of years now and found that this need to use "hacks ... to avoid it" is totally naive and irrational. Agreed, GOPATH is awkward--at first. Because it's different from what most programmers are used to. But once it's adopted it actually makes a lot of sense, and looking back it seems bizarre to ever have such a strong desire to avoid it at all costs--and the costs are high. The…

I should add that another big part of the reason I've been avoiding the language for years is the cult of Golang telling me my complaints are "totally naive and irrational".

This kind of attitude around package management and especially dependency injection have been major turn-offs for me as well. More than a sufficient number of the community has been emitting a high-pressure stream of NIH combined with ignorance of existing solutions, and it has been crippling the community for years.

Re: Go += Package Versioning

#165

Earlier quoted context omitted.

You CAN ship v1.2.1 after v1.3.0 is live. I have tested this with the vgo prototype and it works fine (see github.com/joncalhoun/vgo_main): $ vgo list -m -u MODULE VERSION LATEST github.com/joncalhoun/vgo_main - - github.com/joncalhoun/vgo_demo v1.0.1 (2018-02-20 18:26) v1.1.0 (2018-02-20 18:25) Notice that v1.0.1 was released AFTER v1.1.0 What the minimum version is doing is giving our code a way to automatically re…

What happens if: 1. I depend on foo with constraint ">1.5.0". The current minimum version of foo that meets that is 1.7.0. 2. Later, foo 1.6.0 is published. 3. I run go get. If I understand the proposal correctly, that go get will now spontaneously downgrade me to foo 1.6.0. That defies the claim that builds are always reproducible.

[deleted]

Re: Go += Package Versioning

#166
post #128

As I look this over a couple things really jump out at me... 1. VCS tags are mutable. That's lock files store revision ids. Go is being used to build immutable infrastructure but the proposed package management system uses mutable versions. 2. The proposal is less featureful that dep, npm/yarn in JS, composer in PHP, maven, crates for rust, and others. I wonder how people will react to that.

Mutable tags are my primary concern here, yeah. It seems pretty mitigate-able by using that (wonderful IMO[1]) `v1.2.3-date-sha` syntax though - it's just not human editable[2].

[1]: it fixes so many readability problems with SHA-pinned lock files, easily shows downgrades in `diff` output, and `sort` likely produces the exact result you wanted.

[2]: which may not be a problem, since you could in theory just re-run the tool to fix it when you enter "v1.2.3" by hand.

Re: Go += Package Versioning

#167
post #61

Earlier quoted context omitted.

This is horrible advice. I work in a lot of languages (including Go), which gives me some perspective. On the extreme ends of this issue we have Maven, which specifies hard version numbers (fuzzing is possible but culturally taboo) and NPM, which only recently acquired the ability to lock version numbers and still has a strong culture of "always grab latest". The Maven approach is unquestionably superior; code always…

Lock files are best of both worlds: you specify the latest at check in time and then freeze whatever was picked. No need to reinvent the world.

The picture becomes more complicated when you consider a hierarchy of packages, because presumably the packages you depend on would have their own lock files, which represent the versions those packages have been tested with.

npm will pick the latest version of the dependencies compatible with the package.json of the packages (package-lock.json, is not published as part of the package). This means that with npm we may end up using a version of a transitive dependency which is much later than than the one that your direct dependency was tested with.

The proposal for vgo will take the minimum version compatible with the packages, and hence pick a version which is closer to the actually tested one.

Re: Go += Package Versioning

#168

I'm going to comment mostly on the parts of the proposal that I think are wrong, but don't take this to be an overall negative response. I'm excited to see smart folks working on this, and package management is a really hard problem. There are no silver bullets to code reuse. Context for those who don't know: I along with Natalie Weizenbaum wrote pub[1], the package manager used for Dart. > Instead of concluding from…

How would it be possible to guarantee reproducible/deterministic compatibility without using something like the hash of the entire library as an implicit "version"? (say, the git SHA)

I believe that the Nix OS does something along these lines to guarantee deterministic builds.

A number of times in at least a couple of languages, I've seen a library keep the same (exact) version number but make some small "bugfix" change that ended up breaking things. Often, nothing stops someone from doing that.

Re: Go += Package Versioning

#169
This is a well reasoned proposal. I've got a couple qualms.

Let's say you care about strict build reproducibility. You want some form of manifest + lock file that defines exactly what versions of dependencies your package expects. Great; more power to you.

The only reason you'd want that is if you aren't storing your dependencies with your source code in source control. Otherwise, what's the point? If you were vendoring and committing it, you already have strict reproducibility, and you have the "supported versions" defined in the git repositories that come alongside your dependencies.

So, adding this manifest+lock file allows you to get away with not vendoring+committing. Awesome. You've gained reproducibility, but certainly not strict reproducibility.

Dependency package maintainers can rewrite git history. They can force push. They can push breaking changes with minor versions. They can delete their repositories. All of these things have already happened in NPM, either maliciously or accidentally; why do we think they wouldn't happen with Go?

If you want strict reproducibility but are expecting it with just a manifest+lock file, without dependency vendoring+committing, you're not getting it. Full stop.

So, really, by adding a manifest+lock file, you're adding a "third level" of reproducibility (#2 on this list).

1. Low Reproducibility: Pull HEAD on build.

2. Partial Reproducibility: Use tooling to pin to a git commitish, pull this on build.

3a. Full Reproducibility (Easy+Dirty): Vendor+Commit all dependencies.

3b. Full Reproducibility (Hard+Clean): Mirror your dependencies into a new self-controlled git repo.

I am struggling to think of a solid use case that would find real value in #2. I have no doubt that people think it would be valuable, but then the moment the left-pad author deletes his repository, you're going to second guess yourself. You didn't want #2; you wanted #3 and settled for #2 because #3 was too hard or dirty and you convinced yourself that adding tooling and version numbers was keeping you safe. Because semver, right?

Moreover, this comes at the cost of complexity, which is strictly against Go's core doctrine.

Moreover, a #2 solution looks strikingly similar to gopkg.in. Instead of referencing Repository:HEAD we reference Repository:Commitish. The main difference is that gopkg.in is an external service whereas this would be controlled tooling. But by choosing #2 you're already exposing yourself to bad actors, accidents, github going down, all of the above. So you're willing to accept that exposure, but aren't willing to add that little extra exposure of one more service?

I agree that the problem today, even with dep, isn't perfect. But I still strongly believe that the answer lies somewhere in the ideas we already have, not by importing ideas from NPM. We need a #3 that is Easy and Clean, not a #2.

Re: Go += Package Versioning

#170
post #23

I have misgivings about all this version pinning. At first, it seems to make things easier. Programs don't break because some imported package changed. So it looks like a win. At first. Over time, though, version pinning builds up technical debt. You have software locked to old unmaintained versions of packages. If you later try to bring some package up to date, it can break the fragile lace of dependencies locked in…

This is horrible advice. I work in a lot of languages (including Go), which gives me some perspective. On the extreme ends of this issue we have Maven, which specifies hard version numbers (fuzzing is possible but culturally taboo) and NPM, which only recently acquired the ability to lock version numbers and still has a strong culture of "always grab latest". The Maven approach is unquestionably superior; code always…

Maven doesn't specify hard version numbers - the default meaning of "1.0" is >= 1.0, and some packages specify dependencies without a version at all. It's quite possible to get updates of transitive dependencies without realising it (especially since most tooling does not encourage pinning any dependency other than the superficial ones).
Post reply on HN