Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go += Package Versioning

research.swtch.com

Post reply on HN
Print view View original

201–210 of 213 posts

Re: Go += Package Versioning

#201

Post by [deleted] »

munificent wrote: post #154
binarycrusader wrote: post #57

Earlier quoted context omitted.

One potential problem with dependency resolution that focuses solely on minimal or maximal-bound resolution is that it usually ignores the untested version combination problems. That is, a given version that satisfies a version bound may not have necessarily been tested with all of the different combinations of versions of components it is used with. It will be interesting to see how minimal version selection interac…

As far as I know, that problem is effectively unsolvable. For most real-world-sized dependency graphs, the set of all valid combinations of dependency versions is heat-death-of-the-universe huge.

[deleted]

Re: Go += Package Versioning

#202

Post by binarycrusader »

munificent wrote: post #154
binarycrusader wrote: post #57

Earlier quoted context omitted.

One potential problem with dependency resolution that focuses solely on minimal or maximal-bound resolution is that it usually ignores the untested version combination problems. That is, a given version that satisfies a version bound may not have necessarily been tested with all of the different combinations of versions of components it is used with. It will be interesting to see how minimal version selection interac…

As far as I know, that problem is effectively unsolvable. For most real-world-sized dependency graphs, the set of all valid combinations of dependency versions is heat-death-of-the-universe huge.

The way the system I worked on "solved it" was to provide another layer of constraints called "incorporations" that effectively constrained the allowed versions used to resolve dependencies to a version that was delivered for a given OS build. Administrators could selectively disable those constraints for some packages, at which point resolution was solely based on the "standard" dependencies. But by default, dependencies would "resolve as expected".

The "incorporate" dependencies are described here: https://docs.oracle.com/cd/E26502_01/html/E21383/dependtypes...

Re: Go += Package Versioning

#203

Post by ithkuil »

jacobvosmaer wrote: post #199
ithkuil wrote: post #179

Earlier quoted context omitted.

Yeah, I'm also a bit concerned with vendoring going away. The proposed solution to preserving upstream is imho very elegant (caching proxies) and scales better than vendoring, but it requires a bit more infrastructure. Perhaps vgo could be taught to look in a local directory for an exploded content of what logically is the upstream archive (and that local directory could be checked in git, or preserved as a cache by…

I think vendoring is very useful and it would be a step back if it becomes harder. Caching proxies for zip downloads sounds nice, but it's more than just "a bit more infrastructure". I think it would be a huge burden to package publishers if each of them has to manage their own dependency zip mirror as a separate piece of infrastructure. You need version control anyway; checking your dependencies into that same versi…

I don't think package publishers are the ones that need to manage the caching proxies: everyone who wants stable builds that don't break when some upstream deletes an old version of a package (or a networking error) needs a proxying cache.

Vendoring turned your git repo as your poor man's proxying cache. It also made some people unhappy. In my current company we use phabricator for code reviews and it doesn't work well if the commit size is bigger than some threshold.

I love to have the option of not checking in dependencies. I'm not sure this option has to be forced on everybody though.

Re: Go += Package Versioning

#204

Post by Groxx »

munificent wrote: post #198
Groxx wrote: post #162

Earlier quoted context omitted.

So, I think you're right... but this is only a flaw if you as a user specify a lower bound that does not exist. The tool won't do this. And it can be prevented by disallowing referring to versions that don't exist. It's entirely valid (and interesting! I hadn't thought of this one), but I'm not sure if this would happen even once IRL, except for people trying to break the system. Which can be fun, but isn't a risk .

My experience from maintainer a package manager and trying to keep the ecosystem healthy — which mirrors my experience on lots of other systems with many users — is that anything your system allows people to do will be done at some point.

heh, good point :)

as always, of course there's a relevant xkcd: https://xkcd.com/1172/

Re: Go += Package Versioning

#205

Post by Groxx »

neoasterisk wrote: post #178
Groxx wrote: post #164

Earlier quoted context omitted.

This kind of attitude around package management and especially dependency injection have been major turn-offs for me as well. More than a sufficient number of the community has been emitting a high-pressure stream of NIH combined with ignorance of existing solutions, and it has been crippling the community for years.

Dependency injection is very commonly used in the Go ecosystem. Do not confuse resistance about unidiomatic DI frameworks with resistance about DI itself.

Which ones are common? I've been hoping this would be the case, but I've been unable to find evidence for it.

e.g. the most-popular one I can see so far is Facebook's "inject", which has a remarkably small 841 stars: https://github.com/facebookgo/inject

Even Glide, Godep, and Dep have several thousand. And those can barely claim "most", much less "very common".

Re: Go += Package Versioning

#206

Post by tatersolid »

peterevans wrote: post #196
pmarreck wrote: post #193

Earlier quoted context omitted.

What are the criticisms of lockfiles? I've used lockfiles more or less successfully in Rails, Elixir, and of late, Node. I thought it was a proven (if imperfect) idea...

I should note I am fine with lockfiles myself, so I can only speculate as to what RSC/others feel. It is fair to say that lockfiles are not Go—they would be another set of syntax that has nothing else to do with the language aside from their use in package management. So one might argue that it would be desirable to have a solution to Go's package management that was achieved using Go, which are what the module files…

So make the lock file a simple go program that just returns an array?

This approach actually adds tons of flexibility... not sure it’s needed, but you could return different lock data based on whatever logic you needed

Re: Go += Package Versioning

#207

Post by neoasterisk »

Groxx wrote: post #205
neoasterisk wrote: post #178

Earlier quoted context omitted.

Dependency injection is very commonly used in the Go ecosystem. Do not confuse resistance about unidiomatic DI frameworks with resistance about DI itself.

Which ones are common? I've been hoping this would be the case, but I've been unable to find evidence for it. e.g. the most-popular one I can see so far is Facebook's "inject", which has a remarkably small 841 stars: https://github.com/facebookgo/inject Even Glide, Godep, and Dep have several thousand. And those can barely claim "most", much less "very common".

What do you mean which ones? I am talking about dependency injection as a technique/pattern. In Go you can just do it without importing anything.

Please read:

https://medium.com/@benbjohnson/standard-package-layout-7cdb...

(I've linked to the exact point in the article where it talks about DI but you might want to read the rest as well.)

Re: Go += Package Versioning

#208

Post by tatersolid »

sseth wrote: post #167
baq wrote: post #61

Earlier quoted context omitted.

Lock files are best of both worlds: you specify the latest at check in time and then freeze whatever was picked. No need to reinvent the world.

The picture becomes more complicated when you consider a hierarchy of packages, because presumably the packages you depend on would have their own lock files, which represent the versions those packages have been tested with. npm will pick the latest version of the dependencies compatible with the package.json of the packages (package-lock.json, is not published as part of the package). This means that with npm we ma…

This proposed “minimum version” behavior will effectively prevent “automated” security updates, which is what most reasonable people expect from their package manager.

Consider all tens of thousands of CVE bugs found in widely used image, video, XML, etc. libraries. Even “updated” software is often compromised via outdated libraries.

With a “min version” approach, none of those will get patched without explicit action by a developer, who won’t do that because he doesn’t even know about the bug unless he reads the thousands of messages each day on bugtrak.

If anything, history has shown that we should be focusing package management on getting security fixes to the the end user as soon as possible.

This proposal is bad for security.

Re: Go += Package Versioning

#209

Post by sseth »

tatersolid wrote: post #208
sseth wrote: post #167

Earlier quoted context omitted.

The picture becomes more complicated when you consider a hierarchy of packages, because presumably the packages you depend on would have their own lock files, which represent the versions those packages have been tested with. npm will pick the latest version of the dependencies compatible with the package.json of the packages (package-lock.json, is not published as part of the package). This means that with npm we ma…

This proposed “minimum version” behavior will effectively prevent “automated” security updates, which is what most reasonable people expect from their package manager. Consider all tens of thousands of CVE bugs found in widely used image, video, XML, etc. libraries. Even “updated” software is often compromised via outdated libraries. With a “min version” approach, none of those will get patched without explicit actio…

Note that even currently, lockfiles have the effect of locking you to an old version of the software. You could choose to automatically update all dependencies to latest in CI, but then by that token you can run vgo get -u, to get the latest versions here as well.

One of the issues which the vgo developers point out is that the "latest" behaviour has the perverse effect that a module A may declare a dependency of version 1.1 of a module B, and may have never been even tested on that version because the package manager always picked the latest.

In some sense, the vgo approach is saying that across hierarchy of module owners, each one should keep upgrading their manifest to the latest versions of their direct dependencies, and ensure that things keep working, rather than relying on the topmost module to pull in the latest of the entire tree of dependencies. This seems to be good for the entire ecosystem.

Re: Go += Package Versioning

#210

Post by Groxx »

neoasterisk wrote: post #207
Groxx wrote: post #205

Earlier quoted context omitted.

Which ones are common? I've been hoping this would be the case, but I've been unable to find evidence for it. e.g. the most-popular one I can see so far is Facebook's "inject", which has a remarkably small 841 stars: https://github.com/facebookgo/inject Even Glide, Godep, and Dep have several thousand. And those can barely claim "most", much less "very common".

What do you mean which ones? I am talking about dependency injection as a technique/pattern. In Go you can just do it without importing anything. Please read: https://medium.com/@benbjohnson/standard-package-layout-7cdb... (I've linked to the exact point in the article where it talks about DI but you might want to read the rest as well.)

Yeah, it's widespread as a hand-coded thing. It has to be - you essentially have no choice in Go for even basic things like tests.

But that just means that everyone rolls their own thing by hand, with widely-varying feature-sets, method names, bugs, etc. DI frameworks are very much nicer for interop between a bunch of libs.

Post reply on HN