Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparison of Programming Language Package Managers

docs.google.com

Post reply on HN
Print view View original

51–60 of 98 posts

Re: Comparison of Programming Language Package Managers

#51

Post by surrealize »

knucklesandwich wrote: post #5

One of the things that I've found to be important for (and generally under-documented by) package managers is resolution strategy. pip, for instance, eagerly resolves the highest version of a dependency first found in the tree and doesn't backtrack AFAIK [1]. Cargo sounds as though it behaves similarly, but will actually backtrack [2]. I've also seen package managers like 0install that will use SAT solvers to solve d…

There was some discussion about potentially using a SAT solver for cargo in this issue:

https://github.com/rust-lang/cargo/issues/2064

and graydon wrote a POC of using the Z3 solver to find a solution where semver constraints were satisfied, optimizing for the latest versions within those constraints:

https://github.com/graydon/z3-rs/blob/master/tests/semver_te...

Re: Comparison of Programming Language Package Managers

#52

Post by halostatue »

jacques_chester wrote: post #35
halostatue wrote: post #22

At least three things wrong with respect to Bundler/RubyGems: 1. RubyGems is the package manager. Bundler is a meta-layer on top of that which does full dependency resolution to find the right version before installing. Bundler builds on the capabilities provided by RubyGems (and will be integrated into RubyGems in the future). 2. Bundler does support vendoring. It is widely discouraged, but I take advantage of it in…

> It is widely discouraged It is also widely encouraged. Some engineers: 1. Don't trust dependencies to always be there. 2. Write apps that need to work in disconnected environments -- ie with no internet connectivity. I work on the Cloud Foundry buildpacks team for Pivotal. Being able to stage and launch apps in a disconnected environment is A Big Deal for a lot of companies.

Absolutely. That’s sort of a fundamental reason why I wrote Cartage. I like some of what Capistrano does, but the general insistence on being able to `bundle install` on the target server is questionable, and the earlier practice for Rails apps of keeping the gems in Git resulted in other problems.

Heroku mostly does this right with its 'slug' system, and I made Cartage specifically to be able to make deployable packages that use dependencies. You have a build machine that’s connected to the network and can reach RubyGems (and, in the future with some plug-in work, other packaging systems—I expect to add Node, Elixir, and Lua soon) and that will vendor the resulting build and give you a tarball that is just installable with all of the dependencies of your package.

Re: Comparison of Programming Language Package Managers

#53

Post by halostatue »

alexginzburg wrote: post #42
halostatue wrote: post #21

Earlier quoted context omitted.

Because apt-get doesn’t exist on the system I use, and most system packagers can’t/won’t keep up with the rate of change of development, anyway. CPAN, RubyGems, etc. all exist because system packaging systems are (a) specific to a particular distribution, (b) generally many versions behind the development versions, (c) really hard to get into unless you are already a system packager for a particular distribution, and…

using a system package manager does not mean that one needs to use a distribution's (debian, ubuntu, redhat, gentoo, arch, etc) mirror to pull the package, it does mean that you use an OS provided tool to manage all software on your machine. - It does mean that the common libraries are installed once - conflicts between common libraries are more transparent - System tools make sure that package stays up-to date

Except that you haven’t solved most of the problems that I pointed out by saying “use a system package manager” (and have introduced several new ones):

a. I maintain ~13 Ruby gems. I can generate new versions and verify their install on any system that I can run Ruby on. Under “use a system package manager”, I now have to maintain at least four unique package management formats (deb, RPM, emerge, Pacman) just for Linux distributions and at least one for the BSDs. Thanks, you’ve just increased my workload at least five-fold, although I no longer have to worry about Windows or macOS users (wait, I’m a macOS user).

b. Sure I can use a non-distribution mirror (like a PPA), but now you have to have at least six of those mirrors (see point 1) and, again, too bad about Windows and macOS users since they obviously aren’t smart enough to use an OS that has an in-built packaging manager.

c. In truth, you haven’t increased my package load five-fold, you’ve increased it much more than that. I’ve followed instructions for making debs and RPMs…and the only thing that works for me is using FPM because the system package managers are so bloody complex and built for a single particular use-case.

d. You still don’t have a solution for Windows or macOS users because neither of those OSes have system package managers, and no neither ports nor homebrew counts. RubyGems, pip, cabal, npm, etc. may increase the cognitive load but they work for people who don’t use Ubuntu Zany Zebra.

You’ve also introduced a new problem:

e. OS package managers (except probably nix) do not solve the multiple-installed-version problem. RubyGems allows (even encourages) multiple versions of a gem to be installed and applications (through their Bundler Gemfile) can lock to a particular version. I have packages with 1.x, 2.x, and 3.x releases where all three versions are still in use despite some backwards incompatibilities (mime-types 2.x dropped support for Ruby 1.8, deprecated some methods, and changed the default data format and loading mechanism; 3.x dropped support for Ruby 1.9 and removed the deprecated methods, and changed the default data format).

This is not possible in an OS package manager without renaming the installed library in some way. This is mostly fine when you‘re installing compiled software…but it’s not even perfect there. I’ve been working with language-specific package managers for a long time, and I’ve been dealing with how distribution package managers break some of the packages that I work with for just as long. I generally prefer the mess we have now to waiting for the perfect system package manager, especially since there are systems which will never get them.

[fixed formatting]

Re: Comparison of Programming Language Package Managers

#54

Post by halostatue »

jacques_chester wrote: post #37
knucklesandwich wrote: post #27

Earlier quoted context omitted.

Global installs are the root of a lot of headaches when building software in the same way that global mutable state is the root of a lot of headaches when developing it. Nix may be the one system package manager that is the exception to this rule (I don't have experience with it, so I can't vouch towards that, but I hear great things). However apt-get is incredibly bad at getting you the dependencies you need for a r…

> Nix may be the one system package manager that is the exception to this rule Some peers of mine recently experimented with building a Nix buildpack and concluded that it won't work. Nix has hardcoded paths for its core directories, and won't work in a security-constrained container.

Sad face. Nix is the one package manager that looked portable enough and interesting enough to push as a “single” manager for pretty much any system.

Re: Comparison of Programming Language Package Managers

#55

Post by sivoais »

Perl's CPAN has tooling for diffing versions through the use of MetaCPAN (a top-notch site which every language should try to emulate). For example, here is a diff of the URI distribution:

https://metacpan.org/diff/file?target=ETHER%2FURI-1.71%2F&so...

This information is also available through an API for integration into command line tools.

Re: Comparison of Programming Language Package Managers

#56

Post by bluejekyll »

sdegutis wrote: post #28
bluejekyll wrote: post #24

Earlier quoted context omitted.

Although, the pony is still valid that every programming language has decided to implement its own pkg mgr. Like all things in languages, the world needs to be reimplemented. Which is too bad, but I also never see a great alternative. Eventually people want everything they work on to be in the same language, which of course has the advantage that anyone in that language can hack on it, but the disadvantage that we al…

You know, earlier today I was thinking, "wouldn't it be great if we all just used one programming language for everything?" And I tried to imagine which one it could be. And all had different strengths and weaknesses. C is great for raw speed, but ends up with very rigid programs, and is easy to shoot your foot in. Rust solves the second problem but not the first. Ruby solves both, but is much slower. LuaJIT solves a…

I disagree about Rust (I assume you mean rigidity) you just need to learn how to work with it's vtable's through &Traits and/or enums. In terms of speed and foot shots, it definitely beats C.

I think Rust is the greatest programming language of our generation.

But regardless (and I meant irony, not pony in the original post), not everyone will agree with me, so everyone will want their own language. What we need is a general purpose packaging tool (I like cargo, but apt-get could serve equally well) which could support any language.

Re: Comparison of Programming Language Package Managers

#57

Post by mfer »

knucklesandwich wrote: post #5

One of the things that I've found to be important for (and generally under-documented by) package managers is resolution strategy. pip, for instance, eagerly resolves the highest version of a dependency first found in the tree and doesn't backtrack AFAIK [1]. Cargo sounds as though it behaves similarly, but will actually backtrack [2]. I've also seen package managers like 0install that will use SAT solvers to solve d…

You are absolutely right about solver strategies. Most of the existing package managers in Go have no solver. It's all manual and typically the tip of master or latest commit.

Glide has a dumb solver in that it goes with the first best version it finds. It will tell you if there is a later conflict that you can manually resolve.

There is a package, discussed here called gps, that is being prepped to merge into Glide. This solver has a back-off strategy. It's loosely based on the solver in dart. The idea is to find the latest newest version that works for the complete tree.

We've been incrementally working to get a good solver and are almost there.

Re: Comparison of Programming Language Package Managers

#58

Post by nilkn »

moosingin3space wrote: post #45
nilkn wrote: post #4

This should probably use Stack for Haskell as well, which was designed to solve various common issues that arise when using cabal as a package manager.

Is stack really a package manager? It seems to me that it's more of a build manager, delegating package management to cabal/stackage.

I see your point, but I'm not sure if it matters. Stack might be built on top of cabal, but I never use cabal directly at all anymore, and stack cleanly addresses a lot of problems that you'll run into if you exclusively use cabal-install for package management. Stack actively installs packages and dependencies on your system.

Re: Comparison of Programming Language Package Managers

#59

Post by mfer »

sivoais wrote: post #55

Perl's CPAN has tooling for diffing versions through the use of MetaCPAN (a top-notch site which every language should try to emulate). For example, here is a diff of the URI distribution: https://metacpan.org/diff/file?target=ETHER%2FURI-1.71%2F&so... This information is also available through an API for integration into command line tools.

IIRC, and I could be wrong, CPAN was the first to go down the route that many modern toolchains now provide. We've looked at it.

In fact, the original creator of Glide (Go package manager) wrote about Perl and CPAN when talking about Go at http://technosophos.com/2015/09/02/dont-let-go-be-condemned-....

Re: Comparison of Programming Language Package Managers

#60

Post by mfer »

throwaway2016a wrote: post #30
jadell wrote: post #17

Earlier quoted context omitted.

You're not actually downloading the package from Packagist. It acts only as a metadata repository that points you to a source repository. For the others, I would assume they act as both a metadata and a source repository.

Thanks for the clarification. I'm shocked it is the only one like that.

It takes a bit of work to store versions and manage them as a central repo. Composer will fetch the sources, as a tarball from GitHub if possible, and cache them locally. They are skipping being the central store and deferring to root sources.

They removed a lot of complexity by doing that and I bet more than a few people didn't notice.

Post reply on HN