Live data from Hacker News

Ditch your version manager

juliu.is

81–90 of 155 posts

Re: Ditch your version manager

#81

Managing dependencies is a big problem, and I feel like we've given up on solving it directly, and instead built workarounds. If we question our assumptions, the first question is: Why do we need multiple Ruby versions at all? Why isn't the latest version of Ruby sufficient? Well, obviously, Ruby's behavior has changed over time. But why isn't it backward-compatible? Why can't I just run Ruby 3.0 with a flag that tel…

> But why isn't it backward-compatible?

It is, mostly. 1.8->1.9 was famously a big change, but breaking changes since have been comparatively minor.

IMX the most common cause for gem breakage is simply rot - old gem, maintainer lost interest, a tiny change required (often due to an open-ended dependency on another gem) but there's no one to do it. Unfortunately, adopting an abandoned gem is non-trivial if the original maintainer doesn't respond, and a lot of gems were written in the Rails goldrush but have fallen into abeyance since.

Re: Ditch your version manager

#82

> My ideal dependency manager would allow me to specify each and every dependency that is required to work on my projects. It should be easily reproducible, declarative and easy to upgrade. To me it feels like what's left out here is the fact that you need the amount of dependencies that you have to be reasonable in the first place, otherwise no single piece of software is going to help you all that much. For example…

I don't have an issue with the size of a project. On the other hand, I worry about a large number of dependencies and the need to understand the security and compatibility models of their respective authors.

I'm fine to depend on, say, JVM or Qt - large projects. Not with thousands of small packages developed by thousands of independent developers.

Re: Ditch your version manager

#83
post #59

Earlier quoted context omitted.

The fact you need a version manager on top of your new, pristine and reproducible version manager is a bit concerning. Can’t whatever shortcomings exist be addressed in nix itself?

it's not on top of Nix itself but (about to be) a part of it. according to the NixOS Wiki [1] Flakes is an upcoming feature of the base package manager. it's not part of the stable channel yet, and "proper" documentation only currently exists in the unstable manual [2]. [1] https://nixos.wiki/wiki/Flakes [2] https://nixos.org/manual/nix/unstable/command-ref/new-cli/ni...

GP is kind of right though. Flakes are effectively packages for Nix expressions. Even priginal Flakes RFC calls this out.

But the difference is, as you mentioned, flakes are integrated within Nix, and provide better ergonomics.

Re: Ditch your version manager

#84
post #64

Earlier quoted context omitted.

> You can't use multiple versions of a dependency on a single system without completely separating both the build-time and run-time environments Sure you can. It's what version numbers in library file names are for. Works for executable names, too. Until Python 2.7 died, you commonly had 2.7 and 3.x installed at the same time. But maybe you meant something different?

It's impossible at the package manager level. First, install python 2 and python 3. Next, install a script that calls "#!/usr/bin/python". How do you make the script work, without modifying the script , so that it will work with the specific version of Python it was designed against? You can't. You have to either modify the script to point at the specific executable it was designed for, or you have to modify the envi…

The cases that you're raising are all ones that the nix ecosystem has had to find solutions for, but they have generally done so.

The first thing to note is that it's generally discouraged to use full paths like the one you gave for python in shebangs in loose scripts. Instead, it is encouraged to write e.g. /usr/bin/env python. In this case, the python from the current environment will be picked up "impurely" when the script is run.

That said, it's pretty heavily encouraged to package anything that will be used seriously, since this is very easy with nix and gives access to better tools.

For example, there is built in automation for recognizing shell shebangs and rewriting them, based on the dependencies expressed in the build description.

As far as your case of programs shelling out to other programs: that can either happen "impurely" (which would work like that you describe, including the potential drawbacks) or "purely" (where a specific version is captured). In order to attain "purity", nix has automation for generating wrapper scripts which manipulate the PATH before delegating to some underlying executable.

Re: Ditch your version manager

#85

Earlier quoted context omitted.

> Other tool require lots of thought and carefull execution for what Nix gives you for free. How do I pin ruby version to 2.6.3 in Nix? This is given to me for free in other tools. As are reproducible builds because all modern package/dependency managers lock versions.

It is not the same level of reproducibility like Nix and Bazel provides. You can pin ruby to exact version you want with https://nixos.wiki/wiki/Overlays

> It is not the same level of reproducibility like Nix and Bazel provides.

What is "the same level"?

> You can pin ruby to exact version you want with https://nixos.wiki/wiki/Overlays

That page says: "Overlays provide a method to extend and change nixpkgs"

I don't want to change something. I want to simply pin a version. This is neither simple, nor scalable, nor maintainable (also here: https://news.ycombinator.com/item?id=28591202):

  Overriding a version

  self: super:
  {
    sl = super.sl.overrideAttrs (old: {
      src = super.fetchFromGitHub {
        owner = "mtoyoda";
        repo = "sl";
        rev = "923e7d7ebc5c1f009755bdeb789ac25658ccce03";
        # If you don't know the hash, the first time, set:
        # sha256 = "0000000000000000000000000000000000000000000000000000";
        # then nix will fail the build with such an error message:
        # hash mismatch in fixed-output derivation '/nix/store/m1ga09c0z1a6n7rj8ky3s31dpgalsn0n-source':
        # wanted: sha256:0000000000000000000000000000000000000000000000000000
        # got:    sha256:173gxk0ymiw94glyjzjizp8bv8g72gwkjhacigd1an09jshdrjb4
        sha256 = "173gxk0ymiw94glyjzjizp8bv8g72gwkjhacigd1an09jshdrjb4";
      };
    });
  }

Re: Ditch your version manager

#86

First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager. Second, to use this package manager, you first need to install direnv . How do you install it? with brew , a different package manager. Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first. Fourth, finally…

Imagine having a whole language, but still running shell scripts to create folders and replace text. I mean even rpm can create folders declaratively.

Re: Ditch your version manager

#87

First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager. Second, to use this package manager, you first need to install direnv . How do you install it? with brew , a different package manager. Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first. Fourth, finally…

> First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager.

This is not true. The defining point about Nix is that it allows to define all dependencies explicitly. Every derivation in it is built inside of a sandbox with no access to network and restricted filesystem. This ensures that nothing is accidentally missed. Package management is just one of features.

> Second, to use this package manager, you first need to install direnv. How do you install it? with brew, a different package manager.

direnv is not needed to use Nix, it's just a nice add-on. It's like you don't need Ranger to navigate in a file system, but it is nice. Besides, you can install direnv through nix, I use it this way. Even nix is installed by itself as well.

> Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first.

Well, yes you do, but language is actually quite simple it's the nixpkgs (repo containing all packages) that's quite complex. Nix's purely functional, lazily evaluated properties exactly fit as a language describing dependencies.

> Fourth, finally, we get to install a simple Unix program, that any package manager could have provided.

If you were trying to install a package, you don't need to learn anything, you just install it (for example to install ranger you just run nix-env -iA nixpkgs.ranger). Nix language is only needed if you want to create a new package.

> For the fifth trick, freezing dependencies, you first have to have all the correct versions of all the dependencies to do what you want. How do you establish the right ones? Manually. Just like with any other package manager that builds apps against a specific tree of build deps. "Reproducibility!" Because no other package manager could possibly download the same tarballs and run the same commands. Must be all that functional programming magic.

> And sixth, the idea that this will work forever. Right. Because any other distro and package manager could not possibly work again in the future, with a mirror of all the packages in a release. That's only for super cool futuristic package managers. No way to reproduce the same packages in the future with old package managers.

> Look, Nixians, this is all old hat. Every decent package manager can do all these things, just in a less complicated way. Modern systems use containers, and they don't need Nix for that. Nix is basically just the new Gentoo: a distro for hobbyists who tell themselves they're advanced while all the professionals use something else.

Nix delivers what docker promised and ultimately failed to deliver. Docker promised to reproduce developer's environment to production. What it did was to zip developer's computer and deploy it. When docker got adopted, relying on images was impractical, so for deployment Dockerfile was used, but that file is not much different than a shell script.

Nix instead describes the entire dependency tree down to libc. Because the starting state and all dependencies are known it can always create the same result, that's the biggest selling point of Nix to me.

Re: Ditch your version manager

#88
post #31

Everything to install with Nix must be packaged "the Nix way", I assume? How is the breadth and quality of Nix packages? Just for fun I checked the kiwi image builder, from the openSUSE project, which I recently had some versioning problems with, and it was not there.

Nixpkgs is quite good. It has the package I'm looking for most of the time.

According to Repology[1], it's got more packaged projects than any other package repository, and more packages that are up to date than any other repository. It's also pretty on top of CVEs, with only 0.38% of packages having potential vulnerabilities.

Also check out Repology's size/freshness graph[2]

[1] https://repology.org [2] https://repology.org/repositories/graphs

Re: Ditch your version manager

#90
post #80

First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager. Second, to use this package manager, you first need to install direnv . How do you install it? with brew , a different package manager. Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first. Fourth, finally…

First, dependency manager vs package manager. Potayto potatoh. Second. The reason I recommend direnv is because of ergonomics, so you don't have to remember to reload your shell. Most current version managers either do it themselves through shell hooks or ask you to install direnv. Point taken on using brew: I've changed it to use nix-env. Third. Yep. Sort of like having to learn English before you can learn programm…

For the docker issues, that's just one of those quirks you learn and then in the future you make your containers reproducible. Or don't, and just store them after they're built the first time in an artifact repository. Any builds of your app that use a container need to be versioned to prove they've been tested successfully, anyway, and you want to keep them around to do fast rollbacks without having to kick off a build just to roll back. So you don't need reproducibility for reliability. It's mostly used for security: build on two siloed systems, compare the results.

Version-pinning in general is to prevent silent upgrades that break your builds. But you also don't want to version-pin everything, as you want security patches to get automatically applied with regularly-recurring builds. So in practice, you stick with a major branch of something, silently accept upgraded packages or minor version bumps, and have an automated system flag if you aren't running the latest security packages.

All of that works fine with Docker and Apt. More often than breakage due to silent upgrade is the security patch problem, where somebody pinned to a version 4 years ago and has never patched, and it's been so long that all the downstream effects of patching force you to do an entire lift-and-shift to new everything. Gradual continuous updates (stick to a major version, run tests) is the best middle ground. Modern projects with Docker containers have standardized their tagging around this.

Post reply on HN