My First Impressions of Nix

> Part of the difficulty is it means different things to different people.

That's definitely how it seems to me. The pro-Nix stuff I see is generally about the theory much more than the practice. Which was also my experience with functional languages when their hype cycle was last on the rise.

On the one hand, that's fine. I like ideas, and I think taking an idea and running with it can be really interesting. You can clearly see that in history's various art movements, for example. On the other, for people who are just trying to get things done, it's often alienating and tedious, because the people in the grip of their Big Idea often seem heedless of other perspectives, and frequently can be quite evangelical about it.

Personally, my strategy with Nix, as with the various functional languages, is to keep a distance from it, waiting and seeing. Perhaps it will influence more mainstream projects, bringing the benefits to me without a lot of upheaval. Perhaps I'll have a project that really needs its particular benefits, and so I'll take on the cost of a paradigm switch. But in the meantime, I have stuff to do.

I think there's a bit of confusion caused by equating Nix "derivations" with "packages" of traditional package managers.

Nix mainly concerns itself with derivations [1]. They're build recipes for creating binary artifacts that are meant to be consumed by the Nix daemon. The Nix daemon instantiates derivations by building the artifact and storing it to a store path under /nix/store. Store paths are unique to each derivation.

When people say Nix is reproducible, they mean that derivations are reproducible [2]. This is because anything that might cause the build to change is captured as inputs to the derivation. Every input is explicitly specified by the author of the derivation. This means that when a dependency gets updated, the resulting derivation and store path would change. The new derivation might fail to build, but the old one would still continue to build regardless of how much time has passed since it was first built. So if a latest package in Nixpkgs is broken, you can always go back to a known good commit to get a working derivation while waiting for the package maintainer to fix it [3].

Traditional package managers don't have a concept of a derivation. Instead, they have packages. Those packages have no reproducibility whatsoever. Even if they built successfully in the past, they might not build today. That's because a traditional package is only identified by its name and version, as opposed to a Nix derivation which is identified by its content (= the build recipe) [4]. Traditional package managers see two incompatible builds with the same name and version as the same package, replaceable with each other. Worse, most package managers don't require versions to be specified as part of dependencies. Whether a package builds or not is then dependent on the current state of the central package repository. Again, this isn't the case with Nix derivations.

[1]: Internally, Nix doesn't even have the concept of a package. A package is a concept that we humans use to group related derivations together.

[2]: To be clear, derivations aren't bit-by-bit reproducible. For example, CPU caches would be observable during builds because in general, process sandboxes don't prevent hardware information leakage. However, it's reproducible in a practical sense because people would have to go out of their way to make software builds dependent on things like CPU state. People might do that as a joke, but not for any serious reason.

[3]: Ideally, tests and reviews should catch any breakage but sometimes it happens. Hence the rolling release branch is marked "unstable." Fortunately, it's also easy to apply fixes locally before they're available in Nixpkgs because Nix makes it straightforward to create a custom derivation by extending existing ones.

[4]: Not to be confused with content addressed derivations, which identifies derivations by the resulting binary artifact.

It seems to me that this "hash all the things" setup seems absolutely made for IPFS integration, which AIUI would mean that everyone who chooses to seed those nix stores would incrementally reduce the bandwidth burden upon the "main" nix store. I'm not sure if nix hashes take the $(uname -sm) into consideration, so that could be a hurdle, too

I can very easily imagine that it would need to be opt-in, to keep security peeps from losing their minds, but my suspicion is that a non-zero number of actual nix users would choose to turn it on in support of the community

The goal of a downstream Linux distribution is never to reproduce whatever builds you run on your own machine as an upstream developer. It's to produce a collection of installable software that meets various constraints and goals, like cohesion (can all be installed and managed uniformly), minimal size, easy/manageable security updates, integration (compatibility and so on). That can involve things like building the software against particular library versions mandated by downstream needs or even patching it. Some distros try hard to avoid patching upstream and some don't, and in all distros there may be cases where other priorities take precedence over the value of leaving upstream untouched.

In the case of Nixpkgs and Python, the community wants to maintain a collection of Python libraries that are all interoperable, and Python doesn't support vendorization well enough to allow multiple versions of the same library in a single Python process, which is one reason for preferring singular versions of most Python libraries in Nixpkgs. The other factor is likely just reducing the maintenance across Nixpkgs by maintaining as few redundant versions within the tree as possible.

If you want to control/determine the entire runtime your end users use, you have to do the packaging work required to ship them that runtime with some tooling that's capable of the reproducibility you desire. Python doesn't have one a reproducible package manager, so your options are basically creating your own Nix package (probably as a flake.nix in your repo), Docker, and Flatpak.

That said, it's perfectly possibly to include multiple minor releases of Django 4 in a single snapshot of the Nixpkgs tree and maybe that should be done. Have you talked with the maintainers of your downstream package of Nixpkgs to let them know Django breaks things on minor releases, and so using different versions of Django 4 interchangeably is not tested or supported in your application?

It doesn’t help that the term is hugely overloaded. Nix might mean the package manager, the shell, the operating system. There are so many valid permutations of letters; why they chose to reuse Nix for everything is a mystery to me.

The other issue is complexity. If you can manage to figure out the jargon, you’re greeted with the requirement that you completely port an entire project to get any benefit, and that’s non trivial. It requires learning a whole new language, and when the (highly opinionated) language conflicts with other tools’ ideas, for example pip, the documentation generally bashes the other tool, boasts how much better it is, and then proceeds to have devs write dozens of lines in a new language they invented when one line of Python used to be enough.