The Curse of NixOS

From everything I've read it seems like NixOS would really take off if they had great docs like Rust (both api docs and compiler error messages). Well, that and debugging.

Yes, there is system.replaceRuntimeDependencies [1] that does what you ask: It replaces dependencies recursively in derivation outputs through binary patching, creating _new_ store paths (so immutability is preserved). It does so by replacing all occurrences of the store paths of the original dependency (e.g., libc) with the new ones [2].

[1] https://search.nixos.org/options?channel=21.11&show=system.r...

[2] https://github.com/NixOS/nixpkgs/blob/fad04722fc3d692e3511e5...

I too have tried nixOS and nix many times in the past few years (and just recently went back to having a computer with it permanently installed, albeit not my main one).

I think another underappreciated reason it's been difficult to use is the CLIs involved are often really arcane (in ways that often remind me of using GNU find, a program I don't think is one that scores high UX marks).

For example, uninstalling something with `nix-env` is spelled `nix-env -e something`, and querying is like `nix-env -iA ...` (yes there's a long form of the options, and most people tell you to never use nix-env, but these were the examples I recalled, I think the UX is similar across other tools). Trying to decipher what these command lines mean and searching for which ones do what you want is just really challenging, they don't follow the "rules" I'm familiar with for other CLI tools. (Maybe they follow norms from some other part of the OSS community though.)

The new `nix` CLI seems to be trying to "dumb" things down a bit (or optimistically -- make a CLI that is more transparent about what operations it's doing), which is great, but it too exhibits some odd behaviors. E.g. recently after an upgrade I found it wouldn't work without passing `nix --extra-features flakes --extra-features nix-experimental-command`, even if I seemingly wasn't doing anything related to flakes. It is indeed marked experimental, so I can't fault it too much, but yeah I've found significant challenge as a new user in just trying to understand the various nix package manager binaries, their options, and how they map to "normal" package manager operations (even ignoring the procedural vs. declarative angle for how you actually manifest changes).

EDIT: Other things I recall get me every time I reinstall nixOS is re-learning that you get to run `nix-collect-garbage` manually, that `nix-channel --add` doesn't automatically update, and I see I wrote down to remember that package names are case sensitive (though I didn't recheck that recently, hopefully that's changed?).

GoboLinux also uses patchelf.

There are some fundamental problems that no distribution can solve (not even Nix). Every level of abstraction has its own set of dependency issues, yet there exists no abstraction or concept that handles conflicts in every one of those levels. There are a lot of hacks and kludges used by Nix in order to try to get there, but it's only most of the way. We've hit the limit of what we can kludge.

I think the future of software dependency conflict resolution requires a fundamental shift in software design. We have to start new paradigms for software where handling interoperability with conflicting dependencies is a first-order feature of all systems.

BerkeleyDB is a famous example of conflicts. You can't have one BerkeleyDB database shared by multiple apps designed for conflicting versions of BerkeleyDB. If you have two apps doing two different things with the same database, but they use different versions, you need two separate databases.

This could be solved if both the database library and the applications were written with a version-independent abstraction. Regardless of the API/ABI, you have the app say "what versions do you support?", and then "you want v3? here's your interface", or "you want v4? here's this other interface". Internally, each app/db will use its own interface, but a translation layer will handle the v3 or v4 layer, without the application even being aware of it. As long as one of the apps is newer than the other, that newer app will have backwards compatibility with the old app. We won't need to install multiple stacks of dependencies (ala Nix) because different versions of apps will just be compatible. When in doubt, install a newer version.

At first this seems unworkable. What, require everyone to constantly support every old interface?? But I think we can start over with some new concepts and primitives to handle the majority of use cases so the programmer (mostly) doesn't have to think about it. We already have relocatable versioned ABIs for things like Glibc where it advertises what minimum/maximum version it supports. We just take that a few steps further and make backwards compatibility the default. Maybe we get new build systems and new ways of linking and calling code, and ship old code bundled along with the new? We need new research to develop these new methods, because the current system we're dealing with has no more fixes possible.