61–70 of 155 posts
This seems like a good thing for system-level packages, but what about those language-level packages? Could Nix replace both dockerfiles (or a list of apt packages) and pip/conda?
For Docker, you just call a Nix function and specify what programs to include (both implicitly and explicitly via use) and Nix automatically includes the transitive closure of what you need
Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing. But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions. Containers are great and all, they s…
I doubt anyone behind Nix is writing patches for every single application that exists, and every version of every application, in order to achieve this. The only other way to do it is by re-building all of the deps and installing them in an environment specific to an application (what containers do), and then having an interface to be able to choose what versions of what environments to combine with which applications in what ways. You can't even create a Bash one-liner using two different apps/environments and ensure it'll work, unless you explicitly define what versions of the shell commands you're going to use together. It's impossible to automate. Hence "regular" distros with specific package trees, and containers to provide the interface to mix application/environment pairs.
There is no better solution, because all software development itself is inherently flawed. It's all 1970s programming: a single system with a single set of dependencies per application (both build-time & run-time). There is no solution to that problem other than to reinvent how software works so that versions are explicit in all code execution on a system. An application would have to request code execution with a specific version, and the system would have to figure out how to provide that version of that code at run time.
Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing. But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions. Containers are great and all, they s…
One thing I like with Nix is that it can build containers, so it's totally compatible with deployment systems that require containers. I don't mean using Nix commands in a Dockerfile, I mean having a Nix derivation that produces a container. I've been writing normal Dockerfiles at work recently and the whole toolchain is annoying on multiple fronts; I'd love to jump to Nix instead, except it's really hard to convince…
Using nix for building docker images [1] is brilliant, especially when combined with flakes. You don't need messy shell scripts to set up the environment and can get 100% reproducibility.
You just build a regular Nix derivation (aka package) with the software you need and then just specify that in your image.
It does take some work to get the image size down though.
I'll also mention Nixery [2], which is a Docker registry that automatically builds images based on Nix.
[1] https://nix.dev/tutorials/building-and-running-docker-images
Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing. But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions. Containers are great and all, they s…
"Supporting multiple versions" is a misnomer. You can't use multiple versions of a dependency on a single system without completely separating both the build-time and run-time environments, which is what containers do. If Nix did that implicitly it would mean that the applications would all have to be patched - not only to load specific library versions, but to call exec() on specific versions of binaries of complete…
Sure you can. It's what version numbers in library file names are for. Works for executable names, too. Until Python 2.7 died, you commonly had 2.7 and 3.x installed at the same time.
But maybe you meant something different?
Nix, guix and traditional package managers that support multiple versions (like dnf) are better solutions than just containerizing the whole thing. But here we are, some projects only give development instructions based on containers, others support only container based deployment. You need to go out of the way trying to convert dockerfiles into regular install/setup instructions. Containers are great and all, they s…
> Containers are great and all, they solve many problems but they're not the only solution and theyre definitely not the best solution to every problem. There a large number of developers around now who've never done traditional (RPM, DEB, SysV) packaging and don't understand it. So while the large distros like Redhat and Debian push on with it, the development community only sees Dockerfiles, Snaps, Flatpaks etc.
> My ideal dependency manager would allow me to specify each and every dependency that is required to work on my projects. It should be easily reproducible, declarative and easy to upgrade. To me it feels like what's left out here is the fact that you need the amount of dependencies that you have to be reasonable in the first place, otherwise no single piece of software is going to help you all that much. For example…
You're not wrong, but that's a different problem. Nix makes sure that you get a predictable and reproducible tree of dependencies, and allows different applications to depend on different versions of the same dependencies. That is, it's a solution to DLL hell. It's solid engineering based on solid theory, and it really does let you manage configuration with a level of reliability that most other package managers only…
It's your job when downloading a package to use or when adding a dependency to make sure you trust the source - this is not a dependency management problem.
A good dependency manager should help you better filter trusted sources or at least have a good build file layout to make it obvious what the sources are but that's it.
First, you want a "dependency manager". That's not what Nix is, clearly. Nix is a package manager. Second, to use this package manager, you first need to install direnv . How do you install it? with brew , a different package manager. Third, you have to learn a new functional programming language. Right. Because normally to put together a toolbox, I often learn to speak a few phrases in Swahili first. Fourth, finally…
Though there is a wealth of plugins, what's nice is that it's trivial to write your plugin. So if something doesn't exist yet, or you have some internal tool you need to version, it takes less than an hour to write a plugin.
In this post:
--- start quote ---
How about a different version of Ruby or Node? Let’s say that our project depends on Ruby 2.6 and Node 10. We can go and search for those specific versions
--- end quote ---
So, to begin with, we still need a "version manager", because we want specific package versions. But look at how this is implemented in nix:
let
pkgs = import { };
in
pkgs.mkShell {
buildInputs = [
pkgs.hello
pkgs.ruby_2_6
pkgs.nodejs-10_x
];
}
And these packages aren't even correct. If you do go and search for ruby, for example [1], you get the following:
ruby Version: 2.7.4
ruby_3_0 Version: 3.0.2
ruby_2_6 Version: 2.6.8
And they call this reproducible builds?
And that's before getting into the ridiculous
--- start quote ---
All the software that we installed depends on the specific version of the nixpkgs channel that we installed on our system [whose only version is a commit hash in a git repo]
--- end quote ---
So you need an extra tool [2] for, quote, "painless dependencies for Nix projects."
Yes, sure. I'm definitely ditching my version managers in favor of this tool, that hasn't solved these issues in 18 years of its existence.
[1] https://search.nixos.org/packages?channel=21.05&from=0&size=...
Earlier quoted context omitted.
"Supporting multiple versions" is a misnomer. You can't use multiple versions of a dependency on a single system without completely separating both the build-time and run-time environments, which is what containers do. If Nix did that implicitly it would mean that the applications would all have to be patched - not only to load specific library versions, but to call exec() on specific versions of binaries of complete…
> You can't use multiple versions of a dependency on a single system without completely separating both the build-time and run-time environments Sure you can. It's what version numbers in library file names are for. Works for executable names, too. Until Python 2.7 died, you commonly had 2.7 and 3.x installed at the same time. But maybe you meant something different?
First, install python 2 and python 3. Next, install a script that calls "#!/usr/bin/python". How do you make the script work, without modifying the script, so that it will work with the specific version of Python it was designed against? You can't. You have to either modify the script to point at the specific executable it was designed for, or you have to modify the environment to point "/usr/bin/python" at either python2 or python3.
The former, "modifying the script", is recompiling all applications to point all of the executables they run at a custom executable, like "/run/app/v2/bin/app".
The latter, "changing the environment", requires that you specify the environment before you run the script (". /etc/python2.env ; /some/script.py"). But then you have to know that, right now, you want to run the script against Python 2. What if you install an upgraded version of the script that's built against Python 3? How do you know which script to run when? You have to have an interface and pick which one to use each time you run it. And every application that runs that script.
You can, of course, install two versions of your script, and rename the scripts, so that you know script2.py runs against python2 and script3.py runs against python3. But what if you have another program that needs to call the script? How does it know which of these 2 custom names to run? Now you'll have to compile a new custom package for this new program, and rename it too, and hard-code that one to call the specific new custom script name that targets the specific Python version.
And on, and on, and on, you will need to have duplicate packages for every possible dependency graph that could possibly exist of different versions of different programs. You would end up with 16,000 packages of this python script, each with a custom name and custom paths. Because some combination of dependent packages could lead to a recursive bug where the wrong app eventually calls the wrong script.
And it also requires patching literally every single application that is ever packaged by Nix to be able to rename applications and libraries. Every C program that ever calls "wc" needs to be patched to run "wc" version 2.0.0, "wc" version 2.0.1, "wc" version 3.0.0, etc, for every version of wc that may exist, and for every version of wc that may call a different program which also has multiple versions.
This is why the problem is unsolvable with any tooling whatsoever. There is no way to automatically determine the correct dependencies without modifying and packaging every application for every dependency tree that could possibly exist. The only solution is what containers give you: an interface that forces you to pick a version of an application with one dependency tree, and lets you figure out how to combine them (how to run containers together).
The only way to solve it without either 16,000 variants of one package, or an interface to pick environments at exec time, is to fundamentally change how software is designed and run today.