Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why We're Moving on from Nix

blog.railway.com

Post reply on HN
Print view View original

31–40 of 144 posts

Re: Why We're Moving on from Nix

#31

Post by matthewbauer »

k__ wrote: post #14
ris wrote: post #10

The main problem here is wanting to hang on to the "bespoke version soup" attitude that language package managers encourage (and is totally unsustainable). The alternative Mise doesn't appear to have any ability to understand version constraints between packages and certainly doesn't run tests for each installed package to ensure it works correctly with the surrounding versions. So you're not getting remotely the sam…

"the "bespoke version soup" attitude that language package managers encourage" Care to elaborate what that means and what the alternative is?

Not sure on the exact take of the OP, but:

Package maintainers often think in terms of constraints like I need a 1.0.0 - constraints are not always right (say pkg1==1.9.0 actually breaks things)

- constraints of each dependency combined ends up giving very little degrees of freedom in constraint solving, so that you can’t in fact just take any pkg1 and use it

- even if you can use a given version, your package may have a hidden dependency on one if pkg1’s dependencies, that is only apparent once you start changing pkg1’s version

Constraint solving is really difficult and while it’s a cool idea, I think Nixpkgs takes the right approach in mostly avoiding it. If you want a given version of a package, you are forced to take the whole package set with you. So while you can’t say take a version of pkg1 from 2015 and use it with a version of pkg2 from 2025, you can just take the whole 2015 Nixpkgs and get pkg1 & pkg2 from 2015.

Re: Why We're Moving on from Nix

#32

Post by jpgvm »

k__ wrote: post #14
ris wrote: post #10

The main problem here is wanting to hang on to the "bespoke version soup" attitude that language package managers encourage (and is totally unsustainable). The alternative Mise doesn't appear to have any ability to understand version constraints between packages and certainly doesn't run tests for each installed package to ensure it works correctly with the surrounding versions. So you're not getting remotely the sam…

"the "bespoke version soup" attitude that language package managers encourage" Care to elaborate what that means and what the alternative is?

Nix generally speaking has a global "nixpkgs" version (I'm greatly over-simplifying here ofc) in which there is a single version of each package.

This is likely the source of their commit based versioning complaint/issue, i.e the commits in question are probably https://github.com/NixOS/nixpkgs versions if they aren't maintaining their own overlay of derivations.

This is in contrast to systems that allow all of the versions to move independently of each other.

i.e in the Nix world you don't just update one package, you move atomically to a new set of package versions. You can have full control over this by using your own derivations to customise the exact set of versions, in practice most folk using Nix aren't deep enough in it for that though.

Re: Why We're Moving on from Nix

#35

Post by lewo »

> With no way of splitting up the Nix dependencies into separate layers

nix2container [1] is actually able to do that: you can explicitly build layers containing a subset of the dependencies required by your image. An example is provided in this section: https://github.com/nlewo/nix2container?tab=readme-ov-file#is...

For instance, if your images use bash, you can explicitly create a layer containing the bash closure. This layer can then be used across all your images and is only rebuild and repushed if this bash closure is modified.

> > pull in dependencies often results in massive image sizes with a single /nix/store layer

This is the case for the basic nixpkgs.dockerTools.buildImage function but this is not true with nix2container, nor with nixpkgs.dockerTools.streamLayeredImage. Instead of writing the layers in the Nix store, these tools build a script to actually push the image by using existing store paths (which are Nix runtime dependencies of this script). Regarding the nix2container implementation, it builds a JSON file describing the Nix store paths for all layers and uses Skopeo to push the image (to a Docker deamon, a registry, podman, ...), by consuming this JSON file.

(disclaimer: i'm the nix2container author)

[1] https://github.com/nlewo/nix2container

Re: Why We're Moving on from Nix

#36

Post by 0xbadcafebee »

matthewbauer wrote: post #31
k__ wrote: post #14

Earlier quoted context omitted.

"the "bespoke version soup" attitude that language package managers encourage" Care to elaborate what that means and what the alternative is?

Not sure on the exact take of the OP, but: Package maintainers often think in terms of constraints like I need a 1.0.0 - constraints are not always right (say pkg1==1.9.0 actually breaks things) - constraints of each dependency combined ends up giving very little degrees of freedom in constraint solving, so that you can’t in fact just take any pkg1 and use it - even if you can use a given version, your package may ha…

> Constraint solving is really difficult and while it’s a cool idea, I think Nixpkgs takes the right approach in mostly avoiding it. If you want a given version of a package, you are forced to take the whole package set with you.

Thank you, I was looking for an explanation of exactly why I hate Nix so much. It takes a complicated use case, and tries to "solve" it by making your use-case invalid.

It's like the Soylent of software. "It's hard to cook, and I don't want to take time to eat. I'll just slurp down a bland milkshake. Now I don't have to deal with the complexities of food. I've solved the problem!"

Re: Why We're Moving on from Nix

#37

Post by jonhohle »

matthewbauer wrote: post #31
k__ wrote: post #14

Earlier quoted context omitted.

"the "bespoke version soup" attitude that language package managers encourage" Care to elaborate what that means and what the alternative is?

Not sure on the exact take of the OP, but: Package maintainers often think in terms of constraints like I need a 1.0.0 - constraints are not always right (say pkg1==1.9.0 actually breaks things) - constraints of each dependency combined ends up giving very little degrees of freedom in constraint solving, so that you can’t in fact just take any pkg1 and use it - even if you can use a given version, your package may ha…

There’s no clear definition (in most languages, of major/minor/patch versioning). Amazon did this reasonably well when I was there, though the patch version was implicitly assigned and the major and minor required humans to follow the convention:

You could not depend on a patch version directly in source. You could force a patch version other ways, but each package would depend on a specific major/minor and the patch version was decided at build time. It was expected that differences in the patch version were binary compatible.

Minor version changes were typically were source compatible, but not necessarily binary compatible. You couldn’t just arbitrarily choose a new minor version for deployment (well, you could, but without expecting it to go well).

Major versions were reserved for source or logic breaking changes. Together the major and minor versions were considered the interface version.

There was none of this pinning to arbitrary versions or hashes (though, you could absolutely lock that in at build time).

Any concept of package (version) set was managed by metadata at a higher level. For something like your last example, we would “import” pkg2 from 2025, bringing in its dependency graph. The 2025 graph is known to work, so only packages that declare dependencies on any of those versions would be rebuilt. At the end of the operation you’d have a hybrid graph of 2015, 2025, and whatever new unique versions were created during the merge, and no individual package dependencies were ever touched.

The rules were also clear. There were no arbitrary expressions describing version ranges.

Re: Why We're Moving on from Nix

#38

Post by [deleted] »

jonhohle wrote: post #37
matthewbauer wrote: post #31

Earlier quoted context omitted.

Not sure on the exact take of the OP, but: Package maintainers often think in terms of constraints like I need a 1.0.0 - constraints are not always right (say pkg1==1.9.0 actually breaks things) - constraints of each dependency combined ends up giving very little degrees of freedom in constraint solving, so that you can’t in fact just take any pkg1 and use it - even if you can use a given version, your package may ha…

There’s no clear definition (in most languages, of major/minor/patch versioning). Amazon did this reasonably well when I was there, though the patch version was implicitly assigned and the major and minor required humans to follow the convention: You could not depend on a patch version directly in source. You could force a patch version other ways, but each package would depend on a specific major/minor and the patch…

[deleted]

Re: Why We're Moving on from Nix

#39

Post by aidenn0 »

ris wrote: post #10

The main problem here is wanting to hang on to the "bespoke version soup" attitude that language package managers encourage (and is totally unsustainable). The alternative Mise doesn't appear to have any ability to understand version constraints between packages and certainly doesn't run tests for each installed package to ensure it works correctly with the surrounding versions. So you're not getting remotely the sam…

You can actually have both if you do it right. It's trivial to build a rust package with Nix from a Cargo.lock file, for example. Nixpkgs is contrary to bespoke version soup, but Nix itself can be fine with it.

Re: Why We're Moving on from Nix

#40

Post by smilliken »

nialv7 wrote: post #18
setheron wrote: post #17

Nix gives you a commit guarantee rather than arbitrary versions. You're going to put yourself in a bad time when you have edge cases: glibc changes or conflicting shared libraries. It sounds like it's a little bit too late, but I'm happy to provide some consulting on how you can get it to work idiomatically with Nix. Product looks cool!

nix solves the shared library incompatibility problem by being extremely conservative. every time anything changes, consequential or not - a comment got modified, documentation changes, a testcase got added, etc. - it will rebuild all dependents. and not just that, but all dependents of dependents, and dependents of dependents of dependents, on and on. this often results in massive massive rebuilds. sure you are not…

The reason someone changes a dependency at all is because they expect a difference in behavior. No one would feel the motivation to go update a dependency if they aren't getting something out of it, that's a waste of effort and an unnecessary risk.

Each person doesn't have to perform the build on their own. A build server will evaluate it and others will pull it from the cache.

The greater waste that nix eliminates is the waste of human time spent troubleshooting something that broke in production because of what should have been an innocent change, and the lost business value from the decreased production. When you trust your dependencies are what you asked for, it frees the mind of doubt and lets you focus on troubleshooting more efficiently towards a problem.

Aside, I spent over a decade on Debian derived distros. I never once had one of these distros complete an upgrade successfully between major versions, despite about 10 attempts spread over those years, though thankfully always on the first sacrificial server attempted. They always failed with interesting issues, sometimes before they really got started, sometimes borking the system and needing a fresh install. With NixOS, the upgrades are so reliable they can be done casually during the workday in production without bothering to check that they were successful. I think that wouldn't be possible if we wanted the false efficiency of substituting similar but different packages to save the build server from building the exact specification. Anything short of this doesn't get us away from the "works on my machine" problem.

Post reply on HN