Live data from Hacker News

20 years of Nix

20th.nixos.org

41–50 of 113 posts

Re: 20 years of Nix

#41

Earlier quoted context omitted.

There are two sides to this problem, the first is to improve the UX, but the second is to clearly describe a compelling reason for people to adopt. It is very tempting to only blame the first, but I think we need to also need to tell a better story and highlight the values in a better way. This would then give people a reason to get past the UX issues in the hopes of achieving those desired values. For example; peopl…

I disagree, I think the value proposition for reproducibility is clear, it's just that the learning curve "is too damn high!" I'm highly motivated to learn and use Nix (or Guix for that matter) but I've bounced off of it three or four times now, and I'm the kind of weirdo who learns new PLs for fun. Someone once said that you don't learn Nix, you reverse engineer it.

I agree with you. I can see the value of reproducibility, declarative system setup, etc.

I would love that. But there's no way I'm fighting software with such a bad UX.

Re: 20 years of Nix

#42
post #27
post #4

I have been exploring nix for the past few months and my experience with nix has been both exhilarating and frustrating, simultaneously. On one hand, I find it hard to imagine not using nix now, but on the other hand, I hesitate to recommend it to other colleagues due to its steep learning curve, ux issues and potential for footguns. I sincerely hope that nix community improves the UX to make it more accessible to ne…

What are the footguns? I feel that footgun descriptions always have the most insightful bits about technology.

A few antipatterns/annoyances I've come across over the years:

Importing paths based on environment variables:

There is built-in support for this, e.g. setting the env var `NIX_PATH` to `a=/foo:b=/bar`, then the Nix expressions `` and `` will evaluate to the paths `/foo` and `/bar`, respectively. By default, the Nix installer sets `NIX_PATH` to contain a copy of the Nixpkgs repo, so expressions can do `import ` to access definitions from Nixpkgs.

The reason this is bad is that env vars vary between machines, and over time, so we don't actually know what will be imported.

These days I completely avoid this by explicitly un-setting the `NIX_PATH` env var. I only reference relative paths within a project, or else reference other projects via explicit git revisions (e.g. I import Nixpkgs by pointing the `fetchTarball` function at a github archive URL)

Channels:

These always confused me. They're used to update the copy of Nixpkgs that the default `NIX_PATH` points to, and can also be used to manage other "updatable" things. It's all very imperative, so I don't bother (I just alter the specific git revision I'm fetching, e.g. https://hackage.haskell.org/package/update-nix-fetchgit helps to automate such updating).

Nixpkgs depends on $HOME:

The top-level API exposed by the Nixpkgs repository is a function, which can be called with various arguments to set/override things; e.g. when I'm on macOS, it will default to providing macOS packages; I can override that by calling it with `system = "x86_64-linux"`. All well and good.

The problem is that some of its default values will check for files like ~/.nixpkgs/config.nix, ~/.config/nixpkgs/overlays.nix, etc. This causes the same sort of "works on my machine" headaches that Nix was meant to solve. See https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/...

I avoid this by importing Nixpkgs via a wrapper, which defaults to calling Nixpkgs with empty values to avoid its impure defaults; but still allows me to pass along my own explicit overrides if needed.

The imperative nix-env command:

Nix provides a command called 'nix-env' which manages a symlink called ~/.nix/profile. We can run commands to "install packages", "update packages", "remove packages", etc. which work by building different "profiles" (Nix store paths containing symlinks to a bunch of other Nix store paths).

This is bad, since it's imperative and hard to reproduce (e.g. depending on what channels were pointing to when those commands were run, etc.). A much better approach is to write down such a "profile" explicitly, in a git-controlled text file, e.g. using the `pkgs.buildEnv` function; then use nix-env to just manage that single 'meta-package'.

Tools which treat Nix like Apt/Yum/etc.

This isn't something I haven't personally done, but I've seen it happen in a few tools that try to integrate with Nix, and it just cripples their usefulness.

Package managers like Apt have a global database, which maps manually-written "names" to a bunch of metadata (versions, installed or not, names of dependencies, names of conflicting packages, etc.). In that world names are unique and global: if two packages have the name "foo", they are the same package; clashes must be resolved by inventing new names. Such names are also fetchable/realisable: we just plug the name and "version number" (another manually-written name) into a certain pattern, and do a HTTP GET on one of our mirrors.

In Nix, all the above features apply to "store paths", which are not manually written: they contain hashes, like /nix/store/wbkgl57gvwm1qbfjx0ah6kgs4fzz571x-python3-3.9.6, which can be verified against their contents and/or build script (AKA 'derivation'). Store paths are not designed to be managed manually. Instead, the Nix language gives us a rich, composable way to describe the desired file/directory; and those descriptions are evaluated to find their associated store paths.

Nixpkgs provides an attribute set (AKA JSON object) containing tens of thousands of derivations; and often the thing we want can be described as 'the "foo" attribute of Nixpkgs', e.g. '(import {}).foo'

Some tooling that builds-on/interacts-with Nix has unfortunately limited itself to only such descriptions; e.g. accepting a list of strings, and looking each one up in the system's default Nixpkgs attribute set (this misunderstanding may come from using the 'nix-env' tool, like 'nix-env -iA firefox'; but nix-env also allows arbitrary Nix expressions too!). That's incredibly limiting, since (a) it doesn't let us dig into the structure inside those attributes (e.g. 'nixpkgs.python3Packages.pylint'); (b) it doesn't let us use the override functions that Nixpkgs provides (e.g. 'nixpkgs.maven.override { jre = nixpkgs.jdk11_headless; }'); (c) it doesn't let us specify anything outside of the 'import {}' set (e.g. in my case, I want to avoid NIX_PATH and altogether!)

Referencing non-store paths:

The Nix language treats paths and strings in different ways: strings are always passed around verbatim, but certain operations will replace paths by a 'snapshot' copied into the Nix store. For example, say we had this file saved to /home/chriswarbo/default.nix:

  # Define some constants
  with {
    # Import some particular revision of Nixpkgs
    nixpkgs = import (fetchTarball {...}) {};

    # A path value, pointing to /home/chriswarbo/defs.sh
    defs = ./defs.sh;

    # A path value, pointing to /home/chriswarbo/cmd.sh
    cmd = ./cmd.sh;
  };
  # Return a derivation which builds a text file
  nixpkgs.writeScript "my-super-duper-script" ''
    #!${nixpkgs.bash}/bin/bash
    source ${nixpkgs.lib.escapeShellArg defs}
    ${cmd} foo bar baz
  ''
Notice that the resulting script has three values spliced into it via ${...}:

- The script interpreter `nixpkgs.bash`. This is a Nix derivation, so its "output path" will be spliced into the script (e.g. /nix/store/gpbk3inlgs24a7hsgap395yvfb4l37wf-bash-5.1-p16 ). This is fine.

- The path `cmd`. Nix spots that we're splicing a path, so it copies that file into the Nix store, and that store path will be spliced into the script (e.g. /nix/store/2h3airm07gp55rn9qlax4ak35s94rpim-cmd.sh ). This is fine.

- The string `nixpkgs.lib.escapeShellArg defs`, which evaluates to the string `'/home/chriswarbo/defs.sh'`, and that will be spliced into the script. That's bad, since the result contains a reference to my home folder! The reason this happens is that paths can often be used as strings, getting implicitly converted. In this case, the function `nixpkgs.lib.escapeShellArg` transforms strings (see https://nixos.org/manual/nixpkgs/stable/#function-library-li... ), so:

- The path `./defs.sh` is implicitly converted to the string `/home/chriswarbo/defs.sh`, for input to `nixpkgs.lib.escapeShellArg` (NOTE: you can use the function `builtins.toString` to do the same thing explicitly)

- The function `nixpkgs.lib.escapeShellArg` returns the same string, but wrapped in apostrophes (it also adds escaping with backslashes, but our path doesn't need any)

- That return value is spliced as-is into the resulting script

To avoid this, we should instead splice the path into a string before escaping; giving us nested splices like this:

    source ${nixpkgs.lib.escapeShellArg "${defs}"}

Re: 20 years of Nix

#43

Doe nix work at all? Or is it just not actually functional on top of MacOS? I've tried a dozen times over the years and I've never gotten it working. Seriously, how is anything so hard to use still around after 20 years!

I ship a development environment to a fleet of ~100 engineer laptops based on Nix. If it didn't work, I'd be out of a job

Why would engineers need their environment shipped to them by a human?

Re: 20 years of Nix

#44
I tried to install NixOS using the live cd last week in a Hyper-V VM, but it failed to get anywhere due to SquashFS errors.

That seemed pretty low-level so I'm putting it back on the back burner. User friendliness doesn't seem to be a top priority, and that's fine.

Re: 20 years of Nix

#45

Earlier quoted context omitted.

I ship a development environment to a fleet of ~100 engineer laptops based on Nix. If it didn't work, I'd be out of a job

Why would engineers need their environment shipped to them by a human?

Technically they `git pull` it themselves, but the fact that it continues to work involves a human.

Re: 20 years of Nix

#46
It feels so painful to go back to ‘regular’ Linux now. I'm so concerned about config file entropy and version incompatibility that Nix has solved for me. I'm happy I took the Nix Pill though and completely skipped over Docker and it's often-unnecessary overhead. Nix store is a better solution to reproducible builds, and the syntax is a lot better than LISP for Guix or whatever Skylark is trying to be.

Currently I'm setting up a second machine to distribute builds and share a cache on my local network. Overriding C flags Gentoo-style for better optimization is supported, but it can take a while to build--especially with LTO--as Hydra only builds for generic x86_64 so sharing optimized kernels and other software is great. I successfully got a shared znver3 LTO-optimized Linux 6.1.19 kernel with ZFS support yesterday! I just wish I could have built in parallel the kernel on the faster PC and the ZFS stuff on the slower one and resynced the build input derivations when it was finished after running `nixos-rebuild switch --flake ..`.

For the future, I hope distributed Nix caches become the norm like BitTorrent and we can all share optimized builds.

Re: 20 years of Nix

#47
post #34

Earlier quoted context omitted.

Sarcasm aside, here’s a Nix manual from 2004: https://releases.nixos.org/nix/nix-0.5/manual/manual.html . (The compile time of Nix itself is unpleasant, but not exactly exceptional among programs written in the modern C++ style. The eval time even for Nixpkgs even on a ten-year-old i5 is annoying but not a terrible problem the way it’s used now, though even on a recent Android device it’s admittedly measured in minut…

With the Hydra binary cache I can quite comfortably update my Avoton C2550 based router or Raspberry Pi 4 AirPlay receiver within a few minutes. It’s only if I need to build a non trivial package that I have to make sure I build on my desktop or in a VM on my MacBook.

I doubt your setup needed recompiling the Nix binary itself at any point. Even I did that more out of love of adventure than for any practical reasons, it’s just that the scars are still there. (Still less than those from the time when the LibreOffice build was broken on Hydra and a routine system update jumped right into trying to perform it locally, even if that indeed was what I technically asked for...)

And you know what, the long evaluation thing might just be a bug. Or at least I don’t see any other reason why (e.g.) `nix-shell -p yt-dlp` works reasonably fast on my Nix-on-Droid[1] installation but `nix shell nixpkgs#yt-dlp` takes minutes.

[1] https://github.com/t184256/nix-on-droid

Re: 20 years of Nix

#48

I tried to install NixOS using the live cd last week in a Hyper-V VM, but it failed to get anywhere due to SquashFS errors. That seemed pretty low-level so I'm putting it back on the back burner. User friendliness doesn't seem to be a top priority, and that's fine.

> I tried to install NixOS using the live cd last week in a Hyper-V VM, but it failed to get anywher e due to SquashFS errors.

I'm curious what happened here because just a few weeks ago I was using a NixOS live-cd to rescue a botched gentoo VM on my windows machine (managed with Hyper-V manager).

If you give it another shot, run into the issue again, and document it, the Nix community would almost certainly help you debug it and figure out what went wrong.

Re: 20 years of Nix

#49
post #38

Earlier quoted context omitted.

I think this project is really promising. My main concern is that it puts another layer of abstraction atop an already complex (and at times leaky) abstraction. I’d love to see more clear docs about what devenv is actually doing under the covers, and how to escape-hatch into Nix land when I inevitably need to tweak something. Also, similarly, how do I map Nix docs (often just a set of example expressions) into equiva…

I think that's a valid concern. You read the nix pills and think you know what you're doing and then it turns out that the community has wrapped the things you've learned about in things you've never heard of, so you still can't learn from other people's repos.

This. I've been actively trying nix-based tooling on and off for my projects because it is legitimately solve the problem around sandboxing, versioning, reproducibility, consistency, etc. Recently, I'm trying to use asdf (and its faster alternative, rtx) and I keep telling myself "huh, nix could solve this problem better". But, damn it is infuriating to learn. Like other commenter said, it is the escape hatch that I'm missing so much. I really really want to convince myself to learn nix The Right Way. But, it feels like you will have another learning curve when using a nix-wrapper tooling.

I still have a high hope for the future of nix. And I believe the time nix will rise in popularity is when they have sorted the UX-related issues.

Re: 20 years of Nix

#50

Earlier quoted context omitted.

And here I was thinking about Gentoo...

Oh that takes me back to trying to explain to some Gentoo users back in the day that that yea you can optimise stuff better for your slow ass computer. But you now have to spend most of your time compiling code on your slow ass computer. I always found Arch to be a much more reasonable system for this use case because compiling everything from scratch was just pointless, at least back then. But it still maoes it easy…

> Oh that takes me back to trying to explain to some Gentoo users back in the day that that yea you can optimise stuff better for your slow ass computer. But you now have to spend most of your time compiling code on your slow ass computer.

Honestly it depends what your goal is with gentoo. My whole sell with gentoo is that custom packages are just so easy. I can take some garbage toolchain from an embedded manufacturer and wire up an ebuild for it that "just works". Nix is similar once you get it working but the nixpkg and nixos DSLs just kinda suck in their own ways. So it may take me 10-15 min to write and install an ebuild while the nix package may take me an hour or two.

Same goes for situations where you need weird kernel configs. It's so easy to just recompile the kernel on gentoo once you have stuff set up. The OS is built expecting a majority of users will want to tweak their kernels so the OS's tooling doesn't fight you when you do. I find this often less well handled in other distros.

As for arch, it is pretty close IMHO but (while it may have changed since I used it last) I found that the "bleeding-edge" focus makes dealing with old janky dependencies to be pretty painful.

Gentoo certainly isn't the first choice for the majority of people but it IMHO is a really good choice if you are going to be working with funky proprietary toolchains (o/ hi embedded engs).

Post reply on HN