My First Impressions of Nix

I suspect you may take other technology you use as granted, and have forgotten the inherent subtlety that you've had to work through to attain the working understanding you have today. Consider the confusion you might have being an aspiring developer, on your first day cracking open a book on $INSERT_LANG_HERE, and then someone mentions Docker. Imagine your confusion:

"What is docker? It builds things? So, like, a "build system"? But it invokes something called make, or cmake, or... well, aren't those build systems? If that isn't confusing, I don't know what is. Oh, you say it's software that runs on Linux -- what's that? So docker is an OS? No? It runs on an OS? And you're telling me it doesn't just build stuff, it also can coordinate the execution of stuff? Oh, you're telling me that's docker-compose -- that isn't docker itself? I mean, it had 'docker' in the name. Oh. So separate project to 'docker the container image making thingy', but can be used with that 'docker'. Good God this is confusing."

Much like the Docker and its ecosystem as whole -- along with all the enabling/leveraged tech underlying it, like chroots, user/pid/network/etc namespaces, union filesystems, seccomp, etc -- there is some inherent complexity in Nix (and its broader ecosystem of tools).

Just like docker has docker, docker-compose, docker images, stateful docker containers (is it running? dead? what subtree of my filesystem is mounted where in the container? etc), a whole syntax for "docker files", etc, we have multiple things that come together to make "the whole of Nix" what it is.

As a whole, for Nix we have (non-exhaustive, but hopefully broad enough to help paint a picture):

- Nix the language (analogous to Dockerfile syntax), which is used by

- Nix the package builder/manager (analogous to the `docker` binary)

- Nixpkgs, which is a collection of packages (a bit of a stretch if you take it too literally, but analogous to a collection of Dockerfiles, each for a different software)

- NixOS, an operating system that leverages the packages declared in Nixpkgs.

That hopefully explains the "WHAT", but says nothing of "WHY".

So to touch on the "WHY" a bit:

- Dockerfiles are not reproducible. What builds today may fail tomorrow, or next month, or on odd numbered days, or whatever.

- Nix packages are reproducible. If it builds today, it will build on any machine, any time. Guaranteed.

- On distributions like Debian/Ubuntu/etc (essentially anything that isn't NixOS or inspired thereby, like Guix), when you install a package, and something goes wrong, your system can end up in an in-determinant state that requires human intervention to manually sort out how to unbreak things. Just google (or just recall the last time it's happened to you -- and if it hasn't happened to you, it eventually will) people desperately asking for help to unbreak their system after, say, Ubuntu's apt (or whatever) gets screwy and says it can't install/uninstall anything else because $INSERT_CRYPTIC_APT_BUZZWORDS_HERE. Oh, and add to that the compounding effects of the fact that any package can include raw bash commands as post-(un)install steps -- that's just asking for packages to fail to (un)install cleanly depending on what else the user does/doesn't have installed at that point in time.

- On NixOS (and similar) the system state is guaranteed to either update fully or not at all -- as a consequence of its design, none of the chaos mentioned can happen. It's not that it's unlikely. I literally mean it simply can't happen. System updates are guaranteed all or nothing. Also, if you find out that some configuration change isn't what you wanted, you can roll back in instant (that's not hyperbole -- rolling back entails no slow error prone file copying/writing/etc, literally just one symlink change, and "poof" you're on the previous version).

In a nutshell, from top to bottom of the stack, the selling point is: imagine a world where "oops -- that failed in production, but... well, it worked on my machine, I promise!" simply can't happen. Imagine the time savings, and the reduction in anxiety if you knew that, if you could successfully build and/or deploy a particular piece of software once locally/in-staging/etc, you have full confidence that you can repeat that on any machine at any time.

Admittedly, if you enjoy those struggles with conventional non-Nix(OS) setups, there isn't much value to Nix.

It's state at a different level than most configuration management software deals with. When Puppet reasons about the state of an existing system, it asks things like 'what's on the PATH right now?'. NixOS doesn't— it reconstructs the directory tree under /run/current-system/sw/bin without asking that. Same with 'modification' of files that live under /etc. What you and the GP write here is nuance worth admitting, but I wanted to highlight what people are getting at when they say Nix's approach is 'stateless'. What they mean is that Nix's approach to state is really qualitatively different, and hands state off to be managed by a lower level part of the system as much as possible.

> It is definitely possible with Nix to use the precise versions of what's in your requirements.txt, but I'm not sure if the Nixpkgs maintainers would allow all that extra duplication upstream.

They do for end user applications, but not for Python libraries. The libraries in Nixpkgs are expected to be interoperable, which requires converging certain versions because otherwise transitive dependencies on varying library versions mean that libraries used together are subject to serious, mysterious bugs. But applications packaged in Nixpkgs can pull in an exact set of libraries of their own if that's what it takes for them to run reliably.

It's not even really about static compilation. NixOS (and many Linux distros) include tons of dynamically linked C applications that just do a way, way better job of compatibility. Imagine if GNU grep were as fussy about only being built against 1 version of glibc as many Python libraries and applications seem to be about their dependencies.

Idk about a blog post, but he did a talk along those lines at the most recent NixCon: https://youtube.com/watch?v=asc1D5yPZhQ&

I'm taking that approach with the package I've been working on, which has a somewhat pathological (by Nix standards) Gradle build which does things like

  - manually download a copy of Elastic search outside of the normal Java dependencies scheme
  - run NPM to fetch remote libraries to build web assets at build time
  - *also* run Yarn, for some reason
  - use Git at build time

The ways it does all of these things are actually fairly thoughtful (for example, it does checksum the artifacts it manually grabs at build time to verify their contents), but they don't play nice with running builds in offline mode or under a user that has no $HOME. But it's one of those freeform 'my build tool configuration is a weird DSL in an imperative, general purpose, Turing-complete language' situations, and I'm not very familiar with either the language (Groovy) or the DSL. So it's a lot of quirks to cope with.

I've made quite a bit of progress in building it from source by making a few small patches and eventually disabling the sandbox for now, but it's still dying on a weird test failure for reasons I don't yet understand. At this point I'm just back to munging the binaries provided by upstream because I was mostly building from source to learn about the project and how it's distributed/deployed anyway.

I messed a bit with gradle2nix for a better-behaved, old school FOD-based build with Gradle in offline mode, but that was pretty brittle as gradle2nix is unmaintained, and due to some design limitations it couldn't actually capture all dependencies. I'm kinda interested in working out something better but on the other hand, this is a third-party package and I don't myself use Gradle or Groovy for any kind of development, so mastering Gradle's quirks and wrangling it into the Nix sandbox for this package is more of a yak shave than a practical skills investment for me.