Live data from Hacker News

Nix is the ultimate DevOps toolkit

tech.channable.com

41–50 of 243 posts

Re: Nix is the ultimate DevOps toolkit

#41
A lot of this sounds familiar to me— I maintain a hodgepodge build tool/pipeline at my org whose current output is a monolithic mega-deb file which is becoming unwieldy on several fronts (storage, transfer, compression time). I'm really interested in the nix philosophy of separate paths and versioning by hash rather than number, but unfortunately my needs are quite specific, and not having actual experienced nix experts around is a barrier. The temptation right now is to roll our own solution which generates nix-style hash debs that otherwise cooperate with the larger Debian ecosystem we're already very familiar with, but I could probably be sold on giving nix itself a closer look if I could hear from folks (or get pointers to documentation) on a few key points:

- What is the story in nix for packaging untagged branches of software, or reasoning about "snapshots" where pools of unreleased repos/packages are able to be treated as a single versionable unit? (think not just a nightly build, but builds corresponding to unmerged feature branches, etc)

- Does the nix hash account for dependencies only changing the hash for ABI-impacting changes such as when a header file changes? Or does it change dependent hashes always? Or never?

- I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?

- How much pain is it to roll a nix package "by hand" (basically with the dpkg-deb equivalent tool rather than the dpkg-buildpackage equivalent tool)?

- Nix isn't supported in Artifactory (RTFACT-19998 has been open since 2019). Nominally, I can use the dumb WebDAV option, but is that going to affect my user experience and/or will it be a maintenance headache?

- What is the apt/nix interop story? I would likely need it to be bidirectional, so that my nix workspace could depend on system debs that I don't want to port over, but also potentially have "gateway" debs which able to do the opposite, of depending on the nix workspace from a deb shim, and installing/updating it in the postinst.

I've researched all of these in the past at various points and had a lot of trouble finding clear, straight answers on them. I'd love to hear from other users or nix experts, though, who could help calm our fears on this.

Re: Nix is the ultimate DevOps toolkit

#44
post #32

Earlier quoted context omitted.

What kind of tutorial would you like to read?

My dream tutorial shows how to move from what I have to the new thing, because that teaches me what new ways of thinking I need to adopt. Like right now at work we have a small team making a website with docker-compose to specify the dev environment, pip-tools to pin python dependencies, and saltstack to deploy new state to bare metal in production. How exactly, command by command, would nix solve the same problems?…

> How exactly, command by command, would nix solve the same problems?

This might be problematic, because to understand these commands, you would have to understand Nix. It's not just a tool that replaces your existing tool, it's a total workflow change, and without understanding what's happening under the hood, you can't really appreciate how much it makes your life easier. I could show you a `nix-build -A deploy.somemachine`, but without understanding why it's called like that, and what exactly it does, and how I build most of that myself using nixpkgs tooling, you wouldn't really 'get' it, because you probably wouldn't understand how is that better than doing an `ansible-playbook -i foo.yml`, and I wouldn't be able to explain without explaining Nix/nixpkgs from scratch.

Nix, compared to other DevOps tools, is alien technology, and requires unlearning a lot of things, learning a whole new language and ecosystem, beginning to think in terms of that ecosystem and general programmability and functional programming, and then building on top of this to solve your problems. Nix doesn't immediately bring solutions to your problems, it brings you a new way of thinking about these problems and solving them in ways you didn't even consider before.

I know this is all handwavy, and that is unfortunate, but in my opinion unavoidable when dealing with radically different tooling. From my experience, the current 'classic' devops tooling (Docker, Terraform, ...) has reached a local maximum of usability. Getting out of that maximum requires a vastly different approach, and thus requires a much longer learning experience than what most people are used to. If you keep switching to different tools that behave how you expected them to, you're really just using the same thing as before, but with a different syntax.

Re: Nix is the ultimate DevOps toolkit

#45

I spend a few hours looking at nix about a year ago and found it impenetrable. I simply do not grok the syntax or what the functions do. I tried searching for the functions shown in the examples on the website to no avail. I searched packages, options, and even resorted to ctrl-f while clicking through the site "documentation"... It sounds awesome, but its in dire need of some better documentation if it wants to be a…

I still don't understand the language syntax after using it for several years, I just make it work through trial an error. The documentation is pretty unhelpful too. It's ultra ambiguous.

Recently my laptop died, and I just needed another one going in a hurry and I tried Xubuntu because it's the only distribution which had Wifi drivers that worked for a 12 year old Mac I had lying around without any trickery.

It's funny because Nix worked fine, but my Ubuntu installation just stays out of my way and lets me work too. I started to make me question whether all the added Nix complexity really added much value. Would investing in learning Nix really pay off over spending my time learning a new programming language with real world application?

I also think there is something to be said for the fact that, unless you team mates are using Nix,good luck using any of the standard tooling or scripts others will use. One of my favorites are shell scripts that start with `#!/bin/bash` never working and always having to patch them to use `#!/usr/bin/env bash` I understand the latter is the correct way to do things in 2021, but we all know it's not going to happen :)

Re: Nix is the ultimate DevOps toolkit

#46
The language and documentation is pretty awful, however I use NixOS (and therefore Nix) on my work desktop and still absolutely love it. The ability to create reproducible builds, test out tools without installing them permanently, and roll the OS back if you break something is just excellent.

For example I spent ages this morning trying to get a gRPC Rust environment working and battled the nix file for a good two hours but now it works I know it will always work and never have to do it again.

Re: Nix is the ultimate DevOps toolkit

#47

A lot of this sounds familiar to me— I maintain a hodgepodge build tool/pipeline at my org whose current output is a monolithic mega-deb file which is becoming unwieldy on several fronts (storage, transfer, compression time). I'm really interested in the nix philosophy of separate paths and versioning by hash rather than number, but unfortunately my needs are quite specific, and not having actual experienced nix expe…

> What is the story in nix for packaging untagged branches of software, or reasoning about "snapshots" where pools of unreleased repos/packages are able to be treated as a single versionable unit?

A Nix derivation (a.k.a. 'packaging unit') can be built from sources either fetched remotely, or from a local directory, or a combination of both; plus any other derivation that it depends on. Any version semantics come from how you implement them, abstracted and formalized in any way you want. Nix is a programming language, you are free to do what you want.

> Does the nix hash account for dependencies only changing the hash for ABI-impacting changes such as when a header file changes? Or does it change dependent hashes always? Or never?

By default, any change to any dependency will cause all dependent derivations to be rebuilt, and their hashes to also change. You can 'break' this by introducing stable ABI barriers preventing rebuilds and performing runtime loads of dependencies from .so (or whatever, executing some binaries from $PATH). This is for instance how my firefox can handle any OpenGL library that it can use at runtime, without having a firefox rebuilt for every possible GPU driver out there.

> I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?

You will have to generate nix source code, or importable nix source code, from that system, and pull that into your local nix source tree. All nix evaluation is done from purely defined locally available sources, or sources fetched from remote resources, but marked with some consistent hash.

> How much pain is it to roll a nix package "by hand" (basically with the dpkg-deb equivalent tool rather than the dpkg-buildpackage equivalent tool)?

Nix does not work this way. Nix derivations are a recipe on how to build a package, you have to specify that in one way or another so that Nix can perform the build for you, and reason about its status within a larger build graph. You can 'cheat' building derivations by eg. importing binary builds and only patching them up to work under Nix(OS), but generally you shouldn't.

> Nix isn't supported in Artifactory (RTFACT-19998 has been open since 2019). Nominally, I can use the dumb WebDAV option, but is that going to affect my user experience and/or will it be a maintenance headache?

You will need to run a Nix cache, that CI will push into and other consumers (all having a checkout of your nix definitions) will pull from instead of building everything from scratch themselves.

> What is the apt/nix interop story? I would likely need it to be bidirectional, so that my nix workspace could depend on system debs that I don't want to port over, but also potentially have "gateway" debs which able to do the opposite, of depending on the nix workspace from a deb shim, and installing/updating it in the postinst.

Not aware of anything out of the box. Nix builds only run on Nix (as they all need dependencies from /nix/store/..., and that is populated by Nix itself), so don't expect to easily run them on Debian as normal dpkgs without some disgusting home-rolled hacks. You could technically consume binary .debs from Debian within nix derivations (see previous point), but that's ugly. Generally, just install Nix on all your hosts.

Re: Nix is the ultimate DevOps toolkit

#48
post #46

The language and documentation is pretty awful, however I use NixOS (and therefore Nix) on my work desktop and still absolutely love it. The ability to create reproducible builds, test out tools without installing them permanently, and roll the OS back if you break something is just excellent. For example I spent ages this morning trying to get a gRPC Rust environment working and battled the nix file for a good two h…

Do you truly understand how it works though? I was similar. I loved my setup and it did mostly just "work" but I found it unnerving that eventually when it does break, I wouldn't know how to fix it.
Post reply on HN