Live data from Hacker News

Nix is the ultimate DevOps toolkit

tech.channable.com

221–230 of 243 posts

Re: Nix is the ultimate DevOps toolkit

#221

As an aside, I've just started trying to move my laptop to NixOS this past week, so far it seems pretty nice. I have a Macbook as well, and I think I should be able to share the Nix stuff with that - which would be cool. As for the article, I hadn't actually considered using it as a replacement for Ansible etc - but I could see why it might work better. I could pin packages etc to the versions, and get the exact same…

Re: using nix in place of ansible/chef/puppet/salt etc.

One of things that makes it far superior to these solutions, is that while many of them try to be declarative it still an iterative hack to emulate it.

For example if you use your CM to install a package on the system, or create a file in /etc. If you remove that statement the file or package will still be there.

In NixOS it will be as if it was never there. The reason for it is that the existing CM work by modifying current OS, while NixOS rebuilds the entire OS every time you make a change (thanks to cacheing that operation doesn't take as long as it would normally do).

As for using things in production, I generally like things that are well established. So it took many years before I got the courage to propose it, but frankly I don't think it ever let me down. As I used it always act predictable, both the NixOS as well as build (as long as you pin down nixpkgs).

The only thing to worry about is if the outside environment demands a change you better know how to do it. For example application works fine, suddenly you have higher traffic and things are crapping out and maybe you need to change some settings or there was a bug in one of your dependency and a new version isn't in nixpkgs. You better know how to do those things.

Re: Nix is the ultimate DevOps toolkit

#222
post #144

Earlier quoted context omitted.

As somebody who wants to try out nix, but has almost no knowledge of it, I would like to find resources that show the "right" way, or perhaps the "best" way to use nix. Do you know of existing documentation that explains that, according to your opinion?

I think https://nix.dev is what you are looking for. It's a really great resource (and no need to go all in and do _everything_ in nix)

Ahh that's great, thank you. I've been running nixos on my laptop for a year but have never come across that somehow.

Re: Nix is the ultimate DevOps toolkit

#223
post #164

Earlier quoted context omitted.

If you try to make a Go package, for example, you end up having to provide a hash. To generate the hash `vendorSha256`, you have to run `nix-prefetch`, but that program is broken on MacOS. You can also stub the hash (e.g. with lib.fakeSha256 ). nix-build will fail with a hash mismatch and tell you the correct hash. I mentioned elsewhere that I spent a weekend working with the Nix discord to try to install VS Code wit…

> You can also stub the hash (e.g. with lib.fakeSha256). nix-build will fail with a hash mismatch and tell you the correct hash. Are you kidding me?

What do you mean? This is exactly how hash verification is supposed to work. You either supply the correct hash or the verification fails, telling you the actual and expected hashes. Works the same way on any package manager that verifies its sources.

Re: Nix is the ultimate DevOps toolkit

#224
post #105

Earlier quoted context omitted.

I switched to NixOS after tripping over my laptop's power chord during an update, which broke Linux in such horrifying ways that I didn't dare attempt recovery. I agree that I'm not sure the extra complexity is worth it for day to day use, but just the ability to avoid catastrophic failure and near instant setup on new machines is worth it I think.

Many times repairing those kinds of situations in Linux is as simple as: - boot linux livecd - mount your broken filesystem - bind mount the important bits from the live kernel (/proc) - chroot Like this: cd / mount -t ext2 /dev/sda1 /mnt #Here's your broken install mount -t proc proc /mnt/proc mount -t sysfs sys /mnt/sys mount -o bind /dev /mnt/dev chroot /mnt /bin/bash #boom you are in. Now do whatever you can to r…

>Now do whatever you can to repair the system. depends on what broke.

What needs to be done to fix the system may not be obvious and the fix may not be simple. Furthermore, the trial and error for fixing may leave a trail of new issues.

And that is why I use NixOS. I have to wrestle with it to get something working, but once I'm done I can guarantee it will keep working until the heat death of universe. Compared to that, Ubuntu and others let me get off the land fast but I never know when I'll crash or if I'll able to fly back from it.

Re: Nix is the ultimate DevOps toolkit

#225

Earlier quoted context omitted.

Organizing that mess should make every user a casual user, serious it not?

hmm? Genuinely unsure what you mean.

The word serious was meant to be should. My phone keyboard picked the wrong word, and I waited too long to edit.

Re: Nix is the ultimate DevOps toolkit

#227
post #203

Earlier quoted context omitted.

Though, on the other hand, the comment says a new CLI is close to release, so I guess people are trying to improve things. I don't want to be too critical, I love Nix. But the experimental new UI was already in development (and available) when I started using Nix in 2018. It has also changed quite a lot recently and is now very strongly tied to flakes. However, the flakes RFC was withdrawn [1]. Given that flakes are…

I believe it is only withdrawn for further work on it.

Yes. But since there was an initial RFC, I would expect there to be another RFC when flakes are ready.

Re: Nix is the ultimate DevOps toolkit

#228
post #32

Earlier quoted context omitted.

What kind of tutorial would you like to read?

My dream tutorial shows how to move from what I have to the new thing, because that teaches me what new ways of thinking I need to adopt. Like right now at work we have a small team making a website with docker-compose to specify the dev environment, pip-tools to pin python dependencies, and saltstack to deploy new state to bare metal in production. How exactly, command by command, would nix solve the same problems?…

So nix is kind of weird thing, because parts of it you can use to replace part of components but hen you can create a whole solution (and ultimately that works better). In that case it's like asking to write a tutorial for python.

To illustrate what I mean here is my history with nix (simplified, but it's meant to show the evolution):

So I started using nix as a reproductive dev environment. Initially using shell.nix to have all my tooling. This is great if you also have direnv installed, which basically automatically makes tools related to the project to appear.

Anyway I started then using various tools that would translate python project to nix initially was also a big fan of setup.py/cfg and pip-tools. I wasn't happy with either of them until I found poetry2nix. Poetry is also quite decent so now that's my favorite way to build a project.

So what poetry2nix does is translates on the fly pyproject.toml and poetry.lock file to nix expression. With it you can build project in nix you can also create a shell that has python with all packages available as described by poetry. So that's now quite decent reproducible environment.

I realized that to make the dev environment nice I'm adding a lot of nix boilerplate to every project, also as I'm learning new things and improving it it is getting hard to synchronize that boilerplate across other projects.

So I created a new repo where I put everything there, I also learned how to use modules system (which they created for NixOS) ultimately I made repo that you reference in default.nix and shell.nix in my project it loads project.nix file which describes information about my project. Allows me to set various aspect of the application, like what additional packages should be installed in dev shell (maybe I need a local postgresql installation). How the project will be deployed (if it is meant to run as AWS lambda, a serverless (npm application) is added and package.json/package-lock.json files are sourced. If it is a docker asks what binary should be executed in the container. Whether to compile using minified python (with missing some of core libraries) etc.

That ended to be some Nix code, but simplified things greatly, and makes things usable by people not familiar with Nix.

We use gitlab for CI/CD so I created a special gitlab-runner for nix builds. It actually is already built in in NixOS and only needs to be enabled, and they also show configuration in example. Basically how it works is that it stills spins a docker container for the build, but exposes /nix/store (read only) and the unix socket to trigger build. I also created an S3 bucket and configured it for storing caches of builds.

Having this runner lets me utilize one of major benefits of Nix which is caching (if all inputs (source files, architecture, interpreters etc) don't change then output will be the same). This is great, because if I only change source file, and don't change dependencies, the pipeline will only rebuild my app. If I work on a branch and branch passes all tests, I merge it and merge is just fast forward, then build job in master branch just takes couple seconds, because nothing change. If the branch wasn't rebased and actual merge was done and some files are now different the nix will rebuild things that were modified.

Deployment of my python app as a lambda ends up being just:

    nix-shell --run "sls deploy"
nix-shell takes care that the dev environment is created and inside of it invokes "sls deploy" command. This similarly utilizes caching (that persists between builds) so it is faster than what we normally used like loading a docker container and then running it.

This is quite decent and I'm still testing it on a single app, but it basically cut deployment time from 10 minutes to 5 minutes. And I know I can cut it further (serverless doesn't know about nix, but it understands poetry, so once again again it downloads all dependencies it needs, extracts them and bundles them into the lambda. I plan to populate its cache from the packages that nix stores when is running the build).

There is also possibility that I'm aware of, but haven't explored yet, but seems interesting. It might require some work. For example you could configure your local computer to use the machine that is used for CI/CD build to help with local builds. Similarly you could expose the cache as well to prevent unnecessary rebuilds. You probably shouldn't do that if the user isn't trustworthy, otherwise you might need to create some additional safety layer.

Now in your scenario you have bare bone servers to deploy. If those would run NixOS I believe the deployment step would be simply to deliver new configuration.nix referencing your application to them and also have those machines configured to use the same cache as you had with CI/CD so they won't have to rebuild everything again. There are several tools that make the process easier, for people who use public cloud there's terraform support, NixOps (last time I used it was the weakest part of Nix, but maybe it improved, it was great tool when I wanted to test everything a la hashicorp's vagrant)

Re: Nix is the ultimate DevOps toolkit

#229
post #177

Earlier quoted context omitted.

Or you could, you know, use any kind of backup/restore software, spend like 5 minutes to learn how to use it, and use those many hours invested in learning Nix to watch football and drink beer :-)

I feel like restoring a backup is a lot more disruptive than the update just not happening and you being back where you started automatically.

For a catastrophic event that maybe happens once per decade?

https://xkcd.com/1205/

Re: Nix is the ultimate DevOps toolkit

#230
post #7

I tried to learn before. But the docs made it incredibly hard. They went into excruciating detail of how it works. I still never made my own nix package. I’m wondering as a solo dev, who uses his personal machine, and two servers, if the hard work of learning it will pay off. But it definitely looks interesting.

What kind of tutorial would you like to read?

I don't think it fits the tutorials you're creating on nix.dev, but I would love a deep dive how to use module system that nixpkgs provides. What I mean is how to use it to create my own solution. For example if I would create own configuration file like flake.nix or configuration.nix how could I process it. I think I figured it out, but I'm still not certain if I'm using it correctly.
Post reply on HN