Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nix is the ultimate DevOps toolkit

tech.channable.com

Post reply on HN
Print view View original

141–150 of 243 posts

Re: Nix is the ultimate DevOps toolkit

#141

Post by chriswarbo »

georgyo wrote: post #130
chriswarbo wrote: post #104

Earlier quoted context omitted.

> Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time. For anything that doesn't fit a common nixpkgs function (like `buildPyt…

That is a gross oversimplification. An javascript front end can have thousands of dependencies, and the nix build environment does not have network access unless it's computing a fixed output. To do that manually would be a nightmare, so you must automated either with tools you write or tools someone else has written. Then you still have to figure out how to combine it all together. Also on allowing network access in…

> That is a gross oversimplification

It's really not. I've used this successfully for Javascript projects (running npm and yarn), Python projects (running pip install), Scala projects (running sbt), Haskell projects (running cabal), Racket projects (running raco), Go projects (running go get), etc.

It's pretty much like writing a Makefile, except we have to specify the buildInputs, and there are occasional annoyances like tools trying to write to $HOME (requiring an `export HOME="$PWD"` line at the top of the script).

> the nix build environment does not have network access unless it's computing a fixed output

Yes it does, if we avoid the sandbox. This used to be done by specifying a `__noChroot = true;` attribute; I don't know if that's changed in more recent Nix versions.

> Also on allowing network access in the sandbox requires elevated privileges

Sandboxing is a sensible default, but for practicality I usually set it to "relaxed" on NixOS and disable it entirely on macOS (since that often uses stuff from the host system).

> which means you cannot do that method on environments you don't have full control.

Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines)

> And it would create builds that are likely not reproducible.

Yes, that's why I mentioned we might want to factor out the input-fetching at some point.

> Then you still have to figure out how to combine it all together.

That has nothing to do with Nix. As long as a project has a build script, or a Makefile, or a list of commands in a README, or whatever, then we can plug it into that template and see if it works. Are you complaining that the Nix package manager doesn't have specific documentation for how to build every random application ever written?

Re: Nix is the ultimate DevOps toolkit

#142

Post by skratlo »

Even though the article sounds like it's written by educated person, it puzzles me that the author:

> Installing packages with APT is not atomic: APTs updates files in-place one by one. Because we start Python processes many times per second, if one started during a deployment, it could import a mix of the old and new code. This regularly lead to puzzling bugs

Don't you stop all your services in preinstall and start them again in postinstall? Or, if you're packaging your python libs in deb pkgs, just stop the services before upgrading the system and start them again once you're done.

Re: Nix is the ultimate DevOps toolkit

#143

Post by epistasis »

Shoue wrote: post #59
jpe90 wrote: post #51

I tried Nix a few days ago. I set it up on an existing Arch install. I installed a couple of packages with "nix-env -i [package] and then tried to update them with "nix-env -u" as instructed in the official documentation: https://nixos.org/manual/nix/stable/#ch-basic-package-mgmt This ended up breaking the entire install. After a few hours of troubleshooting I found that the reason it broke was that it updated itself…

I wish Nix would just get rid of nix-env. It's not the way it's meant to be used, you should be using "nix run" or nix-shell for temporary usage, and home-manager for dotfiles and user dependencies. Using Nix like an imperative package manager is not really an improvement over existing ones, the declarative bit is where it truly shines.

As somebody who wants to try out nix, but has almost no knowledge of it, I would like to find resources that show the "right" way, or perhaps the "best" way to use nix. Do you know of existing documentation that explains that, according to your opinion?

Re: Nix is the ultimate DevOps toolkit

#144

Post by 2mol »

epistasis wrote: post #143
Shoue wrote: post #59

Earlier quoted context omitted.

I wish Nix would just get rid of nix-env. It's not the way it's meant to be used, you should be using "nix run" or nix-shell for temporary usage, and home-manager for dotfiles and user dependencies. Using Nix like an imperative package manager is not really an improvement over existing ones, the declarative bit is where it truly shines.

As somebody who wants to try out nix, but has almost no knowledge of it, I would like to find resources that show the "right" way, or perhaps the "best" way to use nix. Do you know of existing documentation that explains that, according to your opinion?

I think https://nix.dev is what you are looking for. It's a really great resource (and no need to go all in and do _everything_ in nix)

Re: Nix is the ultimate DevOps toolkit

#145

Post by Ericson2314 »

jpe90 wrote: post #51

I tried Nix a few days ago. I set it up on an existing Arch install. I installed a couple of packages with "nix-env -i [package] and then tried to update them with "nix-env -u" as instructed in the official documentation: https://nixos.org/manual/nix/stable/#ch-basic-package-mgmt This ended up breaking the entire install. After a few hours of troubleshooting I found that the reason it broke was that it updated itself…

Well, that's bad. `-u` is bad, and that the cross compilation suffix somehow went after the version is also bad.

Re: Nix is the ultimate DevOps toolkit

#146

Post by mikepurvis »

georgyo wrote: post #57
mikepurvis wrote: post #41

A lot of this sounds familiar to me— I maintain a hodgepodge build tool/pipeline at my org whose current output is a monolithic mega-deb file which is becoming unwieldy on several fronts (storage, transfer, compression time). I'm really interested in the nix philosophy of separate paths and versioning by hash rather than number, but unfortunately my needs are quite specific, and not having actual experienced nix expe…

That's a lot of questions - What is the story in nix for packaging untagged branches of software If you want to do unreleased software, you would create overlays that added them. You're overlays most likely would not need to define the build, just create an override of the one in nixpkgs with a new source: https://discourse.nixos.org/t/how-to-override-package-versio... - Does the nix hash account for dependencies onl…

> Nix only writes new files to a remote store. As such nix caches can be served by anything that can serve files. I personally upload to S3 and have artifactory use that as a remote.

What about the maintenance side, though, like being able to clean/reap old builds from the cache, reason about which ones are part of important/supported chains vs throwaway builds from merge pipelines, etc?

Re: Nix is the ultimate DevOps toolkit

#147

Post by Ericson2314 »

Shoue wrote: post #59
jpe90 wrote: post #51

I tried Nix a few days ago. I set it up on an existing Arch install. I installed a couple of packages with "nix-env -i [package] and then tried to update them with "nix-env -u" as instructed in the official documentation: https://nixos.org/manual/nix/stable/#ch-basic-package-mgmt This ended up breaking the entire install. After a few hours of troubleshooting I found that the reason it broke was that it updated itself…

I wish Nix would just get rid of nix-env. It's not the way it's meant to be used, you should be using "nix run" or nix-shell for temporary usage, and home-manager for dotfiles and user dependencies. Using Nix like an imperative package manager is not really an improvement over existing ones, the declarative bit is where it truly shines.

At the very least, get rid of `-u`. Some parts of nix-env are used underneath the hood for NixOS and home-manager I beleive, but not that.

Re: Nix is the ultimate DevOps toolkit

#148

Post by chriswarbo »

georgyo wrote: post #131
chriswarbo wrote: post #98

Earlier quoted context omitted.

> the only time a semicolon is used is to terminate an assignment (=) I find it easier to think of "attribute sets" like `{ foo = bar; }` more like a JSON objects (e.g. `{ "foo": bar }`) rather than "assignment". Nix does have a `let` syntax, which probably looks more like assignment, but I've never used it in the 7 years I've been using Nix. I find `with myAttrs; myExpr` to be far better (except for the WontFix issu…

> I find `with myAttrs; myExpr` To make it clearer, he is saying that he does with { foo = "bar"; }; instead of let foo = "bar"; in The differences here so other people can follow along. 1. with will not shadow variables defined higher up. This can be either desirable or very confusing. 2. let is recursive by default, meaning you can reference other variables in that block. This behavior can be mimicked with `with re…

Yep. The advantage of `with` is that we're dealing with first-class values (attribute sets, like `{ foo = "bar"; }`), rather than an implicit environment/context. My rationale is at https://github.com/NixOS/nix/issues/1361#issuecomment-390420...

The thing about shadowing is that `with` will not shadow anything that's in the implicit context, which includes things defined by `let` (hence why I avoid it) and function arguments (unavoidable). We can shadow other `with` bindings as much as we like, e.g.

    with { foo = 1; };
    with { foo = 2; };
    with { foo = 3; };
    with { bar = with { foo = 4; }; foo; };
    [ foo bar ]
This evaluates to [ 3 4 ], since later bindings to `foo` will shadow/override earlier ones.

This doesn't work for bindings that are in the implicit environment/context, for example:

    with {
      f = foo: with { foo = 42; }; foo;
    };
    f 123
This evaluates to 123, since the binding `with { foo = 42; }; ...` doesn't shadow/override the argument `foo`.

The rationale for this behaviour is to avoid changes made to an attrset from overriding explicit arguments. For example:

    foo:
      with bar;
      x + foo
This is presumably meant to behave like `foo: bar.x + foo`, but a future change to `bar` might include a `bar.foo` attribute, in which case we don't want the above to change behaviour to `foo: bar.x + bar.foo`.

I understand this rationale, especially when using huge, constantly-changing attrsets like `with nixpkgs; ...`, but it's still annoying ;)

Re: Nix is the ultimate DevOps toolkit

#149

Post by fctorial »

nuker wrote: post #102
detaro wrote: post #70

Earlier quoted context omitted.

"cloud stuff" and "containers" (which people also do manage using Nix) are not requirements for something to be DevOps.

Try telling it to any recruiter lol

The problem "containers" solve does not exist in nix land.

If all you've got is a hammer...

Edit: removed "clouds"

Re: Nix is the ultimate DevOps toolkit

#150

Post by fctorial »

grandinj wrote: post #115

Nix sounds like SELinux. A great idea in theory, but in practice just too hard. There is a great deal of genius in coming up with these ideas. There is just as much genius in finding the 80-90% solution that incorporates most of the original, but in a workable way. Looking forward to seeing where stuff like Nix ends up.

You'd be surprised how well a nixos system works. Every other distro looks like a meme in comparison, for development work at least.
Post reply on HN