Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nixery – Docker images on the fly with Nix

nixery.dev

Post reply on HN
Print view View original

21–30 of 90 posts

Re: Nixery – Docker images on the fly with Nix

#21

Post by steeleduncan »

WJW wrote: post #19
hamandcheese wrote: post #7

Earlier quoted context omitted.

Do you actually use nix and have an experience to share?

Not OP, but the evangelism about "never having build problems anymore" does perplex me a bit. In the languages I have been programming in (Haskell, Ruby, some Python and elm and JS and Rust), I can't recall having any significant build problems in the last ~8 years or so anyway. What does everyone do that their build keeps breaking?

Those languages all have sensible package managers built in, nix adds more value with languages like C/C++, or when you have multiple binaries interacting.

For example, I had a C++ project break catastrophically when upgrading from Ubuntu 19.10 to 20.04. It built fine, but wouldn't boot. Undoubtedly the root cause was my fault, but I couldn't trace it and the timing was terrible, so I had a hack to build inside a 19.10 container. Nix would have saved me a lot of pain in that case by pinning the compiler or whatever dependency broke it.

Nix is also useful in that it pins the entire ecosystem in the case that your project isn't fully contained in one language. To take your Haskell example, if you use Stack it will pin every piece of Haskell code you might import, but I've had problems where an external tool changes its output, requiring changes to my tool. In that case nix is like Stack, but for your entire OS, building that tool in a reproducible manner.

Re: Nixery – Docker images on the fly with Nix

#22

Post by alias_neo »

BreakfastB0b wrote: post #18
nickysielicki wrote: post #12

I don’t know if it’s just because I hopped on the bandwagon in the past few months, but it really is starting to feel like Nix is gaining momentum. You can use it on Mac, WSL, Ubuntu multiuser, Docker, in an nspawn container, or through NixOS on a drive. You can build a livecd iso or a Raspberry Pi image from the same flake that you use for everything else. I work in embedded with Yocto every day, and I can’t help bu…

We recently adopted it at my company for managing local dev machines, project environments, and CI. It definitely has some warts, often the best documentation is “read the source code”, but man is it an awesome tool. I’ve switched all of my machines / servers over to it and I’ll never look back. Now I’m looking at my iPad and iPhone and wishing I could manage them through Nix too. I’d put it at a comparable difficult…

Though the documentation may be lacking, or difficult for beginners, I'd like to point out that I've found the community to be extremely helpful, patient and welcoming when asking questions and for help on Matrix.

Re: Nixery – Docker images on the fly with Nix

#23

Post by ananthakumaran »

WJW wrote: post #19
hamandcheese wrote: post #7

Earlier quoted context omitted.

Do you actually use nix and have an experience to share?

Not OP, but the evangelism about "never having build problems anymore" does perplex me a bit. In the languages I have been programming in (Haskell, Ruby, some Python and elm and JS and Rust), I can't recall having any significant build problems in the last ~8 years or so anyway. What does everyone do that their build keeps breaking?

Probably you are lucky to not run into the nokogiri compilation issue. For production codebases, usually, this is not an issue because the build breakages are fixed one at a time by upgrading gem, etc. This is a big issue for personal projects, I am no longer able to run many old projects (last commit made before 5 years), because they depend on older library version (like libglew), which is not supported by any recent distros.

Re: Nixery – Docker images on the fly with Nix

#24

Post by drewm1980 »

exdsq wrote: post #4

Oh great! More indirection. Now when I want to deploy my web app I can check my private nixery.dev deployment is properly configured in Nix to build my Docker images so I can deploy my containers to the cloud so someone can access me Rest API. And the cost of guaranteeing builds will probably work? Running your own nixery service, learning Nix, and learning Docker. I would love someone to do a cost-benefit analysis o…

Is the extra layer of indirection you don't like Docker or this nix -> Docker integration?

Compared to just installing all your nix packages in one Docker layer, this does introduce build complexity. But it's analogous to the complexity of a compiler for a static language... the thing that comes out is not more complex than what went in, so at least the complexity doesn't propagate. The images produced by this should be interchangeable with their single-layer counterparts; the caching will just be better when the code is rebuilt and re-distributed.

If you're all-in on nix, does the container ecosystem even add value? The author of this software thought so, at least when he posted in 2018: "Tying in to the schedulers, orchestration, and monitoring is very valuable"

Note, I have not used this; I'm just also frustrated by software development getting eaten by incidental complexity.

Re: Nixery – Docker images on the fly with Nix

#25

Post by drewm1980 »

WJW wrote: post #19
hamandcheese wrote: post #7

Earlier quoted context omitted.

Do you actually use nix and have an experience to share?

Not OP, but the evangelism about "never having build problems anymore" does perplex me a bit. In the languages I have been programming in (Haskell, Ruby, some Python and elm and JS and Rust), I can't recall having any significant build problems in the last ~8 years or so anyway. What does everyone do that their build keeps breaking?

Pull in a lot of transitive dependencies, without a system in place that automatically and strictly vendors or pins the versions of everything, and the probably of your build succeeding will converge to zero over time. Humans screw up semver all the time even when they're aware of hyrum's law and are doing their very best not to break user code.

I would say this starts becoming an issue when you're around 20-30 devs maintaining software that's a few years old.

All that's just for rebuilding when their are no changes to your code... pulling in security updates is a whole additional mess if your software is exposed to adversaries.

Re: Nixery – Docker images on the fly with Nix

#26

Post by chriswarbo »

exdsq wrote: post #4

Oh great! More indirection. Now when I want to deploy my web app I can check my private nixery.dev deployment is properly configured in Nix to build my Docker images so I can deploy my containers to the cloud so someone can access me Rest API. And the cost of guaranteeing builds will probably work? Running your own nixery service, learning Nix, and learning Docker. I would love someone to do a cost-benefit analysis o…

> I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash.

Nix does essentially the same job as Make. The differences are:

- Make embeds a shell code interpreter, whilst Nix just execs a binary; given its path, a list of args and a set of env vars. (Note that almost all Nix definitions use bash as their binary!)

- Make does meta-programming with a mixture of "automatic variables" ('$- Make relies on timestamps to figure out whether to re-use existing outputs; Nix relies on the hash of the definition (this works recursively, since hashes are included in filenames; hence changing a reference will alter all the hashes up the dependency tree).

- Make runs commands in the directory where 'make' was invoked, Nix runs commands in a temp folder (and optionally restricts network and filesystem access)

- Make runs commands with the same environment it was invoked with, Nix specifies the environment of commands in the build definition

Nix also has a killer feature that Make can't do, called "import from derivation". This lets us define a build process, like 'fetch this git repo', then import and use Nix definitions from its result. In comparison, Makefiles can't (reliably) fetch and import each other; e.g. my C project's Makefile can't fetch the GCC source tarball, and depend on its Makefile's "install" rule to provide a compiler.

My hypothesis is that this deficiency of Make is the reason for a whole bunch of unneeded complexity in the software world (e.g. "package managers", "OS distributions", "configuration managers", etc.)

From a practical point of view, Nix is almost always used as a wrapper on top of something else (Make/Ant/Maven/Cabal/etc.); but that's just because most projects benefit from those "ecosystems". Note that we could just as well wrap such Ant/Maven/Cabal/etc. projects in a layer of Make instead of Nix, but nobody does since it wouldn't give us any benefit ;)

If you're happy to ignore those "ecosystems" and just have a simple "bash + Make" project, you could instead have a simple "bash + Nix" project and avoid all the layers of Make/Ant/Maven/Cabal/etc. (as well as any Docker, Ansible, Apt/RPM, etc. that others might also decide to layer on top!)

Re: Nixery – Docker images on the fly with Nix

#27

Post by denvrede »

hamandcheese wrote: post #9

This is really cool, but I don’t know if I see the appeal for actual nix users — if you are a nix user and have it set up in CI, you can easily build docker images yourself using buildLayeredImage. And then if you aren’t a nix user, why would you use this? Installing packages with, say, apt, is decidedly not where my pains with docker have arose.

If I need an image with a specific set of tools it's cumbersome to build a whole workflow to build, store and maintain these images. Having a service that can receive a custom list of Nix packages and returns an image that I can instantly use would be really, really, really nice.

Re: Nixery – Docker images on the fly with Nix

#28

Post by xvector »

BreakfastB0b wrote: post #18
nickysielicki wrote: post #12

I don’t know if it’s just because I hopped on the bandwagon in the past few months, but it really is starting to feel like Nix is gaining momentum. You can use it on Mac, WSL, Ubuntu multiuser, Docker, in an nspawn container, or through NixOS on a drive. You can build a livecd iso or a Raspberry Pi image from the same flake that you use for everything else. I work in embedded with Yocto every day, and I can’t help bu…

We recently adopted it at my company for managing local dev machines, project environments, and CI. It definitely has some warts, often the best documentation is “read the source code”, but man is it an awesome tool. I’ve switched all of my machines / servers over to it and I’ll never look back. Now I’m looking at my iPad and iPhone and wishing I could manage them through Nix too. I’d put it at a comparable difficult…

How can one use Nix to manage project-specific dependencies that typically store Dotfiles in annoying locations like the home folder?

I am aware of home-manager but am not sure how (or if) it would work for per-project dot file management.

Re: Nixery – Docker images on the fly with Nix

#29

Post by civodul »

In the same spirit but in the form of a readily-usable command (rather than a service), 'guix pack' can produce application bundles in a reproducible fashion, in the Docker format as well as in other formats:

https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix...

Hopefully the smart layering strategy that Nixery uses will eventually make it into 'guix pack'!

Re: Nixery – Docker images on the fly with Nix

#30

Post by catern »

xvector wrote: post #28
BreakfastB0b wrote: post #18

Earlier quoted context omitted.

We recently adopted it at my company for managing local dev machines, project environments, and CI. It definitely has some warts, often the best documentation is “read the source code”, but man is it an awesome tool. I’ve switched all of my machines / servers over to it and I’ll never look back. Now I’m looking at my iPad and iPhone and wishing I could manage them through Nix too. I’d put it at a comparable difficult…

How can one use Nix to manage project-specific dependencies that typically store Dotfiles in annoying locations like the home folder? I am aware of home-manager but am not sure how (or if) it would work for per-project dot file management.

Can you say what specifically? Many language-specific package managers match your description and are completely handled by Nix, but maybe you're talking about something else?
Post reply on HN