Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd

Hello! This is Edgar and Robbie and we built nix-snapshotter. nix-snapshotter brings native understanding of Nix packages to containerd.

We built this because Nix is a great fit for making efficient containers. They don't need an OS because Nix captures all dependencies exactly. However, the current process of creating Nix images is subpar because one needs to transform Nix packages into a format that container runtimes understand.

Using nix-snapshotter, instead of downloading image layers, packages come directly from the Nix store. Packages can be fetched from a binary cache or built on the fly if necessary. All existing non-Nix images continue to be supported, and Nix layers can be interleaved with normal layers.

nix-snapshotter also provides a CRI image service, which allows Kubernetes to resolve image manifests from Nix directly too. This enables for the first time, fully declarative Kubernetes resources, all the way down to the image specification and its contents. With this, you can even run pure Nix images without a Docker Registry at all, if you wish.

We'd love for you to try it out, there is a one-liner for Nix users to boot a VM with everything pre-configured: https://github.com/pdtpartners/nix-snapshotter

Show HN: Nix Snapshotter – Native understanding of Nix packages for containerd
github.com

Hi flurie, Nixery exposes an API (in the form of an Docker registry) to dynamically build Nix-based images, but still tar & compresses Nix packages into layer tarballs. Since the image spec has a limit of 128 layers (due to overlayfs), a heuristic is used to put popular packages together. However, in practice there is still a large amount of duplication between images that share the same packages due to this heuristic based strategy. You also may deploy the same packages outside of containers but have to duplicate that same data (in a slightly different format) on a Docker Registry.

With `pkgs.nix-snapshotter.buildImage`, containerd natively understands Nix packages, so everything is pulled at package granularity without the layer limit. This means all container content is either already in your host nix store or fetched from your Nix binary cache. Think delta image pulls, which isn't possible with regular images since one bit difference will change the layer hash and thus duplicate.

That said, nothing stops Nixery from building nix-snapshotter images. Then we'll have an Docker Registry that dynamically builds native Nix images.

So if the package isn't in cache, then suddenly k8s node is building it from source?

To answer that, I think you’d first need to define what “putting my Nix config in e.g. a Dockerfile” means.

Do you mean using a Dockerfile to run a Nix package build, embedding the resulting files (and transitive dependencies) in the image file? If so, this objectively superior in several ways: 1) only one copy of any package exists on disk: the one in the Nix store, instead of multiple images having independent copies. 2) Related to this de-duplication, it will be faster to go from pristine host (virtual machine, dev machine, whatever) to having containers up and running, as you no longer have bulky images to transfer over the network. 3) I haven’t looked at the finer details, but I’m pretty sure this means that the host can build the required packages once, meaning better caching — instead of multiple Dockerfiles rebuilding the same packages, the host builds once and all dependent containers get use the host’s Nix store, again without any copying of files involved.

Edit:

From hinshun’s comment above, point #3 is incorrect — either the host’s Nix store would need a copy already present, or the package would need to be available in a remote store/cache. Even with that correction, you still avoid redundant package builds — either the package is available remotely/locally, or the image fetch fails — so things are still only ever built once :)

So that means that it needs the host to run nix, right? I suppose that could be useful, but if I'm running nix-built images as containers it's usually because I want to run somewhere that's not "nix native".